[INFO] cloning repository https://github.com/istio/ztunnel [INFO] running `Command { std: "git" "-c" "credential.helper=" "-c" "credential.helper=/workspace/cargo-home/bin/git-credential-null" "clone" "--bare" "https://github.com/istio/ztunnel" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fistio%2Fztunnel", kill_on_drop: false }` [INFO] [stderr] Cloning into bare repository '/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fistio%2Fztunnel'... [INFO] running `Command { std: "git" "rev-parse" "HEAD", kill_on_drop: false }` [INFO] [stdout] 8fda410eb0a515e54f336d14eddb450abea8073d [INFO] testing istio/ztunnel against master#d933cf483edf1605142ac6899ff32536c0ad8b22 for pr-150933 [INFO] running `Command { std: "git" "clone" "/workspace/cache/git-repos/https%3A%2F%2Fgithub.com%2Fistio%2Fztunnel" "/workspace/builds/worker-4-tc1/source", kill_on_drop: false }` [INFO] [stderr] Cloning into '/workspace/builds/worker-4-tc1/source'... [INFO] [stderr] done. [INFO] removed /workspace/builds/worker-4-tc1/source/.cargo/config.toml [INFO] started tweaking git repo https://github.com/istio/ztunnel [INFO] finished tweaking git repo https://github.com/istio/ztunnel [INFO] tweaked toml for git repo https://github.com/istio/ztunnel written to /workspace/builds/worker-4-tc1/source/Cargo.toml [INFO] validating manifest of git repo https://github.com/istio/ztunnel on toolchain d933cf483edf1605142ac6899ff32536c0ad8b22 [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+d933cf483edf1605142ac6899ff32536c0ad8b22" "metadata" "--manifest-path" "Cargo.toml" "--no-deps", kill_on_drop: false }` [INFO] crate git repo https://github.com/istio/ztunnel already has a lockfile, it will not be regenerated [INFO] running `Command { std: CARGO_HOME="/workspace/cargo-home" RUSTUP_HOME="/workspace/rustup-home" "/workspace/cargo-home/bin/cargo" "+d933cf483edf1605142ac6899ff32536c0ad8b22" "fetch" "--manifest-path" "Cargo.toml", kill_on_drop: false }` [INFO] [stderr] Updating crates.io index [INFO] [stderr] Updating git repository `https://github.com/janrueth/boring-rustls-provider` [INFO] [stderr] Downloading crates ... [INFO] [stderr] Downloaded tonic-prost v0.14.2 [INFO] [stderr] Downloaded prost-types v0.14.1 [INFO] [stderr] Downloaded tonic-prost-build v0.14.2 [INFO] [stderr] Downloaded tracing-serde v0.2.0 [INFO] [stderr] Downloaded tonic-build v0.14.2 [INFO] [stderr] Downloaded prost-derive v0.14.1 [INFO] [stderr] Downloaded prost v0.14.1 [INFO] [stderr] Downloaded equator v0.4.2 [INFO] [stderr] Downloaded dtor-proc-macro v0.0.6 [INFO] [stderr] Downloaded enum-ordinalize v4.3.2 [INFO] [stderr] Downloaded is-terminal v0.4.17 [INFO] [stderr] Downloaded fslock v0.2.1 [INFO] [stderr] Downloaded ctor-proc-macro v0.0.6 [INFO] [stderr] Downloaded prometheus-parse v0.2.5 [INFO] [stderr] Downloaded split-iter v0.1.0 [INFO] [stderr] Downloaded dtor v0.1.1 [INFO] [stderr] Downloaded mappings v0.7.1 [INFO] [stderr] Downloaded enum-ordinalize-derive v4.3.2 [INFO] [stderr] Downloaded arcstr v1.2.0 [INFO] [stderr] Downloaded findshlibs v0.10.2 [INFO] [stderr] Downloaded educe v0.6.0 [INFO] [stderr] Downloaded hickory-client v0.25.2 [INFO] [stderr] Downloaded cpp_demangle v0.4.5 [INFO] [stderr] Downloaded x509-parser v0.18.0 [INFO] [stderr] Downloaded flurry v0.5.2 [INFO] [stderr] Downloaded duration-str v0.17.0 [INFO] [stderr] Downloaded symbolic-demangle v12.16.3 [INFO] [stderr] Downloaded hickory-server v0.25.2 [INFO] [stderr] Downloaded prometheus-client v0.24.0 [INFO] [stderr] Downloaded boring v4.19.0 [INFO] [stderr] Downloaded aws-lc-rs v1.15.0 [INFO] [stderr] Downloaded rcgen v0.14.5 [INFO] [stderr] Downloaded neli v0.6.5 [INFO] [stderr] Downloaded ppp v2.3.0 [INFO] [stderr] Downloaded moka v0.12.11 [INFO] [stderr] Downloaded protobuf-parse v3.7.2 [INFO] [stderr] Downloaded pprof v0.15.0 [INFO] [stderr] Downloaded seize v0.3.3 [INFO] [stderr] Downloaded prefix-trie v0.7.0 [INFO] [stderr] Downloaded protobuf-codegen v3.7.2 [INFO] [stderr] Downloaded symbolic-common v12.16.3 [INFO] [stderr] Downloaded rustls-openssl v0.3.0 [INFO] [stderr] Downloaded tls-listener v0.11.0 [INFO] [stderr] Downloaded tonic v0.14.2 [INFO] [stderr] Downloaded resolv-conf v0.7.5 [INFO] [stderr] Downloaded jemalloc_pprof v0.8.1 [INFO] [stderr] Downloaded notify-debouncer-full v0.3.2 [INFO] [stderr] Downloaded local-ip-address v0.6.5 [INFO] [stderr] Downloaded debugid v0.8.0 [INFO] [stderr] Downloaded tikv-jemallocator v0.6.1 [INFO] [stderr] Downloaded pprof_util v0.8.1 [INFO] [stderr] Downloaded tikv-jemalloc-ctl v0.6.1 [INFO] [stderr] Downloaded keyed_priority_queue v0.4.2 [INFO] [stderr] Downloaded pingora-pool v0.6.0 [INFO] [stderr] Downloaded equator-macro v0.4.2 [INFO] [stderr] Downloaded prometheus-client-derive-encode v0.5.0 [INFO] [stderr] Downloaded neli-proc-macros v0.1.4 [INFO] [stderr] Downloaded netns-rs v0.1.0 [INFO] [stderr] Downloaded file-id v0.2.3 [INFO] [stderr] Downloaded lru v0.14.0 [INFO] [stderr] Downloaded pingora-timeout v0.6.0 [INFO] [stderr] Downloaded ctor v0.5.0 [INFO] [stderr] Downloaded aligned-vec v0.6.4 [INFO] [stderr] Downloaded textnonce v1.0.0 [INFO] [stderr] Downloaded prost-build v0.14.1 [INFO] [stderr] Downloaded tikv-jemalloc-sys v0.6.1+5.3.0-1-ge13ca993e8ccb9ba9847cc330696e02839f328f7 [INFO] [stderr] Downloaded boring-sys v4.19.0 [INFO] [stderr] Downloaded aws-lc-sys v0.33.0 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:cfa559dde23f44738f9990739aaa3487d8df3af37eb5df0572f7cf76004a604a" "/opt/rustwide/cargo-home/bin/cargo" "+d933cf483edf1605142ac6899ff32536c0ad8b22" "metadata" "--no-deps" "--format-version=1", kill_on_drop: false }` [INFO] [stdout] 05d2fc7dc465bc3116ad205e4d4336fad751d9ba2530afdfdcc1b1d350f7f9c5 [INFO] running `Command { std: "docker" "start" "-a" "05d2fc7dc465bc3116ad205e4d4336fad751d9ba2530afdfdcc1b1d350f7f9c5", kill_on_drop: false }` [INFO] running `Command { std: "docker" "inspect" "05d2fc7dc465bc3116ad205e4d4336fad751d9ba2530afdfdcc1b1d350f7f9c5", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "05d2fc7dc465bc3116ad205e4d4336fad751d9ba2530afdfdcc1b1d350f7f9c5", kill_on_drop: false }` [INFO] [stdout] 05d2fc7dc465bc3116ad205e4d4336fad751d9ba2530afdfdcc1b1d350f7f9c5 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:cfa559dde23f44738f9990739aaa3487d8df3af37eb5df0572f7cf76004a604a" "/opt/rustwide/cargo-home/bin/cargo" "+d933cf483edf1605142ac6899ff32536c0ad8b22" "build" "--frozen" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] 3c7ec97e1587cea6020fd03f4a177ba076bcb0550c8e6b49072ff019d58deb02 [INFO] running `Command { std: "docker" "start" "-a" "3c7ec97e1587cea6020fd03f4a177ba076bcb0550c8e6b49072ff019d58deb02", kill_on_drop: false }` [INFO] [stderr] Compiling proc-macro2 v1.0.103 [INFO] [stderr] Compiling libc v0.2.177 [INFO] [stderr] Compiling portable-atomic v1.11.1 [INFO] [stderr] Compiling critical-section v1.2.0 [INFO] [stderr] Compiling zerocopy v0.8.27 [INFO] [stderr] Compiling find-msvc-tools v0.1.5 [INFO] [stderr] Compiling stable_deref_trait v1.2.1 [INFO] [stderr] Compiling num-traits v0.2.19 [INFO] [stderr] Compiling hashbrown v0.16.0 [INFO] [stderr] Compiling either v1.15.0 [INFO] [stderr] Compiling crossbeam-utils v0.8.21 [INFO] [stderr] Compiling getrandom v0.3.4 [INFO] [stderr] Compiling once_cell v1.21.3 [INFO] [stderr] Compiling bitflags v2.10.0 [INFO] [stderr] Compiling rustix v1.1.2 [INFO] [stderr] Compiling thiserror v2.0.17 [INFO] [stderr] Compiling linux-raw-sys v0.11.0 [INFO] [stderr] Compiling fs_extra v1.3.0 [INFO] [stderr] Compiling fastrand v2.3.0 [INFO] [stderr] Compiling dunce v1.0.5 [INFO] [stderr] Compiling num-conv v0.2.0 [INFO] [stderr] Compiling time-core v0.1.8 [INFO] [stderr] Compiling log v0.4.28 [INFO] [stderr] Compiling aho-corasick v1.1.4 [INFO] [stderr] Compiling deranged v0.5.5 [INFO] [stderr] Compiling time-macros v0.2.27 [INFO] [stderr] Compiling itertools v0.14.0 [INFO] [stderr] Compiling icu_properties_data v2.1.1 [INFO] [stderr] Compiling regex-syntax v0.8.8 [INFO] [stderr] Compiling protobuf v3.7.2 [INFO] [stderr] Compiling indexmap v2.12.0 [INFO] [stderr] Compiling aws-lc-rs v1.15.0 [INFO] [stderr] Compiling rustls-pki-types v1.13.0 [INFO] [stderr] Compiling crossbeam-epoch v0.9.18 [INFO] [stderr] Compiling prettyplease v0.2.37 [INFO] [stderr] Compiling quote v1.0.42 [INFO] [stderr] Compiling syn v2.0.110 [INFO] [stderr] Compiling serde_json v1.0.145 [INFO] [stderr] Compiling tracing-core v0.1.36 [INFO] [stderr] Compiling crunchy v0.2.4 [INFO] [stderr] Compiling rustix v0.38.44 [INFO] [stderr] Compiling crossbeam-channel v0.5.15 [INFO] [stderr] Compiling tiny-keccak v2.0.2 [INFO] [stderr] Compiling minimal-lexical v0.2.1 [INFO] [stderr] Compiling bytes v1.11.1 [INFO] [stderr] Compiling nom v7.1.3 [INFO] [stderr] Compiling jobserver v0.1.34 [INFO] [stderr] Compiling time v0.3.47 [INFO] [stderr] Compiling getrandom v0.1.16 [INFO] [stderr] Compiling cc v1.2.46 [INFO] [stderr] Compiling form_urlencoded v1.2.2 [INFO] [stderr] Compiling rayon-core v1.13.0 [INFO] [stderr] Compiling parking_lot_core v0.9.12 [INFO] [stderr] Compiling mio v1.1.0 [INFO] [stderr] Compiling signal-hook-registry v1.4.6 [INFO] [stderr] Compiling socket2 v0.6.1 [INFO] [stderr] Compiling parking_lot v0.12.5 [INFO] [stderr] Compiling regex-automata v0.4.13 [INFO] [stderr] Compiling cmake v0.1.54 [INFO] [stderr] Compiling uuid v1.18.1 [INFO] [stderr] Compiling rand_core v0.9.3 [INFO] [stderr] Compiling tempfile v3.23.0 [INFO] [stderr] Compiling getrandom v0.2.16 [INFO] [stderr] Compiling which v4.4.2 [INFO] [stderr] Compiling fixedbitset v0.5.7 [INFO] [stderr] Compiling petgraph v0.7.1 [INFO] [stderr] Compiling crossbeam-deque v0.8.6 [INFO] [stderr] Compiling rusticata-macros v4.1.0 [INFO] [stderr] Compiling aws-lc-sys v0.33.0 [INFO] [stderr] Compiling ring v0.17.14 [INFO] [stderr] Compiling memoffset v0.6.5 [INFO] [stderr] Compiling thread_local v1.1.9 [INFO] [stderr] Compiling moka v0.12.11 [INFO] [stderr] Compiling cpp_demangle v0.4.5 [INFO] [stderr] Compiling ciborium-io v0.2.2 [INFO] [stderr] Compiling plotters-backend v0.3.7 [INFO] [stderr] Compiling anstyle v1.0.13 [INFO] [stderr] Compiling object v0.37.3 [INFO] [stderr] Compiling rustls v0.23.35 [INFO] [stderr] Compiling clap_lex v0.7.6 [INFO] [stderr] Compiling multimap v0.10.1 [INFO] [stderr] Compiling ahash v0.8.12 [INFO] [stderr] Compiling const-random-macro v0.1.16 [INFO] [stderr] Compiling findshlibs v0.10.2 [INFO] [stderr] Compiling rand_core v0.5.1 [INFO] [stderr] Compiling rand_core v0.6.4 [INFO] [stderr] Compiling debugid v0.8.0 [INFO] [stderr] Compiling inotify-sys v0.1.5 [INFO] [stderr] Compiling clap_builder v4.5.51 [INFO] [stderr] Compiling memmap2 v0.9.9 [INFO] [stderr] Compiling plotters-svg v0.3.7 [INFO] [stderr] Compiling num-integer v0.1.46 [INFO] [stderr] Compiling itertools v0.10.5 [INFO] [stderr] Compiling memoffset v0.9.1 [INFO] [stderr] Compiling gimli v0.32.3 [INFO] [stderr] Compiling adler2 v2.0.1 [INFO] [stderr] Compiling rustc-demangle v0.1.26 [INFO] [stderr] Compiling semver v1.0.27 [INFO] [stderr] Compiling tagptr v0.2.0 [INFO] [stderr] Compiling cfg_aliases v0.2.1 [INFO] [stderr] Compiling symbolic-demangle v12.16.3 [INFO] [stderr] Compiling rust_decimal v1.39.0 [INFO] [stderr] Compiling cast v0.3.0 [INFO] [stderr] Compiling oid-registry v0.8.1 [INFO] [stderr] Compiling rustc_version v0.4.1 [INFO] [stderr] Compiling num-bigint v0.4.6 [INFO] [stderr] Compiling chrono v0.4.42 [INFO] [stderr] Compiling nix v0.30.1 [INFO] [stderr] Compiling rayon v1.11.0 [INFO] [stderr] Compiling miniz_oxide v0.8.9 [INFO] [stderr] Compiling symbolic-common v12.16.3 [INFO] [stderr] Compiling plotters v0.3.7 [INFO] [stderr] Compiling regex v1.12.2 [INFO] [stderr] Compiling inotify v0.9.6 [INFO] [stderr] Compiling const-random v0.1.18 [INFO] [stderr] Compiling sharded-slab v0.1.7 [INFO] [stderr] Compiling criterion-plot v0.5.0 [INFO] [stderr] Compiling mio v0.8.11 [INFO] [stderr] Compiling clap v4.5.51 [INFO] [stderr] Compiling matchers v0.2.0 [INFO] [stderr] Compiling is-terminal v0.4.17 [INFO] [stderr] Compiling filetime v0.2.26 [INFO] [stderr] Compiling tracing-log v0.2.0 [INFO] [stderr] Compiling addr2line v0.25.1 [INFO] [stderr] Compiling nibble_vec v0.1.0 [INFO] [stderr] Compiling resolv-conf v0.7.5 [INFO] [stderr] Compiling nu-ansi-term v0.50.3 [INFO] [stderr] Compiling anes v0.1.6 [INFO] [stderr] Compiling prometheus-client v0.24.0 [INFO] [stderr] Compiling oorandom v11.1.5 [INFO] [stderr] Compiling endian-type v0.1.2 [INFO] [stderr] Compiling arrayvec v0.7.6 [INFO] [stderr] Compiling hdrhistogram v7.5.4 [INFO] [stderr] Compiling synstructure v0.13.2 [INFO] [stderr] Compiling radix_trie v0.2.1 [INFO] [stderr] Compiling notify v6.1.1 [INFO] [stderr] Compiling nix v0.23.2 [INFO] [stderr] Compiling lru v0.14.0 [INFO] [stderr] Compiling yasna v0.5.2 [INFO] [stderr] Compiling num_cpus v1.17.0 [INFO] [stderr] Compiling nix v0.26.4 [INFO] [stderr] Compiling tonic-build v0.14.2 [INFO] [stderr] Compiling pem v3.0.6 [INFO] [stderr] Compiling crossbeam-queue v0.3.12 [INFO] [stderr] Compiling itertools v0.12.1 [INFO] [stderr] Compiling spin v0.10.0 [INFO] [stderr] Compiling instant v0.1.13 [INFO] [stderr] Compiling dtoa v1.0.10 [INFO] [stderr] Compiling serde_derive v1.0.228 [INFO] [stderr] Compiling zerocopy-derive v0.8.27 [INFO] [stderr] Compiling tokio-macros v2.6.0 [INFO] [stderr] Compiling zerofrom-derive v0.1.6 [INFO] [stderr] Compiling displaydoc v0.2.5 [INFO] [stderr] Compiling yoke-derive v0.8.1 [INFO] [stderr] Compiling zerovec-derive v0.11.2 [INFO] [stderr] Compiling tracing-attributes v0.1.31 [INFO] [stderr] Compiling thiserror-impl v2.0.17 [INFO] [stderr] Compiling thiserror-impl v1.0.69 [INFO] [stderr] Compiling futures-macro v0.3.31 [INFO] [stderr] Compiling prost-derive v0.14.1 [INFO] [stderr] Compiling async-trait v0.1.89 [INFO] [stderr] Compiling enum-as-inner v0.6.1 [INFO] [stderr] Compiling zerofrom v0.1.6 [INFO] [stderr] Compiling asn1-rs-impl v0.2.0 [INFO] [stderr] Compiling yoke v0.8.1 [INFO] [stderr] Compiling futures-util v0.3.31 [INFO] [stderr] Compiling zerotrie v0.2.3 [INFO] [stderr] Compiling zerovec v0.11.5 [INFO] [stderr] Compiling asn1-rs-derive v0.6.0 [INFO] [stderr] Compiling equator-macro v0.4.2 [INFO] [stderr] Compiling pin-project-internal v1.1.10 [INFO] [stderr] Compiling enum-ordinalize-derive v4.3.2 [INFO] [stderr] Compiling async-stream-impl v0.3.6 [INFO] [stderr] Compiling tracing v0.1.44 [INFO] [stderr] Compiling prometheus-client-derive-encode v0.5.0 [INFO] [stderr] Compiling thiserror v1.0.69 [INFO] [stderr] Compiling tinystr v0.8.2 [INFO] [stderr] Compiling potential_utf v0.1.4 [INFO] [stderr] Compiling icu_locale_core v2.1.1 [INFO] [stderr] Compiling icu_collections v2.1.1 [INFO] [stderr] Compiling protobuf-support v3.7.2 [INFO] [stderr] Compiling prost v0.14.1 [INFO] [stderr] Compiling backtrace v0.3.76 [INFO] [stderr] Compiling asn1-rs v0.7.1 [INFO] [stderr] Compiling enum-ordinalize v4.3.2 [INFO] [stderr] Compiling unsafe-libyaml v0.2.11 [INFO] [stderr] Compiling seize v0.3.3 [INFO] [stderr] Compiling equator v0.4.2 [INFO] [stderr] Compiling base64 v0.12.3 [INFO] [stderr] Compiling prost-types v0.14.1 [INFO] [stderr] Compiling ppv-lite86 v0.2.21 [INFO] [stderr] Compiling half v2.7.1 [INFO] [stderr] Compiling pin-project v1.1.10 [INFO] [stderr] Compiling aligned-vec v0.6.4 [INFO] [stderr] Compiling winnow v0.7.13 [INFO] [stderr] Compiling file-id v0.2.3 [INFO] [stderr] Compiling ciborium-ll v0.2.2 [INFO] [stderr] Compiling rustls-native-certs v0.8.2 [INFO] [stderr] Compiling notify-debouncer-full v0.3.2 [INFO] [stderr] Compiling rand_chacha v0.9.0 [INFO] [stderr] Compiling icu_provider v2.1.1 [INFO] [stderr] Compiling rand v0.9.2 [INFO] [stderr] Compiling prost-build v0.14.1 [INFO] [stderr] Compiling rand_chacha v0.3.1 [INFO] [stderr] Compiling icu_normalizer v2.1.1 [INFO] [stderr] Compiling icu_properties v2.1.1 [INFO] [stderr] Compiling rand_chacha v0.2.2 [INFO] [stderr] Compiling serde v1.0.228 [INFO] [stderr] Compiling rand v0.8.5 [INFO] [stderr] Compiling der-parser v10.0.0 [INFO] [stderr] Compiling rand v0.7.3 [INFO] [stderr] Compiling x509-parser v0.17.0 [INFO] [stderr] Compiling flurry v0.5.2 [INFO] [stderr] Compiling netns-rs v0.1.0 [INFO] [stderr] Compiling educe v0.6.0 [INFO] [stderr] Compiling backoff v0.4.0 [INFO] [stderr] Compiling async-stream v0.3.6 [INFO] [stderr] Compiling ipnet v2.11.0 [INFO] [stderr] Compiling tonic-prost-build v0.14.2 [INFO] [stderr] Compiling tracing-serde v0.2.0 [INFO] [stderr] Compiling ciborium v0.2.2 [INFO] [stderr] Compiling tinytemplate v1.2.1 [INFO] [stderr] Compiling tracing-subscriber v0.3.22 [INFO] [stderr] Compiling textnonce v1.0.0 [INFO] [stderr] Compiling serde_yaml v0.9.34+deprecated [INFO] [stderr] Compiling prefix-trie v0.7.0 [INFO] [stderr] Compiling futures-executor v0.3.31 [INFO] [stderr] Compiling arcstr v1.2.0 [INFO] [stderr] Compiling duration-str v0.17.0 [INFO] [stderr] Compiling criterion v0.5.1 [INFO] [stderr] Compiling tokio v1.48.0 [INFO] [stderr] Compiling http v1.3.1 [INFO] [stderr] Compiling idna_adapter v1.2.1 [INFO] [stderr] Compiling idna v1.1.0 [INFO] [stderr] Compiling ztunnel v0.0.0 (/opt/rustwide/workdir) [INFO] [stderr] Compiling futures v0.3.31 [INFO] [stderr] Compiling ppp v2.3.0 [INFO] [stderr] Compiling prometheus-parse v0.2.5 [INFO] [stderr] Compiling url v2.5.7 [INFO] [stderr] Compiling keyed_priority_queue v0.4.2 [INFO] [stderr] Compiling rustls-pemfile v2.2.0 [INFO] [stderr] Compiling split-iter v0.1.0 [INFO] [stderr] Compiling http-body v1.0.1 [INFO] [stderr] Compiling http-body-util v0.1.3 [INFO] [stderr] Compiling protobuf-parse v3.7.2 [INFO] [stderr] Compiling tracing-appender v0.2.3 [INFO] [stderr] Compiling protobuf-codegen v3.7.2 [INFO] [stderr] Compiling pprof v0.15.0 [INFO] [stderr] Compiling tokio-util v0.7.17 [INFO] [stderr] Compiling hickory-proto v0.25.2 [INFO] [stderr] Compiling tokio-stream v0.1.17 [INFO] [stderr] Compiling pingora-timeout v0.6.0 [INFO] [stderr] Compiling tls-listener v0.11.0 [INFO] [stderr] Compiling pingora-pool v0.6.0 [INFO] [stderr] Compiling tonic v0.14.2 [INFO] [stderr] Compiling h2 v0.4.12 [INFO] [stderr] Compiling tower v0.5.2 [INFO] [stderr] Compiling tonic-prost v0.14.2 [INFO] [stderr] Compiling hyper v1.8.1 [INFO] [stderr] Compiling hickory-resolver v0.25.2 [INFO] [stderr] Compiling hickory-client v0.25.2 [INFO] [stderr] Compiling hyper-util v0.1.18 [INFO] [stderr] Compiling hickory-server v0.25.2 [INFO] [stderr] Compiling rustls-webpki v0.103.8 [INFO] [stderr] Compiling rcgen v0.14.5 [INFO] [stderr] Compiling tokio-rustls v0.26.4 [INFO] [stderr] Compiling hyper-rustls v0.27.7 [INFO] [stderr] Finished `dev` profile [unoptimized + debuginfo] target(s) in 3m 06s [INFO] running `Command { std: "docker" "inspect" "3c7ec97e1587cea6020fd03f4a177ba076bcb0550c8e6b49072ff019d58deb02", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "3c7ec97e1587cea6020fd03f4a177ba076bcb0550c8e6b49072ff019d58deb02", kill_on_drop: false }` [INFO] [stdout] 3c7ec97e1587cea6020fd03f4a177ba076bcb0550c8e6b49072ff019d58deb02 [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:cfa559dde23f44738f9990739aaa3487d8df3af37eb5df0572f7cf76004a604a" "/opt/rustwide/cargo-home/bin/cargo" "+d933cf483edf1605142ac6899ff32536c0ad8b22" "test" "--frozen" "--no-run" "--message-format=json", kill_on_drop: false }` [INFO] [stdout] c81c39c5f09caf83669e1562d415eb9ed32ffcb74136234d3cdfc1720cd77ced [INFO] running `Command { std: "docker" "start" "-a" "c81c39c5f09caf83669e1562d415eb9ed32ffcb74136234d3cdfc1720cd77ced", kill_on_drop: false }` [INFO] [stderr] Compiling serde_core v1.0.228 [INFO] [stderr] Compiling syn v1.0.109 [INFO] [stderr] Compiling serde v1.0.228 [INFO] [stderr] Compiling criterion v0.5.1 [INFO] [stderr] Compiling ztunnel v0.0.0 (/opt/rustwide/workdir) [INFO] [stderr] Compiling x509-parser v0.18.0 [INFO] [stderr] Compiling x509-parser v0.17.0 [INFO] [stderr] Compiling test-case-core v3.3.1 [INFO] [stderr] Compiling dtor-proc-macro v0.0.6 [INFO] [stderr] Compiling ctor-proc-macro v0.0.6 [INFO] [stderr] Compiling diff v0.1.13 [INFO] [stderr] Compiling dtor v0.1.1 [INFO] [stderr] Compiling ctor v0.5.0 [INFO] [stderr] Compiling test-case-macros v3.3.1 [INFO] [stderr] Compiling rcgen v0.14.5 [INFO] [stderr] Compiling test-case v3.3.1 [INFO] [stderr] Compiling pprof v0.15.0 [INFO] [stderr] Compiling neli-proc-macros v0.1.4 [INFO] [stderr] Compiling neli v0.6.5 [INFO] [stderr] Compiling local-ip-address v0.6.5 [INFO] [stderr] Finished `test` profile [unoptimized + debuginfo] target(s) in 1m 59s [INFO] running `Command { std: "docker" "inspect" "c81c39c5f09caf83669e1562d415eb9ed32ffcb74136234d3cdfc1720cd77ced", kill_on_drop: false }` [INFO] running `Command { std: "docker" "rm" "-f" "c81c39c5f09caf83669e1562d415eb9ed32ffcb74136234d3cdfc1720cd77ced", kill_on_drop: false }` [INFO] [stdout] c81c39c5f09caf83669e1562d415eb9ed32ffcb74136234d3cdfc1720cd77ced [INFO] running `Command { std: "docker" "create" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/target:/opt/rustwide/target:rw,Z" "-v" "/var/lib/crater-agent-workspace/builds/worker-4-tc1/source:/opt/rustwide/workdir:ro,Z" "-v" "/var/lib/crater-agent-workspace/cargo-home:/opt/rustwide/cargo-home:ro,Z" "-v" "/var/lib/crater-agent-workspace/rustup-home:/opt/rustwide/rustup-home:ro,Z" "-e" "SOURCE_DIR=/opt/rustwide/workdir" "-e" "CARGO_TARGET_DIR=/opt/rustwide/target" "-e" "CARGO_INCREMENTAL=0" "-e" "RUST_BACKTRACE=full" "-e" "RUSTFLAGS=--cap-lints=forbid" "-e" "RUSTDOCFLAGS=--cap-lints=forbid" "-e" "CARGO_HOME=/opt/rustwide/cargo-home" "-e" "RUSTUP_HOME=/opt/rustwide/rustup-home" "-w" "/opt/rustwide/workdir" "-m" "1610612736" "--user" "0:0" "--network" "none" "ghcr.io/rust-lang/crates-build-env/linux@sha256:cfa559dde23f44738f9990739aaa3487d8df3af37eb5df0572f7cf76004a604a" "/opt/rustwide/cargo-home/bin/cargo" "+d933cf483edf1605142ac6899ff32536c0ad8b22" "test" "--frozen", kill_on_drop: false }` [INFO] [stdout] 2068a663c44919f0acc8124738f921a8b9d04de2cf82d5ac66a771fae8b1640c [INFO] running `Command { std: "docker" "start" "-a" "2068a663c44919f0acc8124738f921a8b9d04de2cf82d5ac66a771fae8b1640c", kill_on_drop: false }` [INFO] [stderr] Finished `test` profile [unoptimized + debuginfo] target(s) in 0.96s [INFO] [stderr] Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/ztunnel-5cc2b77280b46551) [INFO] [stdout] [INFO] [stdout] running 196 tests [INFO] [stdout] test baggage::tests::baggage_parser ... ok [INFO] [stdout] test baggage::tests::baggage_parser_empty_values ... ok [INFO] [stdout] test baggage::tests::baggage_parser_multiline ... ok [INFO] [stdout] test baggage::tests::baggage_header_val_can_be_parsed ... ok [INFO] [stdout] test baggage::tests::baggage_parser_no_header ... ok [INFO] [stdout] test dns::name_util::tests::test_has_domain ... 2026-03-06T15:30:39.929264Z info dns::server starting local DNS server address=localhost:0 component="dns" [INFO] [stdout] ok [INFO] [stdout] test dns::name_util::tests::test_trim_domain ... ok [INFO] [stdout] test config::tests::test_parse_worker_threads ... ok [INFO] [stdout] test admin::tests::test_dump_config ... ok [INFO] [stdout] test dns::handler::tests::record_found ... ok [INFO] [stdout] 2026-03-06T15:30:39.931821Z info dns::server starting local DNS server address=localhost:0 component="dns" [INFO] [stdout] test config::tests::config_from_proxyconfig ... ok [INFO] [stdout] test dns::server::tests::search_domains ... ok [INFO] [stdout] 2026-03-06T15:30:39.929281Z info dns::server starting local DNS server address=localhost:0 component="dns" [INFO] [stdout] 2026-03-06T15:30:39.935509Z info dns::server starting local DNS server address=localhost:0 component="dns" [INFO] [stdout] 2026-03-06T15:30:39.936024Z info dns::server starting local DNS server address=localhost:0 component="dns" [INFO] [stdout] 2026-03-06T15:30:39.936148Z info telemetry new log filter is hickory_server::server=off,info [INFO] [stdout] 2026-03-06T15:30:39.936670Z info telemetry new log filter is hickory_server::server=off,debug [INFO] [stdout] 2026-03-06T15:30:39.936821Z info telemetry new log filter is hickory_server::server=off,access=debug,info [INFO] [stdout] 2026-03-06T15:30:39.936907Z info telemetry new log filter is hickory_server::server=off,warn [INFO] [stdout] 2026-03-06T15:30:39.937098Z info telemetry new log filter is hickory_server::server=off,info [INFO] [stdout] 2026-03-06T15:30:39.937181Z info telemetry new log filter is hickory_server::server=off,off [INFO] [stdout] test dns::forwarder::tests::not_found ... ok [INFO] [stdout] test dns::server::tests::test_get_wildcards ... ok [INFO] [stdout] test dns::server::tests::ipv4_in_6_should_unwrap ... ok [INFO] [stdout] test identity::manager::tests::identity_from_string ... ok [INFO] [stdout] test dns::forwarder::tests::found ... ok [INFO] [stdout] test admin::tests::test_change_log_level ... ok [INFO] [stdout] test dns::server::tests::large_response ... ok [INFO] [stdout] test copy::tests::copy ... ok [INFO] [stdout] test identity::manager::tests::test_get_existing_cert_info_basic ... ok [INFO] [stdout] test dns::server::tests::forward_to_server ... ok [INFO] [stdout] test dns::server::tests::test_to_kube_fqdns ... ok [INFO] [stdout] test identity::manager::tests::test_unused_cleanup ... ok [INFO] [stdout] test dns::server::tests::lookup ... ok [INFO] [stdout] test identity::manager::tests::test_backoff_resets_on_successful_fetch_after_failure ... ok [INFO] [stdout] test identity::manager::tests::test_forget_pending ... ok [INFO] [stdout] test identity::manager::tests::test_certificate_retention_on_refresh_failure ... ok [INFO] [stdout] test identity::manager::tests::test_duplicate_requests ... ok [INFO] [stdout] test identity::manager::tests::test_forget ... ok [INFO] [stdout] test inpod::admin::test::test_proxy_state ... ok [INFO] [stdout] test identity::caclient::tests::wrong_identity ... ok [INFO] [stdout] test identity::caclient::tests::empty_chain ... ok [INFO] [stdout] test identity::manager::tests::test_concurrency ... ok [INFO] [stdout] test identity::manager::tests::test_cache_refresh ... ok [INFO] [stdout] test admin::tests::test_dump_certs ... ok [INFO] [stdout] test identity::manager::tests::test_warmup_existing ... ok [INFO] [stdout] test identity::caclient::tests::fetch_certificate ... ok [INFO] [stdout] test identity::manager::tests::test_warmup_new ... ok [INFO] [stdout] test identity::manager::tests::test_priority ... ok [INFO] [stdout] test copy::tests::copystress ... ok [INFO] [stdout] test inpod::netns::tests::test_run_works ... FAILED [INFO] [stdout] test inpod::protocol::tests::test_parse_add_workload ... ok [INFO] [stdout] test inpod::config::test::test_inpod_config_port_reuse ... FAILED [INFO] [stdout] test inpod::protocol::tests::test_parse_add_workload_with_info ... ok [INFO] [stdout] test inpod::protocol::tests::test_parse_del_workload ... ok [INFO] [stdout] test inpod::protocol::tests::test_parse_del_workload_with_fds_fails ... ok [INFO] [stdout] test inpod::statemanager::tests::add_delete_add_workload_starts_only_one_proxy ... FAILED [INFO] [stdout] test inpod::config::test::test_inpod_config_outbound_sockets ... FAILED [INFO] [stdout] test inpod::config::test::test_inpod_config_no_port_reuse ... FAILED [INFO] [stdout] test metrics::server::test::test_content_type ... ok [INFO] [stdout] test inpod::workloadmanager::tests::test_process_add ... FAILED [INFO] [stdout] test proxy::connection_manager::tests::test_connection_manager_close ... ok [INFO] [stdout] test inpod::statemanager::tests::no_workload_info_rejected ... FAILED [INFO] [stdout] test proxy::connection_manager::tests::test_connection_manager_release ... ok [INFO] [stdout] test proxy::connection_manager::tests::test_policy_watcher_lifecycle ... ok [INFO] [stdout] test inpod::statemanager::tests::add_with_different_netns_keeps_latest_proxy ... FAILED [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::svc_hostname_to_workload_no_waypoint ... ok [INFO] [stdout] test inpod::statemanager::tests::add_workload_starts_a_proxy ... FAILED [INFO] [stdout] test inpod::statemanager::tests::idempotency_add_workload_fails_and_then_deleted ... FAILED [INFO] [stdout] test inpod::statemanager::tests::proxy_added_then_kept_with_new_snapshot ... FAILED [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_waypoint_without_attachment ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_service_via_waypoint_with_wrong_attachment ... ok [INFO] [stdout] test inpod::workloadmanager::tests::test_process_failed ... FAILED [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_workload_no_waypoint ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_service_with_app_tunnel ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_service_with_waypint_referenced_by_vip ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_workload_with_app_tunnel ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_workload_via_waypoint_with_wrong_attachment ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_workload_with_waypoint_referenced_by_vip ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_service_with_waypoint_referenced_by_pod ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::svc_hostname_to_workload_no_waypoint ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_server_ip_mismatch ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_inbound_request::to_workload_with_waypoint_referenced_by_pod ... ok [INFO] [stdout] test proxy::inbound::tests::test_build_response_baggage_feature_gate ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_service_with_waypint_referenced_by_vip ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_server_ip_mismatch ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_service_with_app_tunnel ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_service_via_waypoint_with_wrong_attachment ... ok [INFO] [stdout] test inpod::workloadmanager::tests::test_process_snapshot_with_missing_workload ... FAILED [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_service_with_waypoint_referenced_by_pod ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_workload_with_waypoint_referenced_by_pod ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_workload_via_waypoint_with_wrong_attachment ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_workload_no_waypoint ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_workload_with_waypoint_referenced_by_vip ... ok [INFO] [stdout] test inpod::statemanager::tests::del_workload_before_snapshot_removes_from_snapshot_and_pending ... FAILED [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_waypoint_without_attachment ... ok [INFO] [stdout] test proxy::inbound::tests::test_find_inbound_upstream::to_workload_with_app_tunnel ... ok [INFO] [stdout] test proxy::outbound::tests::build_forwarded ... ok [INFO] [stdout] test inpod::workloadmanager::tests::test_process_failed_announce ... FAILED [INFO] [stdout] test inpod::statemanager::tests::idemepotency_add_workload_starts_only_one_proxy ... FAILED [INFO] [stdout] test proxy::outbound::tests::build_request_destination_waypoint_mismatch_ip ... ok [INFO] [stdout] test inpod::statemanager::tests::idemepotency_add_workload_fails ... FAILED [INFO] [stdout] test proxy::outbound::tests::build_request_destination_waypoint ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_known_dest_remote_node_hbone ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_destination_svc_waypoint ... ok [INFO] [stdout] test proxy::inbound::tests::test_incoming_baggage_parsing_feature_gate ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_passthrough_svc ... ok [INFO] [stdout] test inpod::workloadmanager::tests::test_process_add_and_del ... FAILED [INFO] [stdout] test proxy::outbound::tests::build_request_double_hbone ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_source_waypoint ... ok [INFO] [stdout] test inpod::statemanager::tests::workload_added_while_pending ... FAILED [INFO] [stdout] test proxy::outbound::tests::build_request_passthrough_svc_with_waypoint ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_known_dest_remote_node_tcp ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_target_port ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_known_dest_local_node_hbone ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_host_network ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_empty_service ... ok [INFO] [stdout] test proxy::outbound::tests::multiple_address_workload ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_known_dest_local_node_tcp ... ok [INFO] [stdout] test proxy::outbound::tests::service_ip_families ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_wrong_network ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_unknown_dest ... ok [INFO] [stdout] test proxy::outbound::tests::test_x_forwarded_network_header ... ok [INFO] [stdout] test rbac::tests::destination_ips ... ok [INFO] [stdout] test proxy::outbound::tests::build_request_failover_to_remote ... ok [INFO] [stdout] test rbac::tests::cidr_range ... ok [INFO] [stdout] test rbac::tests::destination_ports ... ok [INFO] [stdout] test proxy::tests::test_parse_forwarded_host ... ok [INFO] [stdout] test rbac::tests::namespaces ... ok [INFO] [stdout] test rbac::tests::not_destination_ports ... ok [INFO] [stdout] test rbac::tests::not_destination_ips ... ok [INFO] [stdout] test rbac::tests::not_service_accounts ... ok [INFO] [stdout] test rbac::tests::not_source_ips ... ok [INFO] [stdout] test rbac::tests::principals ... ok [INFO] [stdout] test rbac::tests::not_namespaces ... ok [INFO] [stdout] test rbac::tests::not_principals ... ok [INFO] [stdout] test rbac::tests::rbac_empty_policy ... ok [INFO] [stdout] test rbac::tests::rbac_nesting ... ok [INFO] [stdout] test rbac::tests::service_accounts ... ok [INFO] [stdout] test rbac::tests::source_ips ... ok [INFO] [stdout] test rbac::tests::string_match::exact_empty_mismatch ... ok [INFO] [stdout] test rbac::tests::string_match::exact_match ... ok [INFO] [stdout] test rbac::tests::string_match::exact_mismatch ... ok [INFO] [stdout] test rbac::tests::string_match::prefix_empty_mismatch ... ok [INFO] [stdout] test rbac::tests::string_match::prefix_match ... ok [INFO] [stdout] test rbac::tests::string_match::prefix_mismatch ... ok [INFO] [stdout] test rbac::tests::string_match::presence_mismatch ... ok [INFO] [stdout] test rbac::tests::string_match::suffix_match ... ok [INFO] [stdout] test rbac::tests::string_match::suffix_empty_mismatch ... ok [INFO] [stdout] test rbac::tests::string_match::presence_match ... ok [INFO] [stdout] test rbac::tests::string_match::suffix_mismatch ... ok [INFO] [stdout] test state::policy::tests::rbac_change_scope ... ok [INFO] [stdout] test socket::socket_tests::test_keep_alive_options ... ok [INFO] [stdout] 2026-03-06T15:30:41.039995Z debug state dry-run: deny policy match policy="ns1/dry-run-deny" [INFO] [stdout] 2026-03-06T15:30:41.040093Z debug state dry-run: allow policy match policy="ns1/dry-run-allow" [INFO] [stdout] 2026-03-06T15:30:41.040475Z debug state no allow policies, allow [INFO] [stdout] test state::tests::assert_rbac_with_dest_workload_info ... ok [INFO] [stdout] 2026-03-06T15:30:41.040836Z debug state no allow policies, allow [INFO] [stdout] 2026-03-06T15:30:41.043407Z debug state dry-run: deny policy match policy="ns1/wildcard" [INFO] [stdout] 2026-03-06T15:30:41.043555Z debug state dry-run: allow policy match policy="ns1/wildcard" [INFO] [stdout] 2026-03-06T15:30:41.043677Z debug state allow policy match policy="ns1/foo" [INFO] [stdout] 2026-03-06T15:30:41.043848Z debug state dry-run: deny policy match policy="ns1/wildcard" [INFO] [stdout] 2026-03-06T15:30:41.043956Z debug state dry-run: allow policy match policy="ns1/wildcard" [INFO] [stdout] 2026-03-06T15:30:41.044068Z debug state no allow policies matched [INFO] [stdout] 2026-03-06T15:30:41.044169Z debug state dry-run: deny policy match policy="ns1/wildcard" [INFO] [stdout] 2026-03-06T15:30:41.044265Z debug state deny policy match policy="ns1/deny" [INFO] [stdout] test state::tests::assert_rbac_logic_deny_allow ... ok [INFO] [stdout] 2026-03-06T15:30:41.045887Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.046400Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel7 [INFO] [stdout] test state::tests::find_upstream_port_mappings::app_tunnel ... ok [INFO] [stdout] test state::tests::find_upstream_port_mappings::ep_mapping ... ok [INFO] [stdout] 2026-03-06T15:30:41.046594Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.047539Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.047539Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.042040Z debug state allow policy match policy="ns1/real-allow" [INFO] [stdout] test rbac::tests::rbac_multi_rule ... ok [INFO] [stdout] test state::tests::find_upstream_port_mappings::svc_mapping ... ok [INFO] [stdout] 2026-03-06T15:30:41.050984Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.051120Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.049146Z debug proxy::pool:client{request=0} didn't find a connection for key 14184913346861640675, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.052262Z debug proxy::pool didn't find a connection for key 9240842750640762394, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.052444Z debug state fetch address network_addr.address=127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.052778Z debug state fetch hostname hostname=default/defaulthost [INFO] [stdout] 2026-03-06T15:30:41.052871Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.053202Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.052836Z debug state fetch address network_addr.address=127.0.0.2 [INFO] [stdout] 2026-03-06T15:30:41.053848Z debug state fetch hostname hostname=default/nothost [INFO] [stdout] 2026-03-06T15:30:41.053841Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.054312Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.054594Z debug proxy::pool:client{request=0} inserting conn mutex for key 14184913346861640675 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.054313Z debug proxy::pool inserting conn mutex for key 9240842750640762394 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.054811Z debug proxy::pool:client{request=0} nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.054961Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.055048Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.055054Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.055208Z debug proxy::pool spawning new pool conn for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.054917Z debug proxy::pool:client{request=0} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.055292Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.055380Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.055371Z debug state:client{request=0} wait for workload wl=default.ns (source-workload) [INFO] [stdout] test state::tests::lookup_address ... ok [INFO] [stdout] 2026-03-06T15:30:41.056579Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.057725Z debug state wait for workload wl=fake.fake (fake) [INFO] [stdout] 2026-03-06T15:30:41.057332Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.058930Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.058939Z debug state got sub, waiting for workload wl=fake.fake (fake) [INFO] [stdout] 2026-03-06T15:30:41.059359Z debug proxy::pool didn't find a connection for key 10348870604864159223, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.057684Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.059582Z debug proxy::pool inserting conn mutex for key 10348870604864159223 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.057944Z debug state:client{request=0} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.060215Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.056874Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.060287Z debug proxy::pool spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.060315Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.060914Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.061180Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.061134Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.048725Z debug proxy::pool:client{request=0} didn't find a connection for key 12973763988511000141, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.061839Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.061888Z debug proxy::pool:client{request=0} inserting conn mutex for key 12973763988511000141 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.055477Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.063195Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.061618Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.063371Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.063400Z debug proxy::pool didn't find a connection for key 10269382516670620387, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.063591Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.062862Z debug proxy::pool:client{request=0} nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.062967Z debug proxy::pool checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:45253[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 2296657327393124964, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.063780Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.063799Z debug proxy::pool:client{request=0} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:39731[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.063580Z debug proxy::pool inserting conn mutex for key 10269382516670620387 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.064035Z debug state:client{request=0} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.062216Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.064173Z debug state:client{request=0} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.064066Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.064798Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.064924Z debug proxy::pool spawning new pool conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.065113Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] test state::tests::assert_rbac_dry_run_with_real_policies ... ok [INFO] [stdout] 2026-03-06T15:30:41.065138Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.065301Z warn state timed out waiting for workload 'fake.fake (fake)' from xds [INFO] [stdout] 2026-03-06T15:30:41.065598Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] test state::tests::test_wait_for_workload_delay_fails ... ok [INFO] [stdout] 2026-03-06T15:30:41.064689Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] test state::workload::tests::byte_to_ipaddr_empty ... ok [INFO] [stdout] test state::workload::tests::byte_to_ipaddr_garbage ... ok [INFO] [stdout] test state::workload::tests::byte_to_ipaddr_unspecified ... ok [INFO] [stdout] 2026-03-06T15:30:41.067126Z debug proxy::pool:client{request=0} didn't find a connection for key 385506954252831254, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.067163Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.065176Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.067623Z debug proxy::pool:client{request=0} inserting conn mutex for key 385506954252831254 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.067749Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.067889Z debug proxy::pool didn't find a connection for key 17834477119691129554, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.065456Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 2296657327393124964, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.069499Z debug state wait for workload wl=default. () [INFO] [stdout] 2026-03-06T15:30:41.071033Z debug proxy::pool:client{request=0} nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.073175Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.073758Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.067299Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.072163Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.073527Z debug state got sub, waiting for workload wl=default. () [INFO] [stdout] 2026-03-06T15:30:41.074247Z debug proxy::pool inserting conn mutex for key 17834477119691129554 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.074507Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.073997Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.071119Z debug proxy::pool:client{request=0} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.074647Z debug proxy::pool spawning new pool conn for 127.0.0.3(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.075222Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.067478Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.075048Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.075347Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.075892Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.075925Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.076069Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.076148Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.075304Z debug state:client{request=0} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.076191Z debug proxy::pool didn't find a connection for key 1106033203852738198, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.076634Z debug state:client{request=0} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.076578Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.076388Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.075788Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.076859Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.077414Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.076704Z debug proxy::pool inserting conn mutex for key 1106033203852738198 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.077780Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.077475Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.077937Z debug rustls::client::hs:client{request=0} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.078206Z debug rustls::client::hs:client{request=0} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.078317Z debug rustls::client::hs:client{request=0} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.077911Z debug proxy::pool spawning new pool conn for 127.0.0.4(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.078798Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.078481Z debug rustls::client::hs:client{request=0} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.078861Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.077254Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.079489Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.078176Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.080486Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.079672Z debug proxy::pool didn't find a connection for key 9104731768476400337, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.081011Z debug proxy::pool inserting conn mutex for key 9104731768476400337 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.081094Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.081529Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.081310Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.080833Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.081701Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel10 [INFO] [stdout] 2026-03-06T15:30:41.081174Z debug proxy::pool spawning new pool conn for 127.0.0.5(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.081472Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.081041Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.083363Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.081918Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.084054Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.084880Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.084168Z debug proxy::pool::test client spent 46ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.084744Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.085296Z debug proxy::pool didn't find a connection for key 5339849372631072067, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.085394Z debug proxy::pool inserting conn mutex for key 5339849372631072067 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.085463Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.085548Z debug proxy::pool spawning new pool conn for 127.0.0.6(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.085693Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.083160Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] test state::workload::tests::byte_to_ipaddr_v4_happy ... 2026-03-06T15:30:41.085650Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.083872Z debug state wait for workload wl=default. () [INFO] [stdout] 2026-03-06T15:30:41.085883Z debug state got sub, waiting for workload wl=default. () [INFO] [stdout] 2026-03-06T15:30:41.085129Z debug proxy::pool didn't find a connection for key 6224894721252289791, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.085831Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.084077Z debug rustls::client::hs:client{request=0} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.086224Z debug proxy::pool didn't find a connection for key 2515837343345295888, making sure lockmap has entry [INFO] [stdout] ok [INFO] [stdout] test state::tests::test_wait_for_workload ... ok [INFO] [stdout] test state::workload::tests::byte_to_ipaddr_v4_loopback ... ok [INFO] [stdout] 2026-03-06T15:30:41.085940Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.086108Z debug rustls::client::hs:client{request=0} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.086505Z debug proxy::pool inserting conn mutex for key 2515837343345295888 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.086082Z debug proxy::pool inserting conn mutex for key 6224894721252289791 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.085832Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.087185Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.087366Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.086384Z debug rustls::client::tls13:client{request=0} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.076348Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.086860Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.087270Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.087410Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.088381Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.085221Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] test state::workload::tests::byte_to_ipaddr_v6_loopback ... ok [INFO] [stdout] test state::workload::tests::byte_to_ipaddr_v6_happy ... ok [INFO] [stdout] 2026-03-06T15:30:41.087061Z debug rustls::client::tls13:client{request=0} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.088453Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.082907Z debug proxy::pool:client{request=0} didn't find a connection for key 4867941314952592034, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.088506Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.088826Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.087520Z debug proxy::pool didn't find a connection for key 12787015852653906917, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.087350Z debug proxy::pool spawning new pool conn for 127.0.0.7(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.088089Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.088345Z debug proxy::pool spawning new pool conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:45253[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.087670Z debug rustls::client::tls13:client{request=0} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.088673Z debug rustls::client::tls13:client{request=0} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.089186Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.089069Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.089659Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.089009Z debug proxy::pool:client{request=0} inserting conn mutex for key 4867941314952592034 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.089009Z debug proxy::pool inserting conn mutex for key 12787015852653906917 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.089088Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.090705Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.089391Z debug rustls::client::hs:client{request=0} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.090811Z debug proxy::pool spawning new pool conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38865[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.090896Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.090976Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.089550Z debug rustls::client::hs:client{request=0} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.090592Z debug proxy::pool:client{request=0} nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.091223Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.091641Z debug xds:local_client load local config: workloads: [INFO] [stdout] - workloadIps: [INFO] [stdout] - 127.0.0.1 [INFO] [stdout] protocol: HBONE [INFO] [stdout] networkMode: Standard [INFO] [stdout] uid: cluster1//v1/Pod/default/local [INFO] [stdout] name: local [INFO] [stdout] namespace: default [INFO] [stdout] serviceAccount: default [INFO] [stdout] node: local [INFO] [stdout] status: Healthy [INFO] [stdout] clusterId: '' [INFO] [stdout] capacity: 1 [INFO] [stdout] services: [INFO] [stdout] default/example.com: [INFO] [stdout] 80: 8080 [INFO] [stdout] default/example2.com: [INFO] [stdout] 80: 8080 [INFO] [stdout] - workloadIps: [INFO] [stdout] - 127.0.0.2 [INFO] [stdout] protocol: TCP [INFO] [stdout] networkMode: Standard [INFO] [stdout] uid: cluster1//v1/Pod/default/local-tcp [INFO] [stdout] name: local-tcp [INFO] [stdout] namespace: default [INFO] [stdout] serviceAccount: default [INFO] [stdout] node: local [INFO] [stdout] status: Healthy [INFO] [stdout] clusterId: '' [INFO] [stdout] capacity: 1 [INFO] [stdout] services: [INFO] [stdout] default/example2.com: [INFO] [stdout] 80: 8080 [INFO] [stdout] default/example.com: [INFO] [stdout] 80: 8080 [INFO] [stdout] policies: [INFO] [stdout] - name: deny-9999 [INFO] [stdout] namespace: default [INFO] [stdout] scope: Namespace [INFO] [stdout] action: Allow [INFO] [stdout] rules: [INFO] [stdout] - - - notDestinationPorts: [INFO] [stdout] - 9999 [INFO] [stdout] dryRun: false [INFO] [stdout] services: [INFO] [stdout] - name: local [INFO] [stdout] namespace: default [INFO] [stdout] hostname: example.com [INFO] [stdout] vips: [INFO] [stdout] - /127.10.0.1 [INFO] [stdout] ports: [INFO] [stdout] 80: 8080 [INFO] [stdout] endpoints: {} [INFO] [stdout] subjectAltNames: [INFO] [stdout] - spiffe://cluster.local/ns/default/sa/local [INFO] [stdout] canonical: true [INFO] [stdout] - name: remote [INFO] [stdout] namespace: default [INFO] [stdout] hostname: example2.com [INFO] [stdout] vips: [INFO] [stdout] - remote/127.10.0.2 [INFO] [stdout] ports: [INFO] [stdout] 80: 8080 [INFO] [stdout] endpoints: {} [INFO] [stdout] subjectAltNames: [] [INFO] [stdout] canonical: true [INFO] [stdout] [INFO] [stdout] 2026-03-06T15:30:41.090802Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.094184Z info xds:local_client local config initialized num_workloads=2 num_policies=1 [INFO] [stdout] 2026-03-06T15:30:41.094651Z debug proxy::pool didn't find a connection for key 17715727410566080097, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.093875Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.094827Z debug proxy::pool inserting conn mutex for key 17715727410566080097 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.092241Z debug proxy::pool:client{request=0} spawning new pool conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:41531[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.095130Z debug state:client{request=0} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.095300Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.094785Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.090721Z debug proxy::pool:client{request=0} didn't find a connection for key 498255846233636669, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.095389Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.095279Z debug state:client{request=0} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.095568Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.094921Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] test state::workload::tests::local_client ... ok [INFO] [stdout] 2026-03-06T15:30:41.095835Z debug proxy::pool spawning new pool conn for 127.0.0.8(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.095939Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.096074Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.096056Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.095430Z debug proxy::pool:client{request=0} inserting conn mutex for key 498255846233636669 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.088671Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.096231Z debug proxy::pool didn't find a connection for key 15085180471312385809, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.092079Z debug h2::client:client{request=0} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.096609Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.096788Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.096944Z debug h2::client:client{request=0} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.096559Z debug proxy::pool inserting conn mutex for key 15085180471312385809 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.097127Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.096300Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.097287Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.094410Z debug h2::client:client{request=0} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.097088Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.097352Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.097712Z debug proxy::pool spawning new pool conn for 127.0.0.9(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.097603Z debug h2::client:client{request=0} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.098461Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.096415Z debug proxy::pool:client{request=0} nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.098513Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.098255Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.098369Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.097804Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.098881Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.089787Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.098428Z debug h2::codec::framed_write:client{request=0} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.099172Z debug proxy::pool didn't find a connection for key 9749699050425861695, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.097332Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.098786Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.099315Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.098571Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.099046Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.099394Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.099404Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.098144Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.098981Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.098582Z debug proxy::pool:client{request=0} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44727[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.097466Z debug h2::codec::framed_write:client{request=0} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.099136Z debug proxy::pool:client{request=0} checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:39731[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 12973763988511000141, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.101101Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.100770Z debug proxy::pool inserting conn mutex for key 9749699050425861695 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.101691Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.102278Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.101180Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.104228Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.102819Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.102585Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.105158Z debug proxy::pool spawning new pool conn for 127.0.0.10(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.104213Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.104229Z debug proxy::pool:client{request=0} checked out connection for ConnectionMeta { key: 12973763988511000141, id: 0 } is now at max streamcount; removing from pool [INFO] [stdout] 2026-03-06T15:30:41.104509Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.102585Z debug state:client{request=0} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.104004Z debug proxy::pool:client{request=0} checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 14184913346861640675, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.104614Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.105216Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.105249Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.104396Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.104953Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.105230Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.101960Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.105631Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.107653Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.105798Z debug state:client{request=0} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.106420Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.106546Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.107432Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.106667Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.106813Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.102449Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.106981Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.106230Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 14184913346861640675, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.107162Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.107403Z debug proxy::pool checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:34869[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 473791882615185783, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.108254Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.108630Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.109073Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.108764Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.108389Z debug proxy::pool didn't find a connection for key 11228261214770619285, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.108686Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.109256Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.108944Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.111230Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.109885Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.108864Z debug proxy::pool checking in new conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:45253[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 6224894721252289791, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.111581Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.109390Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.111727Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.109384Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.108934Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.109690Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 473791882615185783, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.111063Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.110974Z debug proxy::pool inserting conn mutex for key 11228261214770619285 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.105699Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.111852Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.110812Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.112852Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.112885Z debug proxy::pool spawning new pool conn for 127.0.0.11(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.111367Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.111703Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.111546Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.112315Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.113329Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.113093Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] test state::workload::tests::staged_services_cleanup ... ok [INFO] [stdout] 2026-03-06T15:30:41.113433Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 6224894721252289791, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.113803Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.113798Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.113786Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.112931Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] test state::workload::tests::overlapping_workload_ip ... ok [INFO] [stdout] 2026-03-06T15:30:41.113666Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.112598Z debug proxy::pool checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 10407060835540538402, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.114072Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.114669Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.114773Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.111193Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.115785Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.115890Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.111356Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.114263Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.116222Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.116530Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.115613Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.116525Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.116361Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.111876Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.114949Z debug proxy::pool didn't find a connection for key 6357918293284060487, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.115013Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.115793Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.116869Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.115095Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 10407060835540538402, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.116262Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.116826Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.117363Z debug proxy::pool inserting conn mutex for key 6357918293284060487 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.117395Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.115858Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.116701Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.117231Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.116693Z debug h2::proto::settings:client{request=0}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.117572Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.118347Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.119189Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.119201Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.118812Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.118610Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.118913Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.118531Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.119302Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.119384Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.119614Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.119886Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.119948Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.119511Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.119832Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.119698Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.120050Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.120366Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.120475Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.120660Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.120701Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.114899Z debug rustls::client::hs:client{request=0} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] test state::workload::tests::unhealthy_workloads ... ok [INFO] [stdout] 2026-03-06T15:30:41.120888Z debug rustls::client::hs:client{request=0} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.120270Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.120157Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.118430Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.120083Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.119332Z debug proxy::pool spawning new pool conn for 127.0.0.12(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.121178Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.121011Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.121373Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.121884Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.123153Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.122972Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.122253Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.122303Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.122401Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.123543Z debug h2::proto::settings:client{request=0}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.123354Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.123558Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.124086Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.124180Z debug proxy::pool::test client spent 75ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.123682Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.123079Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.122181Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.124516Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 12973763988511000141, id: 1 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.124637Z debug proxy::pool:client{request=1} new connection needed for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:39731[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.124646Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.124073Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.124078Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.125026Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.124876Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.124926Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.125701Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.125047Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.125085Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.125640Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.127685Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.125916Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.127774Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.124721Z debug proxy::pool:client{request=1} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:39731[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.126047Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.126810Z debug proxy::pool::test client spent 77ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.128234Z debug state:client{request=1} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.126348Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.125475Z debug rustls::client::hs:client{request=0} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.127551Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.125574Z debug proxy::pool didn't find a connection for key 10222465753960659478, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.127939Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.128945Z debug rustls::client::tls13:client{request=0} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.128304Z debug state:client{request=1} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.128525Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 1 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.129303Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.128379Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.129964Z debug proxy::pool::test client spent 44ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.130177Z debug proxy::pool inserting conn mutex for key 10222465753960659478 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.130105Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.130130Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.129897Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.130237Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.128883Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.131758Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.129665Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.129557Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.129332Z debug rustls::client::tls13:client{request=0} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.132280Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.123684Z warn xds tried to remove service, but it was not found [INFO] [stdout] 2026-03-06T15:30:41.132714Z debug proxy::pool:client{request=1} checked out connection for ConnectionMeta { key: 14184913346861640675, id: 1 } is now at max streamcount; removing from pool [INFO] [stdout] 2026-03-06T15:30:41.132957Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.133116Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_no_match [INFO] [stdout] 2026-03-06T15:30:41.132977Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.130381Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.132592Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 14184913346861640675, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.133241Z debug proxy::pool spawning new pool conn for 127.0.0.13(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.130356Z debug pingora_pool::connection idle connection is being closed [INFO] [stdout] 2026-03-06T15:30:41.132913Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.133380Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.133786Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.134030Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.131552Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.130551Z debug rustls::client::hs:client{request=1} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.132475Z debug proxy::h2::client draining outer HBONE connection [INFO] [stdout] 2026-03-06T15:30:41.130452Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.134941Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.133418Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.134784Z debug pingora_pool::connection idle connection is being closed [INFO] [stdout] 2026-03-06T15:30:41.134341Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.135505Z debug proxy::pool didn't find a connection for key 13141878506853951134, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.126974Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.134622Z debug rustls::client::hs:client{request=1} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.133093Z debug rustls::client::hs:client{request=0} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.134950Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.135426Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.133083Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.138421Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.138429Z debug rustls::client::hs:client{request=0} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.133764Z debug state failed to fetch workload for cluster1//v1/Pod/default/ep_almost [INFO] [stdout] 2026-03-06T15:30:41.133517Z debug pingora_pool::connection evict fd: 1 from key 6224894721252289791 [INFO] [stdout] 2026-03-06T15:30:41.135967Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.135577Z debug proxy::pool inserting conn mutex for key 13141878506853951134 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.136407Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.137172Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.135111Z debug pingora_pool::connection evict fd: 0 from key 2296657327393124964 [INFO] [stdout] 2026-03-06T15:30:41.138205Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] test state::tests::test_load_balance ... ok [INFO] [stdout] test strng::test::interning ... ok [INFO] [stdout] 2026-03-06T15:30:41.135787Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.134102Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.138933Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.139355Z debug proxy::pool connection ConnectionMeta { key: 6224894721252289791, id: 1 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.136792Z debug h2::client:client{request=0} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.140344Z debug proxy::pool::test server stream started [INFO] [stdout] test time::tests::test_converter ... ok [INFO] [stdout] 2026-03-06T15:30:41.140620Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.139486Z debug rustls::client::hs:client{request=0} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.140448Z debug proxy::pool connection ConnectionMeta { key: 2296657327393124964, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.141009Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.143304Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.141197Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.140265Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.141408Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.142665Z debug h2::client:client{request=0} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.133638Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.139376Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.143082Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; IO error error=UnexpectedEof [INFO] [stdout] 2026-03-06T15:30:41.143593Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.142601Z debug proxy::h2::client draining outer HBONE connection [INFO] [stdout] 2026-03-06T15:30:41.143104Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.139768Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.142643Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.141569Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.146239Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.146852Z debug proxy::pool spawning new pool conn for 127.0.0.14(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.147973Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.148118Z debug rustls::client::hs:client{request=0} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.149955Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.149991Z debug rustls::client::tls13:client{request=0} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.143277Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.147997Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.148355Z debug rustls::client::hs:client{request=1} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.148983Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.146699Z debug h2::codec::framed_write:client{request=0} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.149637Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.147145Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.146478Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.148430Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.149460Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.151832Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.150552Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.152294Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.149785Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.145473Z debug proxy::pool didn't find a connection for key 12401958628566421154, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.150845Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; IO error error=UnexpectedEof [INFO] [stdout] 2026-03-06T15:30:41.150622Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.153485Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.151945Z debug rustls::client::tls13:client{request=1} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.146933Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.153241Z debug proxy::pool::test client spent 114ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.153975Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.154114Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.154536Z debug proxy::pool inserting conn mutex for key 12401958628566421154 into lockmap [INFO] [stdout] test state::workload::tests::unhealthy_workloads_staged ... ok [INFO] [stdout] 2026-03-06T15:30:41.155833Z debug proxy::pool::test client spent 27ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.151683Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.154250Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.152509Z debug proxy::pool:client{request=0} checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 385506954252831254, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.154099Z debug rustls::client::tls13:client{request=1} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.156670Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.153566Z debug proxy::pool::test client spent 126ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.154161Z debug proxy::pool:client{request=0} checkout - found mutex for pool key ConnectionMeta { key: 10407060835540538402, id: 1 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.155977Z debug proxy::pool:client{request=2} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 2 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.153116Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.157565Z debug proxy::pool:client{request=0} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.158155Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 10407060835540538402, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.154176Z debug proxy::pool didn't find a connection for key 11992319344839659518, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.152981Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.157678Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.158128Z debug proxy::pool:client{request=0} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.153535Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.156374Z debug proxy::pool checking in new conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38865[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 12787015852653906917, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.156802Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.151968Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.157233Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 473791882615185783, id: 1 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.156608Z debug rustls::client::hs:client{request=1} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.156898Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.158280Z debug proxy::pool inserting conn mutex for key 11992319344839659518 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.156428Z debug tls::crl initializing crl manager path="/tmp/.tmpHrURdQ" [INFO] [stdout] 2026-03-06T15:30:41.152940Z debug rustls::client::tls13:client{request=0} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.156146Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.159336Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12787015852653906917, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.162172Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.159602Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.158343Z debug proxy::pool:client{request=2} new connection needed for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.162877Z debug tls::crl crl file is empty, treating as no revocations path="/tmp/.tmpHrURdQ" [INFO] [stdout] 2026-03-06T15:30:41.160041Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:34869[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.162695Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.162417Z debug rustls::client::hs:client{request=0} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.161910Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.160459Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.158461Z debug proxy::pool connection ConnectionMeta { key: 10407060835540538402, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.163258Z debug proxy::pool:client{request=2} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] test tls::crl::tests::test_crl_manager_empty_file ... ok [INFO] [stdout] 2026-03-06T15:30:41.164079Z debug proxy::pool connection ConnectionMeta { key: 473791882615185783, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.163127Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.164251Z debug tls::crl initializing crl manager path="/tmp/.tmp9FLLDc" [INFO] [stdout] 2026-03-06T15:30:41.164508Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.163340Z debug state:client{request=2} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.163492Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.163035Z debug proxy::pool spawning new pool conn for 127.0.0.15(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.163695Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.164994Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.165518Z debug tls::crl failed to initialize crl manager error=CRL error: failed to parse crl 1: TrailingData(CertRevocationList) [INFO] [stdout] 2026-03-06T15:30:41.165497Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.165120Z debug h2::client:client{request=0} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.165523Z debug hyper_util TLS handshake succeeded [INFO] [stdout] test tls::crl::tests::test_crl_manager_invalid_file ... ok [INFO] [stdout] 2026-03-06T15:30:41.164757Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.166486Z debug h2::client:client{request=0} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.165256Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.165132Z debug state:client{request=2} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.155826Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.163508Z debug proxy::pool:client{request=1} starting an idle timeout for connection ConnectionMeta { key: 473791882615185783, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.166591Z debug h2::codec::framed_write:client{request=0} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.165835Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.166726Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.171400Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.172048Z debug proxy::pool spawning new pool conn for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] test state::tests::test_wait_for_workload_eventually ... ok [INFO] [stdout] 2026-03-06T15:30:41.166590Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.165422Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.169854Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.171178Z debug rustls::client::hs:client{request=2} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.166804Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.174127Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.169135Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.174265Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.171607Z debug proxy::pool:client{request=0} checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44727[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 498255846233636669, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.174367Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.171823Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.172139Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.172510Z debug proxy::pool didn't find a connection for key 1496265549087212537, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.168829Z debug h2::client:client{request=1} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.175315Z debug h2::client:client{request=1} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.173606Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel0 [INFO] [stdout] 2026-03-06T15:30:41.171802Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.173445Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.175486Z debug h2::codec::framed_write:client{request=1} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.173816Z debug rustls::client::hs:client{request=2} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.174273Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.182511Z debug proxy::pool:client{request=1} checked out connection for ConnectionMeta { key: 12973763988511000141, id: 1 } is now at max streamcount; removing from pool [INFO] [stdout] 2026-03-06T15:30:41.171588Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.183194Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.175237Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.175817Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.183065Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.172868Z debug tls::crl initializing crl manager path="/nonexistent/path/crl.pem" [INFO] [stdout] 2026-03-06T15:30:41.184097Z warn tls::crl crl file not found, will retry on first validation path="/nonexistent/path/crl.pem" [INFO] [stdout] 2026-03-06T15:30:41.175159Z debug proxy::pool inserting conn mutex for key 1496265549087212537 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.163079Z debug rustls::webpki::anchors add_parsable_certificates processed 2 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.181922Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.173783Z debug tls::crl initializing crl manager path="/tmp/.tmpmV90Ox" [INFO] [stdout] 2026-03-06T15:30:41.184233Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.167368Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.183272Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.183374Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.183380Z debug h2::codec::framed_read:client{request=1}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.183077Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.182261Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.175594Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.183609Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] test tls::crl::tests::test_crl_manager_missing_file ... ok [INFO] [stdout] 2026-03-06T15:30:41.175109Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.184793Z debug tls::crl crl loaded successfully path="/tmp/.tmpmV90Ox" format="PEM" count=1 [INFO] [stdout] test tls::crl::tests::test_crl_manager_valid_crl ... ok [INFO] [stdout] 2026-03-06T15:30:41.184687Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.184859Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.181997Z debug proxy::pool::test client spent 27ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.186301Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.184546Z debug proxy::pool spawning new pool conn for 127.0.0.16(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.185438Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.185072Z debug h2::codec::framed_write:client{request=1}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.185441Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.184270Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 498255846233636669, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.185720Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] test tls::csr::tests::test_csr ... ok [INFO] [stdout] 2026-03-06T15:30:41.188110Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.187099Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.188252Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.186480Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.187346Z debug h2::codec::framed_read:client{request=1}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.187787Z debug rustls::client::hs:client{request=2} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.187784Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.186974Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.185776Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.188946Z debug rustls::client::tls13:client{request=2} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.188252Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.185203Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.189174Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.186706Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 10407060835540538402, id: 2 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.191660Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.188765Z debug h2::codec::framed_write:client{request=1}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.191554Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.189882Z debug proxy::pool didn't find a connection for key 8068171329476349730, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.186250Z debug proxy::pool didn't find a connection for key 230488702111320817, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.190886Z debug proxy::pool::test client spent 33ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.192235Z debug proxy::pool inserting conn mutex for key 230488702111320817 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.191202Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.188534Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.192956Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.191434Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.189758Z debug rustls::client::tls13:client{request=2} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.192118Z debug proxy::pool inserting conn mutex for key 8068171329476349730 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.192280Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.193393Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.192178Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.193501Z debug proxy::pool spawning new pool conn for 127.0.0.17(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.193655Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.193467Z debug rustls::client::hs:client{request=2} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.193773Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.193245Z debug proxy::pool:client{request=1} starting an idle timeout for connection ConnectionMeta { key: 10407060835540538402, id: 2 } [INFO] [stdout] 2026-03-06T15:30:41.188141Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.194195Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.187970Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.192452Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.194463Z debug rustls::client::hs:client{request=0} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.192428Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.192009Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 10407060835540538402, id: 1 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.193490Z debug proxy::pool spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.200919Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.194531Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.194307Z debug h2::proto::settings:client{request=0}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.194925Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.193047Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.195017Z debug h2::client:client{request=2} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.194581Z debug rustls::client::hs:client{request=0} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.192944Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.194165Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.194322Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.202953Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.193184Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.191834Z debug h2::codec::framed_write:client{request=1}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.193752Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.201210Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.201329Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.204817Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.201485Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.202159Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.202408Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.204957Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.198472Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.203278Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.205316Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.204029Z debug proxy::pool didn't find a connection for key 12160162382132988306, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.201970Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.203039Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.205433Z debug proxy::pool inserting conn mutex for key 12160162382132988306 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.204801Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.206043Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.205131Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/ns/sa/default [INFO] [stdout] 2026-03-06T15:30:41.205552Z debug h2::proto::settings:client{request=0}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.202105Z debug h2::client:client{request=2} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.205639Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.205705Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.205532Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.206328Z debug h2::codec::framed_write:client{request=2} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.202695Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.206322Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.206000Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.204469Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.205398Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.205856Z debug proxy::pool::test client spent 138ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.206822Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.206936Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.206227Z debug proxy::pool spawning new pool conn for 127.0.0.18(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.203752Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.207813Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.207866Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.207933Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.208018Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.208061Z debug proxy::pool didn't find a connection for key 11949129260861191224, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.207079Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.205301Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.207846Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 1 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.208214Z debug proxy::pool didn't find a connection for key 12204414564819847720, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.208128Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.208135Z debug proxy::pool inserting conn mutex for key 11949129260861191224 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.189324Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.207364Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.208809Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.207356Z debug proxy::pool:client{request=2} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.209412Z debug proxy::pool spawning new pool conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.206688Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.209456Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.206987Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.209483Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.208649Z debug proxy::pool inserting conn mutex for key 12204414564819847720 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.208683Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.208955Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.209050Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.209635Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.209646Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.209528Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.209662Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 385506954252831254, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.208950Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.208488Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.209297Z debug rustls::client::hs:client{request=0} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.209867Z debug proxy::pool spawning new pool conn for 127.0.0.19(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.207293Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.210431Z debug rustls::client::tls13:client{request=0} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.210109Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.210936Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.209778Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.207149Z debug proxy::pool:client{request=2} starting an idle timeout for connection ConnectionMeta { key: 14184913346861640675, id: 2 } [INFO] [stdout] 2026-03-06T15:30:41.209778Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.209966Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.209784Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 3 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.211193Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.211201Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.210227Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.210843Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.211239Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 4 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.211222Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.210165Z debug proxy::pool:client{request=1} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.211351Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.210215Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.212281Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.210847Z debug rustls::client::tls13:client{request=0} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.211392Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.212569Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.211987Z debug proxy::pool::test client spent 121ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.212687Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.212818Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.212859Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.212577Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.212878Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.212973Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.213062Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.213166Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.211579Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.212485Z debug rustls::client::hs:client{request=0} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.211724Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(5) } [INFO] [stdout] 2026-03-06T15:30:41.218187Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.212445Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] test tls::lib::tests::test_workload_cert ... ok [INFO] [stdout] 2026-03-06T15:30:41.217793Z debug h2::client:client{request=0} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.216821Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.218465Z debug h2::client:client{request=0} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.212039Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 5 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.218592Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.211497Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.218761Z debug h2::codec::framed_write:client{request=0} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.219386Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.218869Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 6 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.219254Z debug h2::codec::framed_read:client{request=1}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.219254Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.217015Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.219884Z debug h2::proto::settings:client{request=1}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.211920Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.218190Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.220244Z debug proxy::pool:client{request=0} checking in new conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:41531[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 4867941314952592034, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.220468Z debug proxy::pool::test client spent 33ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.217276Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.219861Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 7 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.220820Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.220426Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(5) } [INFO] [stdout] 2026-03-06T15:30:41.220268Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.221044Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.218769Z debug proxy::pool didn't find a connection for key 4875247413344461511, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.218896Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.220705Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.222004Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.220983Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 8 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.221297Z debug proxy::pool:client{request=0} checkout - found mutex for pool key ConnectionMeta { key: 10407060835540538402, id: 3 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.221959Z debug h2::codec::framed_read:client{request=1}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.221403Z debug proxy::pool inserting conn mutex for key 4875247413344461511 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.220028Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.221705Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(5), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.223184Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.221871Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.222755Z debug proxy::pool::test client spent 98ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.221466Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.223437Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.222847Z debug h2::codec::framed_read:client{request=1}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.222330Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.223934Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.222252Z debug pingora_pool::connection:client{request=1} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.223424Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.223639Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.224380Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.222438Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 4867941314952592034, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.224180Z debug proxy::pool:client{request=0} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.213398Z debug proxy::pool::test server drain starting... () [INFO] [stdout] 2026-03-06T15:30:41.224197Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.222127Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 9 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.223809Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.224355Z debug h2::codec::framed_write:client{request=1}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.224098Z debug proxy::pool:client{request=1} connection ConnectionMeta { key: 10407060835540538402, id: 2 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.224858Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.225756Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.225843Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.224960Z debug h2::codec::framed_write:Connection{peer=Server} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(2147483647) } [INFO] [stdout] 2026-03-06T15:30:41.224281Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.225053Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.222972Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.227758Z debug proxy::pool spawning new pool conn for 127.0.0.20(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.224980Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 10 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.228147Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 11 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.228233Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 12 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.224576Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.228665Z debug proxy::h2::client draining outer HBONE connection [INFO] [stdout] 2026-03-06T15:30:41.228672Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Ping { ack: false, payload: [11, 123, 162, 240, 139, 155, 254, 84] } [INFO] [stdout] 2026-03-06T15:30:41.225923Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.225453Z debug h2::codec::framed_write:client{request=1}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.224682Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(5), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.225640Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 10407060835540538402, id: 3 } [INFO] [stdout] 2026-03-06T15:30:41.227816Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.225212Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.229308Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.228315Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 13 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.224053Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.229438Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.228974Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.229786Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.229081Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.228800Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.234058Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.229229Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.234522Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.228932Z debug h2::proto::connection:client{request=1}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.234580Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(2147483647) } [INFO] [stdout] 2026-03-06T15:30:41.235133Z debug rustls::common_state:client{request=1}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.234925Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.228033Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.235650Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.228539Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.224861Z debug proxy::pool:client{request=0} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.234200Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.227903Z debug pingora_pool::connection idle connection is being closed [INFO] [stdout] 2026-03-06T15:30:41.235468Z debug proxy::h2::client:client{request=1} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.236340Z debug pingora_pool::connection evict fd: 0 from key 12787015852653906917 [INFO] [stdout] 2026-03-06T15:30:41.233810Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.228837Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; IO error error=UnexpectedEof [INFO] [stdout] 2026-03-06T15:30:41.229605Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.227821Z debug rustls::webpki::anchors add_parsable_certificates processed 2 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.213327Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.236653Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel1 [INFO] [stdout] 2026-03-06T15:30:41.235932Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.236042Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.236173Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.236420Z debug proxy::pool connection ConnectionMeta { key: 12787015852653906917, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.234371Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.240431Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.229717Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 14 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.240613Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 15 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.240694Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 16 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.240789Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 17 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.234452Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.236797Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.236493Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.241085Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.241291Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.240138Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.241468Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.235121Z debug proxy::pool didn't find a connection for key 10261372043896398520, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.241757Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.237087Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.237323Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.239998Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.242068Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.239924Z debug h2::proto::connection:client{request=0}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.242481Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.241575Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.242640Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.242659Z debug rustls::common_state:client{request=0}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.241875Z debug proxy::pool inserting conn mutex for key 10261372043896398520 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.242391Z debug h2::proto::settings:client{request=0}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.242717Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.242890Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.235276Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Ping { ack: false, payload: [11, 123, 162, 240, 139, 155, 254, 84] } [INFO] [stdout] 2026-03-06T15:30:41.242965Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.243038Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.243068Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Ping { ack: true, payload: [11, 123, 162, 240, 139, 155, 254, 84] } [INFO] [stdout] 2026-03-06T15:30:41.237884Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.243054Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.243144Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.242460Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.241671Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.243426Z debug proxy::pool::test client spent 35ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.242924Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.243207Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.243253Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.243691Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.240971Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.243789Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.243853Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.244000Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.244055Z debug proxy::pool:client{request=2} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 2 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.244290Z debug pingora_pool::connection:client{request=1} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.244352Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.244410Z debug proxy::pool:client{request=1} connection ConnectionMeta { key: 385506954252831254, id: 1 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.244582Z debug proxy::pool:client{request=2} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.243502Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.244868Z debug proxy::pool:client{request=2} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 2 } [INFO] [stdout] 2026-03-06T15:30:41.244812Z debug h2::codec::framed_write:Connection{peer=Server} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.244789Z debug proxy::pool:client{request=2} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.245168Z debug proxy::h2::client:client{request=0} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.245169Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.245322Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.245477Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.245494Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] test proxy::pool::test::read_buffering ... ok [INFO] [stdout] test xds::client::tests::test_json_to_value ... ok [INFO] [stdout] 2026-03-06T15:30:41.245616Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.243188Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.245782Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.240862Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 18 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.244882Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.251423Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.243518Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.251544Z debug proxy::pool spawning new pool conn for 127.0.0.21(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.251629Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.246972Z debug proxy::pool::test inbound serve done Err(hyper::Error(Io, Kind(BrokenPipe))) [INFO] [stdout] 2026-03-06T15:30:41.243912Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.251860Z debug proxy::pool::test client spent 169ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.251850Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.251949Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.243256Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.252103Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252192Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252212Z debug h2::proto::connection:client{request=0}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.251682Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.252269Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252363Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252432Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.251142Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.252501Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252562Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252626Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(7) } [INFO] [stdout] 2026-03-06T15:30:41.252632Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252703Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252788Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252845Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252903Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252951Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.252959Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.251756Z error proxy::pool::test server failed: hyper::Error(Io, Kind(BrokenPipe)) [INFO] [stdout] 2026-03-06T15:30:41.253192Z debug proxy::pool::test client spent 31ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.253132Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.253097Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.253021Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.244708Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.254633Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(5) } [INFO] [stdout] 2026-03-06T15:30:41.252036Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 4867941314952592034, id: 1 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.251689Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.254520Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.254778Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] test proxy::pool::test::single_pool ... ok [INFO] [stdout] 2026-03-06T15:30:41.254850Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.255215Z debug proxy::pool didn't find a connection for key 769168371185029786, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.255194Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.254047Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.255329Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.254907Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.255530Z debug proxy::pool::test client spent 11ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.255573Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.255671Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.255723Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.252460Z debug rustls::common_state:client{request=0}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.255796Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.255857Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.255393Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(5) } [INFO] [stdout] 2026-03-06T15:30:41.256109Z debug proxy::h2::client:client{request=0} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.255963Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.255940Z debug proxy::pool:client{request=3} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 3 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.256213Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.253666Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 19 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.256280Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.256377Z debug proxy::pool:client{request=3} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.256402Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 20 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.256498Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 21 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.256575Z debug proxy::pool:client{request=3} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.256727Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:41531[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.256368Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.256925Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.256956Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.257056Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.257073Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.253306Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 10407060835540538402, id: 4 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.256590Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 22 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.253088Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(7) } [INFO] [stdout] 2026-03-06T15:30:41.243409Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Ping { ack: true, payload: [11, 123, 162, 240, 139, 155, 254, 84] } [INFO] [stdout] 2026-03-06T15:30:41.255270Z debug proxy::pool inserting conn mutex for key 769168371185029786 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.256929Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.251664Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.260866Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 4867941314952592034, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.261263Z debug h2::codec::framed_write:Connection{peer=Server} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.261592Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.262017Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.261427Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.262314Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.262681Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.261883Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.261146Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.260910Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.262878Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.260938Z debug proxy::pool:client{request=1} starting an idle timeout for connection ConnectionMeta { key: 4867941314952592034, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.261345Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.266089Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.261587Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 23 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.266453Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.264157Z debug rustls::webpki::anchors:xds{id=1} add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.265936Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.262257Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.269952Z debug h2::proto::settings:client{request=2}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.262419Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.270011Z debug xds::client:xds{id=1} XDS header added: header_foo=foo [INFO] [stdout] 2026-03-06T15:30:41.270081Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.270100Z debug xds::client:xds{id=1} XDS header added: header_bar=bar [INFO] [stdout] 2026-03-06T15:30:41.270066Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.270136Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.266535Z debug proxy::pool::test client spent 14ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.270366Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.270313Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.266395Z debug proxy::pool spawning new pool conn for 127.0.0.22(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.266675Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 24 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.266080Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 10407060835540538402, id: 3 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.270602Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.270519Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.271021Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.270510Z debug proxy::pool:client{request=3} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 3 } [INFO] [stdout] 2026-03-06T15:30:41.263161Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.270175Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.271597Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.271652Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.270797Z debug proxy::pool:client{request=0} checkout - found mutex for pool key ConnectionMeta { key: 4867941314952592034, id: 2 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.270635Z debug pingora_pool::connection:client{request=2} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.271832Z debug proxy::pool:client{request=0} re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:41531[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.271128Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 25 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.272123Z debug proxy::pool:client{request=0} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.271371Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.271685Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.270696Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.271818Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.271181Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.272704Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.267256Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.271914Z debug pingora_pool::connection:client{request=1} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.273236Z debug proxy::pool didn't find a connection for key 17918551596502831353, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.272868Z debug proxy::pool:client{request=1} connection ConnectionMeta { key: 4867941314952592034, id: 1 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.273459Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.273455Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.272475Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.273702Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.273683Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.273428Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 4867941314952592034, id: 2 } [INFO] [stdout] 2026-03-06T15:30:41.272280Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 26 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.274142Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.274252Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 27 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.274335Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 28 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.274410Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 29 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.274463Z debug rustls::webpki::anchors:xds{id=1} add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.274569Z debug proxy::pool::test client spent 118ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.274629Z debug xds::client:xds{id=1} XDS header added: header_foo=foo [INFO] [stdout] 2026-03-06T15:30:41.270395Z debug hyper_util::client::legacy::connect::http:xds{id=1} connecting to 127.0.0.1:40201 [INFO] [stdout] 2026-03-06T15:30:41.273545Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.273766Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.274912Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.274969Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275021Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275072Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.268116Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.275118Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275165Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275159Z debug hyper_util::client::legacy::connect::http:xds{id=1} connected to 127.0.0.1:40201 [INFO] [stdout] 2026-03-06T15:30:41.275216Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275261Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275303Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275350Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275398Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275446Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275492Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275536Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275592Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275639Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275289Z debug rustls::client::hs:xds{id=1} No cached session for IpAddress(V4(Ipv4Addr([127, 0, 0, 1]))) [INFO] [stdout] 2026-03-06T15:30:41.275669Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275715Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.275060Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.272047Z debug proxy::pool::test server drain done [INFO] [stdout] 2026-03-06T15:30:41.271019Z debug proxy::pool:client{request=1} starting an idle timeout for connection ConnectionMeta { key: 10407060835540538402, id: 4 } [INFO] [stdout] 2026-03-06T15:30:41.274485Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 30 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.273356Z debug proxy::pool inserting conn mutex for key 17918551596502831353 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.274711Z debug proxy::pool:client{request=3} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 3 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.275766Z debug rustls::client::hs:xds{id=1} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.274725Z debug xds::client:xds{id=1} XDS header added: header_bar=bar [INFO] [stdout] 2026-03-06T15:30:41.275842Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.275953Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.276320Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 31 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.276023Z debug proxy::pool:client{request=0} checkout - found mutex for pool key ConnectionMeta { key: 498255846233636669, id: 1 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.276594Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.276657Z debug proxy::pool:client{request=0} checked out broken connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44727[spiffe://cluster.local/ns/istio-system/sa/ztunnel], dropping it [INFO] [stdout] 2026-03-06T15:30:41.276632Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 32 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.276791Z debug proxy::pool:client{request=0} new connection needed for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44727[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.276788Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 33 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.276878Z debug proxy::pool:client{request=0} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44727[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.276985Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.276409Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.277090Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.277178Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 34 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.277196Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 498255846233636669, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.276019Z debug hyper_util::client::legacy::connect::http:xds{id=1} connecting to 127.0.0.1:43191 [INFO] [stdout] 2026-03-06T15:30:41.277418Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 35 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.276253Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.276185Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.277024Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel8 [INFO] [stdout] 2026-03-06T15:30:41.277257Z debug proxy::pool spawning new pool conn for 127.0.0.23(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.294622Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.277254Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.277580Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 36 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.278246Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.300039Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 37 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.300205Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 38 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.300334Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 39 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.300452Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 40 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.300557Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.277486Z debug pingora_pool::connection:client{request=2} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.300775Z debug proxy::pool:client{request=2} connection ConnectionMeta { key: 14184913346861640675, id: 2 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.300563Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 41 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.281164Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.301012Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 42 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.301159Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 43 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.301276Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 44 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.301392Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 45 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.301511Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 46 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.301634Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 47 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.277745Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.301770Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 48 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.301994Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 49 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.302125Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 50 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.283644Z debug rustls::webpki::anchors:xds{id=1} add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.302260Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 51 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.302416Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 52 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.302537Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 53 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.302667Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 54 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.302804Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 55 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.302928Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 56 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.301857Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.277796Z debug hyper_util::client::legacy::connect::http:xds{id=1} connected to 127.0.0.1:43191 [INFO] [stdout] 2026-03-06T15:30:41.303045Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 57 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.303862Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 58 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.302032Z debug rustls::client::hs:xds{id=1} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.303965Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 59 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.304116Z debug rustls::client::tls13:xds{id=1} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.276997Z debug state:client{request=0} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.304106Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 60 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.276154Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.304253Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 61 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.304375Z debug rustls::client::tls13:xds{id=1} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.276505Z debug proxy::pool:client{request=3} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.301755Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.301426Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.304530Z debug proxy::pool:client{request=3} checked out connection for ConnectionMeta { key: 14184913346861640675, id: 3 } is now at max streamcount; removing from pool [INFO] [stdout] 2026-03-06T15:30:41.303933Z debug rustls::client::hs:xds{id=1} No cached session for IpAddress(V4(Ipv4Addr([127, 0, 0, 1]))) [INFO] [stdout] 2026-03-06T15:30:41.304390Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 62 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.304390Z debug rustls::client::tls13 Got CertificateRequest CertificateRequestPayloadTls13 { context: , extensions: CertificateRequestExtensions { signature_algorithms: [ECDSA_NISTP384_SHA384, ECDSA_NISTP256_SHA256, ECDSA_NISTP521_SHA512, ED25519, RSA_PSS_SHA512, RSA_PSS_SHA384, RSA_PSS_SHA256, RSA_PKCS1_SHA512, RSA_PKCS1_SHA384, RSA_PKCS1_SHA256], authority_names: [DistinguishedName(301831163014060355040a0c0d636c75737465722e6c6f63616c), DistinguishedName(301831163014060355040a0c0d636c75737465722e6c6f63616c)], .. } } [INFO] [stdout] 2026-03-06T15:30:41.304727Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(9) } [INFO] [stdout] 2026-03-06T15:30:41.282822Z debug pingora_pool::connection:client{request=1} idle connection is being evicted [INFO] [stdout] 2026-03-06T15:30:41.277595Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.303666Z debug proxy::pool didn't find a connection for key 1895892462465854785, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.304241Z debug state:client{request=0} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.304857Z debug proxy::pool inserting conn mutex for key 1895892462465854785 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.304653Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(7) } [INFO] [stdout] 2026-03-06T15:30:41.304953Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.305082Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.272071Z debug proxy::pool:client{request=2} connection ConnectionMeta { key: 385506954252831254, id: 2 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.305058Z debug proxy::pool spawning new pool conn for 127.0.0.24(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.304938Z debug pingora_pool::connection:client{request=1} evict fd: 1 from key 473791882615185783 [INFO] [stdout] 2026-03-06T15:30:41.304701Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 63 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.305310Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.305294Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 64 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.305428Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 65 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.304628Z debug proxy::pool:client{request=3} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.305366Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.305483Z debug proxy::pool:client{request=1} connection ConnectionMeta { key: 473791882615185783, id: 1 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.305549Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 66 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.304857Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.305810Z debug h2::codec::framed_write:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.305871Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.305911Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.305964Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.305980Z debug proxy::pool didn't find a connection for key 5602206303248432968, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.306012Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.305721Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 67 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.306181Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.306211Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.305426Z debug rustls::client::hs:xds{id=1} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.306482Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(7) } [INFO] [stdout] 2026-03-06T15:30:41.279260Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.306134Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 68 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.306658Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(5) } [INFO] [stdout] 2026-03-06T15:30:41.305951Z debug h2::proto::connection:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.306895Z debug rustls::common_state:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.306961Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.306653Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 69 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.306061Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.307208Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.307249Z debug proxy::h2::client done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.307131Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 70 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.307400Z debug rustls::client::common Attempting client auth [INFO] [stdout] 2026-03-06T15:30:41.307405Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(9) } [INFO] [stdout] 2026-03-06T15:30:41.307474Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.307389Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 71 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.307655Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.304654Z debug rustls::client::hs:xds{id=1} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.307968Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.307676Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 72 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.308130Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 73 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.308172Z debug proxy::pool::test client spent 52ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.308177Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.308247Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.308295Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 74 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.309123Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 75 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.307664Z debug proxy::pool::test client spent 54ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.309242Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 76 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.309429Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 77 }, waiting for writelock [INFO] [stdout] test tls::certificate::test::multi_root ... ok [INFO] [stdout] 2026-03-06T15:30:41.309550Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 78 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.309197Z debug proxy::pool:client{request=4} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 4 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.309799Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.309990Z debug proxy::pool::test client spent 39ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.308288Z debug h2::client:xds{id=1} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.307326Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.309681Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 79 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.310336Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.310298Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 4867941314952592034, id: 3 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.310330Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 80 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.310433Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 81 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.309913Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(5) } [INFO] [stdout] 2026-03-06T15:30:41.309931Z debug proxy::pool:client{request=4} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.310749Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.310802Z debug proxy::pool:client{request=4} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.310529Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 82 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.310113Z debug pingora_pool::connection:client{request=3} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.307273Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.309815Z debug rustls::client::hs:client{request=0} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.310970Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 83 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.311055Z debug proxy::pool:client{request=4} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 4 } [INFO] [stdout] 2026-03-06T15:30:41.311089Z debug proxy::pool:client{request=3} connection ConnectionMeta { key: 385506954252831254, id: 3 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.310870Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 4867941314952592034, id: 2 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.311216Z debug h2::client:xds{id=1} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.310472Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:41531[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.311324Z debug h2::codec::framed_write:xds{id=1} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.311344Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.311408Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 84 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.311422Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.311488Z debug proxy::pool:client{request=1} starting an idle timeout for connection ConnectionMeta { key: 4867941314952592034, id: 3 } [INFO] [stdout] 2026-03-06T15:30:41.311808Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.311859Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.306045Z debug proxy::pool inserting conn mutex for key 5602206303248432968 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.311960Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.311548Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 85 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.309491Z debug proxy::pool:client{request=2} checkout - found mutex for pool key ConnectionMeta { key: 10407060835540538402, id: 5 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.312009Z debug proxy::pool spawning new pool conn for 127.0.0.25(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.311875Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.311594Z debug hyper_util::client::legacy::pool:xds{id=1} pooling idle connection for ("https", 127.0.0.1:40201) [INFO] [stdout] 2026-03-06T15:30:41.312130Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.311137Z debug rustls::client::hs:client{request=0} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.312168Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 86 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.312621Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 87 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.312716Z debug pingora_pool::connection:client{request=1} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.314979Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.315103Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 88 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.312209Z debug proxy::pool:client{request=2} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.315385Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 89 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.315670Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 90 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.312619Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.312780Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.306490Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.315812Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 91 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.315995Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 92 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.316081Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(9) } [INFO] [stdout] 2026-03-06T15:30:41.316139Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.312305Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.315510Z debug proxy::pool:client{request=2} checked out connection for ConnectionMeta { key: 10407060835540538402, id: 5 } is now at max streamcount; removing from pool [INFO] [stdout] 2026-03-06T15:30:41.315259Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel13 [INFO] [stdout] 2026-03-06T15:30:41.315415Z debug proxy::pool:client{request=1} connection ConnectionMeta { key: 10407060835540538402, id: 4 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.315755Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.319220Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.security.Authorization" [INFO] [stdout] 2026-03-06T15:30:41.315950Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.319790Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.319930Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.312166Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.320069Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.320215Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.320258Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.316586Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.302603Z debug xds::client:xds{id=1} XDS header added: header_foo=foo [INFO] [stdout] 2026-03-06T15:30:41.316102Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 93 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.316393Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.311721Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.320562Z debug xds::client:xds{id=1} XDS header added: header_bar=bar [INFO] [stdout] 2026-03-06T15:30:41.320545Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 94 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.320660Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.320710Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 95 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.316439Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.320870Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 96 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.320998Z debug hyper_util::client::legacy::connect::http:xds{id=1} connecting to 127.0.0.1:45409 [INFO] [stdout] 2026-03-06T15:30:41.312896Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.320675Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(9) } [INFO] [stdout] 2026-03-06T15:30:41.321002Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 97 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321152Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 98 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321242Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 99 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321332Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.320823Z debug proxy::pool::test client spent 11ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.321330Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 100 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321439Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.321455Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 101 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321553Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.321549Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 102 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321645Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.321650Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 103 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321596Z debug proxy::pool:client{request=5} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 5 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321848Z debug hyper_util::client::legacy::connect::http:xds{id=1} connected to 127.0.0.1:45409 [INFO] [stdout] 2026-03-06T15:30:41.320811Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.316281Z debug proxy::pool:client{request=2} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.321762Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 104 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321999Z debug rustls::client::hs:xds{id=1} No cached session for IpAddress(V4(Ipv4Addr([127, 0, 0, 1]))) [INFO] [stdout] 2026-03-06T15:30:41.321009Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.322021Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 105 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.322153Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.321966Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.322185Z debug proxy::pool:client{request=5} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.322151Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 106 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.322250Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.322300Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 107 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.322448Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.322127Z debug pingora_pool::connection:client{request=4} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.322434Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 108 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.322376Z debug proxy::pool:client{request=5} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 5 } [INFO] [stdout] 2026-03-06T15:30:41.322590Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.322575Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 109 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.322682Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.322462Z debug proxy::pool:client{request=5} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.322752Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.322571Z debug proxy::pool:client{request=4} connection ConnectionMeta { key: 385506954252831254, id: 4 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.322382Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.322963Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.323070Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 110 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323101Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(11) } [INFO] [stdout] 2026-03-06T15:30:41.323157Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 111 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323174Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.323239Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 112 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323300Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 113 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323380Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 114 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.321724Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.323464Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 115 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323530Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 116 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323563Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.323618Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 117 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323674Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 118 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323700Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.323768Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 119 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323831Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 120 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323920Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 121 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.323997Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.324089Z debug proxy::pool::test client spent 49ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.324159Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.324278Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.324389Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.322130Z debug rustls::client::hs:xds{id=1} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.321882Z debug proxy::pool didn't find a connection for key 13114846752334268514, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.324317Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.326853Z debug proxy::pool inserting conn mutex for key 13114846752334268514 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.326893Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.326941Z debug proxy::pool spawning new pool conn for 127.0.0.26(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.326976Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.327004Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.327038Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.327106Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.327320Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.327380Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.327431Z debug proxy::pool didn't find a connection for key 9945544581471895133, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.327427Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.327453Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.327486Z debug proxy::pool inserting conn mutex for key 9945544581471895133 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.327525Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.327574Z debug proxy::pool spawning new pool conn for 127.0.0.27(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.327634Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.320467Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.327680Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.327782Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(7) } [INFO] [stdout] 2026-03-06T15:30:41.327809Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.327829Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(11) } [INFO] [stdout] 2026-03-06T15:30:41.328000Z debug proxy::pool didn't find a connection for key 17527473550082483247, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.328075Z debug proxy::pool inserting conn mutex for key 17527473550082483247 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.328122Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.328126Z debug proxy::pool::test client spent 6ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.328163Z debug proxy::pool spawning new pool conn for 127.0.0.28(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.328206Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.323371Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.328324Z debug proxy::pool:client{request=6} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 6 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.328345Z debug rustls::client::hs:xds{id=1} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.328358Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(11) } [INFO] [stdout] 2026-03-06T15:30:41.328095Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.328462Z debug rustls::client::tls13:xds{id=1} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.328490Z debug proxy::pool:client{request=6} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.328556Z debug proxy::pool::test client spent 19ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.327618Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.328706Z debug proxy::pool:client{request=3} checkout - found mutex for pool key ConnectionMeta { key: 10407060835540538402, id: 6 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.322987Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.328820Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.328830Z debug proxy::pool:client{request=3} new connection needed for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.328869Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.328872Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(11) } [INFO] [stdout] 2026-03-06T15:30:41.328916Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.328921Z debug proxy::pool:client{request=3} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:44157[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.328955Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.328998Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.329002Z debug state:client{request=3} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.329038Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.328673Z debug proxy::pool::test client spent 18ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.329070Z debug pingora_pool::connection:client{request=5} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.321878Z debug rustls::client::hs:client{request=0} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.329081Z debug state:client{request=3} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.328333Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.329079Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.324005Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 122 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.329141Z debug proxy::pool:client{request=6} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 6 } [INFO] [stdout] 2026-03-06T15:30:41.333751Z debug proxy::pool:client{request=5} connection ConnectionMeta { key: 385506954252831254, id: 5 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.328601Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(7) } [INFO] [stdout] 2026-03-06T15:30:41.329160Z debug h2::proto::connection:client{request=0}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.324326Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.328215Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.340707Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.330529Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.310942Z debug rustls::client::hs:xds{id=1} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.333810Z debug rustls::client::tls13:client{request=0} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.328670Z debug proxy::pool:client{request=6} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.341174Z debug rustls::client::tls13:xds{id=1} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.334036Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.341290Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.337040Z debug rustls::common_state:client{request=0}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.341359Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.341397Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.334176Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 123 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.341557Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 124 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.341655Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 125 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.341713Z debug proxy::h2::client:client{request=0} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.341762Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 126 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.341843Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 127 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.341854Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.333931Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.342006Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.342039Z debug h2::proto::connection:client{request=2}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.342161Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.342199Z debug rustls::common_state:client{request=2}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.342299Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.342435Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.342445Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.342471Z debug proxy::h2::client:client{request=2} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.342697Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.342727Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.342776Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.342853Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.342908Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.343001Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.340525Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.343073Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.343048Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.337268Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.337533Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.343156Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.343293Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(13) } [INFO] [stdout] 2026-03-06T15:30:41.343205Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel11 [INFO] [stdout] 2026-03-06T15:30:41.344809Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.341928Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 128 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.345593Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.344529Z debug rustls::client::tls13:xds{id=1} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.344631Z debug rustls::client::hs:client{request=3} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.345632Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.345249Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(7), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.346827Z debug rustls::client::tls13:client{request=0} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.345642Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 129 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.346965Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 130 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.347052Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.344365Z debug proxy::pool didn't find a connection for key 12249195444314390149, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.345809Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.347137Z debug proxy::pool inserting conn mutex for key 12249195444314390149 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.347156Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.347192Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.347241Z debug proxy::pool spawning new pool conn for 127.0.0.29(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.347304Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.347328Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(13) } [INFO] [stdout] 2026-03-06T15:30:41.345780Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.347416Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: ["*"], resource_names_unsubscribe: ["*"], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.347620Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.347654Z debug proxy::pool::test client spent 19ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.345931Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.332803Z debug rustls::client::tls13:xds{id=1} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.347724Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(5), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.347895Z debug proxy::pool:client{request=7} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 7 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.347930Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.347940Z debug rustls::client::hs:xds{id=1} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.348042Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.348099Z debug proxy::pool:client{request=0} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 4 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.346487Z debug rustls::webpki::anchors:xds{id=1} add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.348191Z debug pingora_pool::connection:client{request=6} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.348215Z debug proxy::pool:client{request=0} new connection needed for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.348283Z debug proxy::pool:client{request=6} connection ConnectionMeta { key: 385506954252831254, id: 6 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.348323Z debug proxy::pool:client{request=0} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.348335Z debug xds::client:xds{id=1} XDS header added: header_foo=foo [INFO] [stdout] 2026-03-06T15:30:41.348414Z debug state:client{request=0} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.348056Z debug proxy::pool:client{request=7} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.348453Z debug rustls::client::hs:client{request=3} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.348482Z debug state:client{request=0} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.348637Z debug proxy::pool:client{request=7} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 7 } [INFO] [stdout] 2026-03-06T15:30:41.348805Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.348826Z info xds::client::tests received request: DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: ["*"], resource_names_unsubscribe: ["*"], initial_resource_versions: {}, response_nonce: "", error_detail: None } [INFO] [stdout] 2026-03-06T15:30:41.348878Z debug proxy::pool:client{request=7} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.348902Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349034Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] test tls::lib::tests::cert_expiration ... ok [INFO] [stdout] 2026-03-06T15:30:41.349138Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.348418Z debug xds::client:xds{id=1} XDS header added: header_bar=bar [INFO] [stdout] 2026-03-06T15:30:41.349194Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.347063Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 131 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.349259Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349321Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.347668Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.349373Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349426Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349481Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349488Z info xds::client::tests received request: DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None } [INFO] [stdout] 2026-03-06T15:30:41.349536Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349553Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.349601Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349327Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 132 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.349640Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349698Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.347352Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.346959Z debug rustls::client::hs:client{request=0} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.349772Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349821Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.349867Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.349133Z debug proxy::h2::client:client{request=0} draining outer HBONE connection [INFO] [stdout] 2026-03-06T15:30:41.350017Z debug pingora_pool::connection:client{request=1} idle connection is being closed [INFO] [stdout] 2026-03-06T15:30:41.350146Z debug pingora_pool::connection:client{request=1} evict fd: 3 from key 4867941314952592034 [INFO] [stdout] 2026-03-06T15:30:41.350211Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.350288Z debug proxy::pool:client{request=1} connection ConnectionMeta { key: 4867941314952592034, id: 3 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.350351Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(15) } [INFO] [stdout] 2026-03-06T15:30:41.350567Z debug hyper_util::client::legacy::connect::http:xds{id=1} connecting to 127.0.0.1:35403 [INFO] [stdout] 2026-03-06T15:30:41.350621Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(7), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.350700Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.349882Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.350837Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.350877Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(15) } [INFO] [stdout] 2026-03-06T15:30:41.350901Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.350983Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.351037Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.351090Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.351116Z debug proxy::pool::test client spent 3ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.351244Z debug rustls::client::hs:xds{id=1} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.351309Z debug proxy::pool:client{request=8} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 8 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.351337Z debug hyper_util::client::legacy::connect::http:xds{id=1} connected to 127.0.0.1:35403 [INFO] [stdout] 2026-03-06T15:30:41.351459Z debug proxy::pool:client{request=8} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.350772Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(5), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.351624Z debug proxy::pool:client{request=8} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.351139Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.351716Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.351636Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; IO error error=UnexpectedEof [INFO] [stdout] 2026-03-06T15:30:41.348884Z debug h2::client:xds{id=1} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.351920Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.345460Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.349704Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 133 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.353932Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 134 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.354028Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 135 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.351478Z debug rustls::client::hs:xds{id=1} No cached session for IpAddress(V4(Ipv4Addr([127, 0, 0, 1]))) [INFO] [stdout] 2026-03-06T15:30:41.352052Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354128Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 136 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.354186Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.351958Z debug proxy::pool didn't find a connection for key 16687963005405547643, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.350604Z debug h2::client:client{request=0} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.354246Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354261Z debug proxy::pool inserting conn mutex for key 16687963005405547643 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.352241Z debug proxy::pool:client{request=8} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 8 } [INFO] [stdout] 2026-03-06T15:30:41.354281Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354313Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.354338Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.351115Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.354358Z debug h2::client:client{request=0} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.354393Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354445Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354493Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354542Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354606Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354664Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354662Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.354721Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354797Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354850Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354196Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 137 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.354914Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354966Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.354978Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 138 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.355015Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.355026Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.355051Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 139 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.355118Z debug rustls::client::hs:xds{id=1} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.355153Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 140 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.355222Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 141 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.355209Z debug h2::client:xds{id=1} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.354346Z debug proxy::pool spawning new pool conn for 127.0.0.30(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.352417Z debug h2::client:xds{id=1} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.365100Z debug h2::client:xds{id=1} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.365237Z debug h2::codec::framed_write:xds{id=1} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.352183Z debug pingora_pool::connection:client{request=7} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.365505Z debug proxy::pool:client{request=7} connection ConnectionMeta { key: 385506954252831254, id: 7 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.365588Z debug hyper_util::client::legacy::pool:xds{id=1} pooling idle connection for ("https", 127.0.0.1:43191) [INFO] [stdout] 2026-03-06T15:30:41.351981Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.355593Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.355381Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.352080Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.365901Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.354846Z debug rustls::client::hs:client{request=3} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.363068Z debug h2::codec::framed_write:xds{id=1} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.354433Z debug h2::codec::framed_write:client{request=0} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.365991Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(17) } [INFO] [stdout] 2026-03-06T15:30:41.366046Z debug rustls::client::tls13:client{request=3} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.355691Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.366176Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.366231Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.366280Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.365973Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.366332Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.366333Z debug rustls::client::tls13:client{request=3} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.366348Z debug hyper_util::client::legacy::pool:xds{id=1} pooling idle connection for ("https", 127.0.0.1:45409) [INFO] [stdout] 2026-03-06T15:30:41.366382Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.366413Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.366467Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.355286Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 142 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.366521Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.366540Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.366590Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.366587Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.366615Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 143 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.366685Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 144 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.366800Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 145 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.366819Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 498255846233636669, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.366890Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 146 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.366910Z debug proxy::pool::test client spent 15ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.366915Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.367065Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.367086Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.366277Z debug proxy::pool:client{request=0} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.367097Z debug proxy::pool:client{request=9} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 9 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.366981Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 147 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.367162Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.367218Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 148 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.367258Z debug proxy::pool:client{request=9} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.366074Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.355445Z debug h2::proto::settings:xds{id=1}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.367073Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.367425Z debug proxy::pool:client{request=9} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.367462Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.367593Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.367635Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(17) } [INFO] [stdout] 2026-03-06T15:30:41.367646Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.367807Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.367895Z debug rustls::client::hs:xds{id=1} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.368006Z debug xds::client:xds{id=1} connected established [INFO] [stdout] 2026-03-06T15:30:41.366585Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.368106Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.367129Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.368159Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.368207Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.368255Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.368303Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.368316Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.367269Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.368502Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.368592Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.368617Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.368675Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.366813Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.366460Z debug rustls::client::hs:client{request=3} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.368028Z debug rustls::client::tls13:xds{id=1} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.368982Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel4 [INFO] [stdout] 2026-03-06T15:30:41.368989Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.368146Z debug pingora_pool::connection:client{request=8} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.368352Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.369368Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.371925Z debug proxy::pool:client{request=8} connection ConnectionMeta { key: 385506954252831254, id: 8 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.369902Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(19) } [INFO] [stdout] 2026-03-06T15:30:41.365888Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.367312Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 149 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.368115Z info xds::client:xds{id=1} Stream established [INFO] [stdout] 2026-03-06T15:30:41.370106Z debug proxy::pool didn't find a connection for key 11463953853427422205, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.377872Z debug proxy::pool inserting conn mutex for key 11463953853427422205 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.369987Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.372067Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.369207Z debug rustls::client::tls13:xds{id=1} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.379360Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.372092Z debug proxy::pool:client{request=9} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 9 } [INFO] [stdout] 2026-03-06T15:30:41.379427Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.379607Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.379650Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.374468Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.376229Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 150 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.371984Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.379887Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 151 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.379487Z debug rustls::client::hs:xds{id=1} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.374242Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.386110Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.367291Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.security.Authorization" [INFO] [stdout] 2026-03-06T15:30:41.377932Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.370249Z debug h2::client:client{request=3} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.388001Z debug h2::client:client{request=3} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.388150Z debug h2::codec::framed_write:client{request=3} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.380168Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(19) } [INFO] [stdout] 2026-03-06T15:30:41.388453Z debug proxy::pool:client{request=3} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.380277Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.388871Z debug h2::codec::framed_read:client{request=3}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.388884Z debug h2::proto::settings:client{request=0}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.386347Z debug h2::client:xds{id=1} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.379704Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.388991Z debug h2::codec::framed_write:client{request=3}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.388479Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.386223Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.389098Z debug h2::codec::framed_write:client{request=3}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.380123Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.389204Z debug h2::codec::framed_write:client{request=3}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.389295Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.380089Z debug proxy::pool::test client spent 13ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.389525Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.389624Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.389686Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.389764Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.378103Z info xds::client:xds{id=1} received on demand request type.googleapis.com/istio.workload.Address/foo0 [INFO] [stdout] 2026-03-06T15:30:41.379979Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 152 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.391922Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 153 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.389872Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.389907Z debug proxy::pool spawning new pool conn for 127.0.0.31(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.402178Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.402279Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.390012Z debug h2::client:xds{id=1} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.402463Z debug h2::codec::framed_write:xds{id=1} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.390372Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.390201Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.385632Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.security.Authorization" [INFO] [stdout] 2026-03-06T15:30:41.392041Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 154 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.402816Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 155 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.402823Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.390126Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.392439Z debug xds::client:xds{id=1} sending request type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.393685Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.403113Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel14 [INFO] [stdout] 2026-03-06T15:30:41.403005Z debug proxy::pool didn't find a connection for key 5923584172921063344, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.404530Z debug proxy::pool inserting conn mutex for key 5923584172921063344 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.404606Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.402911Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 156 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.404668Z debug proxy::pool spawning new pool conn for 127.0.0.32(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.403191Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.399921Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.403237Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.404722Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.389052Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.404779Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.404814Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.403547Z debug hyper_util::client::legacy::pool:xds{id=1} pooling idle connection for ("https", 127.0.0.1:35403) [INFO] [stdout] 2026-03-06T15:30:41.405020Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.405105Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.405191Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.405245Z debug xds::client:xds{id=1} sending initial request resources=0 type_url="type.googleapis.com/istio.security.Authorization" [INFO] [stdout] 2026-03-06T15:30:41.390124Z debug proxy::pool:client{request=10} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 10 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.405447Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.403848Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.405472Z debug proxy::pool:client{request=10} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.403336Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.405640Z debug proxy::pool:client{request=3} starting an idle timeout for connection ConnectionMeta { key: 10407060835540538402, id: 6 } [INFO] [stdout] 2026-03-06T15:30:41.405757Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.405807Z debug pingora_pool::connection:client{request=9} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.405887Z debug proxy::pool::test client spent 129ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.405907Z debug proxy::pool:client{request=9} connection ConnectionMeta { key: 385506954252831254, id: 9 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.405937Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.406036Z debug proxy::pool:client{request=10} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 10 } [INFO] [stdout] 2026-03-06T15:30:41.404836Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.406096Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.406137Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.406183Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.406800Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.405254Z debug proxy::pool didn't find a connection for key 6334511224440449612, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.405599Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.406063Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.406957Z debug proxy::pool inserting conn mutex for key 6334511224440449612 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.406219Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.406983Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.407002Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.407028Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.405652Z debug proxy::pool:client{request=10} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.407081Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.407124Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.407163Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.407200Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.407245Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.407285Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.407035Z debug proxy::pool spawning new pool conn for 127.0.0.33(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.406274Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.407401Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.406511Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 157 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.407459Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.407520Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 158 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.407573Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.407612Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 159 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.407675Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 160 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.407780Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 161 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.407865Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 162 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.407948Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 163 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.408167Z debug proxy::pool didn't find a connection for key 12979277504903559511, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.407328Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.407451Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.408163Z debug pingora_pool::connection:client{request=0} idle connection is being closed [INFO] [stdout] 2026-03-06T15:30:41.408290Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.404132Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.407028Z debug test_helpers::xds request DeltaDiscoveryRequest { node: None, type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: ["foo0"], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.412104Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.408029Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 164 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.408408Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.408227Z debug proxy::pool inserting conn mutex for key 12979277504903559511 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.409083Z debug proxy::h2::client:client{request=0} draining outer HBONE connection [INFO] [stdout] 2026-03-06T15:30:41.412296Z info xds::client::tests received request: DeltaDiscoveryRequest { node: None, type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: ["foo0"], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None } [INFO] [stdout] 2026-03-06T15:30:41.415924Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.416005Z debug proxy::pool spawning new pool conn for 127.0.0.34(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.416082Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.416137Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.416848Z debug proxy::pool didn't find a connection for key 7735253954940383155, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.417001Z debug proxy::pool inserting conn mutex for key 7735253954940383155 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.417088Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.417161Z debug proxy::pool spawning new pool conn for 127.0.0.35(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.417246Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.417326Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.409173Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.417833Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.417836Z debug proxy::pool didn't find a connection for key 7374296357216183288, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.417891Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.417975Z debug proxy::pool inserting conn mutex for key 7374296357216183288 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.418013Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.418318Z debug proxy::pool spawning new pool conn for 127.0.0.36(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.418442Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.418475Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.418949Z debug proxy::pool didn't find a connection for key 11205033153775527865, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.419069Z debug proxy::pool inserting conn mutex for key 11205033153775527865 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.419155Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.419233Z debug proxy::pool spawning new pool conn for 127.0.0.37(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.419304Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.419337Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.418275Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.412249Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 165 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.419912Z debug proxy::pool didn't find a connection for key 11190283184882302700, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.412952Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.412231Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.405887Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.420102Z debug proxy::pool inserting conn mutex for key 11190283184882302700 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.420683Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.420224Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.420783Z debug proxy::pool spawning new pool conn for 127.0.0.38(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.420901Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.420963Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.421346Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.412520Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.412388Z debug pingora_pool::connection:client{request=0} evict fd: 1 from key 498255846233636669 [INFO] [stdout] 2026-03-06T15:30:41.420551Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.421232Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.419825Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.421534Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.419897Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 166 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.421928Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.422144Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 167 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.422136Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.421364Z debug proxy::pool didn't find a connection for key 15416495286242971167, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.422219Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 168 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.422288Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 169 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.422378Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 170 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.422470Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 171 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.422399Z debug proxy::pool inserting conn mutex for key 15416495286242971167 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.422568Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 172 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.423217Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(21) } [INFO] [stdout] 2026-03-06T15:30:41.421552Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 498255846233636669, id: 1 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.421909Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.422642Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.422711Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.421890Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.423666Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.423814Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 173 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.423504Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.423498Z debug proxy::pool spawning new pool conn for 127.0.0.39(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.423655Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.423926Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] test proxy::pool::test::server_goaway ... ok [INFO] [stdout] 2026-03-06T15:30:41.424026Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.423101Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.423889Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.424985Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(21) } [INFO] [stdout] 2026-03-06T15:30:41.406386Z debug h2::codec::framed_read:client{request=3}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.424084Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.423889Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.424868Z debug proxy::pool::test client spent 35ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.425118Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.425382Z debug proxy::pool:client{request=11} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 11 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.425234Z debug h2::proto::settings:client{request=3}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.425980Z debug h2::codec::framed_read:client{request=3}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.425307Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.425675Z debug proxy::pool:client{request=11} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.423904Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 174 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.425568Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.425722Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.425797Z debug pingora_pool::connection:client{request=10} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.426431Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel3 [INFO] [stdout] 2026-03-06T15:30:41.425208Z debug h2::proto::settings:xds{id=1}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.426369Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 175 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.426578Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 176 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.424550Z debug proxy::pool didn't find a connection for key 15506702836588009732, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.425314Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.426376Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.426665Z debug proxy::pool inserting conn mutex for key 15506702836588009732 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.426810Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.426625Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.426592Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.426240Z debug proxy::pool:client{request=11} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.426542Z debug proxy::pool:client{request=11} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 11 } [INFO] [stdout] 2026-03-06T15:30:41.427006Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.426787Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.426653Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 177 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.427450Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.427083Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.426423Z debug proxy::pool:client{request=10} connection ConnectionMeta { key: 385506954252831254, id: 10 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.427406Z debug proxy::pool spawning new pool conn for 127.0.0.40(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.426775Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.427852Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.427873Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.427568Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.427911Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.427247Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.427718Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.427495Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.428410Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.426241Z debug h2::codec::framed_read:client{request=3}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.429096Z debug proxy::pool didn't find a connection for key 1895018151725444680, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.429292Z debug proxy::pool inserting conn mutex for key 1895018151725444680 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.429315Z debug h2::codec::framed_read:client{request=3}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.427707Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 178 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.428409Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.427944Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.428300Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.429691Z debug proxy::pool::test client spent 101ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.428698Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.429725Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.428160Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.429466Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.427998Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.429916Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.429949Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.429954Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.429998Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.430028Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.430082Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.430127Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.430166Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.430159Z info xds::client::tests received request: DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None } [INFO] [stdout] 2026-03-06T15:30:41.430204Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.430166Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.430246Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.429835Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.430343Z debug proxy::pool spawning new pool conn for 127.0.0.41(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.429485Z debug h2::proto::settings:xds{id=1}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.430449Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.429449Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 179 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.430494Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.430483Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.430589Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.430773Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 180 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.431130Z debug proxy::pool didn't find a connection for key 12270211808482481288, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.431053Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.431689Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 181 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.430592Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.431513Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.430960Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.431791Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 182 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.431478Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(23) } [INFO] [stdout] 2026-03-06T15:30:41.431760Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.430475Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.431811Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.430595Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.431306Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] test proxy::pool::test::unique_keys_have_unique_connections ... ok [INFO] [stdout] 2026-03-06T15:30:41.431901Z debug proxy::pool inserting conn mutex for key 12270211808482481288 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.432069Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 183 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.434027Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 184 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.434093Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 185 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.432724Z info test_helpers::xds sending response... [INFO] [stdout] 2026-03-06T15:30:41.432774Z info xds::client::tests received request: DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None } [INFO] [stdout] 2026-03-06T15:30:41.434142Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 186 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.433727Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.432212Z debug proxy::pool::test client spent 6ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.434288Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.434050Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.435152Z debug proxy::pool spawning new pool conn for 127.0.0.42(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.435251Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.434274Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.435330Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.435973Z debug proxy::pool didn't find a connection for key 230191555705963720, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.436240Z debug proxy::pool inserting conn mutex for key 230191555705963720 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.433821Z debug h2::codec::framed_write:client{request=3}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.436387Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.434363Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 187 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.436278Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.434386Z debug proxy::pool:client{request=12} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 12 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.436566Z debug proxy::pool spawning new pool conn for 127.0.0.43(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.436618Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.434394Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.434356Z debug test_helpers::xds response... Ok(DeltaDiscoveryResponse { system_version_info: "1.0.0", resources: [Resource { name: "foo0", aliases: [], version: "0.0.1", resource: Some(Any { type_url: "type.googleapis.com/istio.workload.Address", value: [10, 63, 10, 4, 102, 111, 111, 48, 18, 7, 100, 101, 102, 97, 117, 108, 116, 26, 4, 1, 2, 3, 4, 50, 5, 108, 111, 99, 97, 108, 58, 7, 100, 101, 102, 97, 117, 108, 116, 74, 7, 100, 101, 102, 97, 117, 108, 116, 112, 1, 162, 1, 12, 100, 101, 102, 97, 117, 108, 116, 47, 102, 111, 111, 48] }), ttl: None, cache_control: None }], type_url: "type.googleapis.com/istio.workload.Address", removed_resources: [], nonce: "qySmGSHzqmkAAAAAHZcA8iBIZ5HtG9OM" }) [INFO] [stdout] 2026-03-06T15:30:41.435454Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.436619Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 188 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.436828Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] test proxy::pool::test::idle_eviction ... ok [INFO] [stdout] 2026-03-06T15:30:41.436963Z debug pingora_pool::connection:client{request=11} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.438220Z debug proxy::pool didn't find a connection for key 1697580045632589185, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.438383Z debug proxy::pool inserting conn mutex for key 1697580045632589185 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.438086Z debug proxy::pool:client{request=11} connection ConnectionMeta { key: 385506954252831254, id: 11 } was removed/checked out/timed out of the pool [INFO] [stdout] test proxy::pool::test::connections_reused ... ok [INFO] [stdout] 2026-03-06T15:30:41.432261Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(23) } [INFO] [stdout] 2026-03-06T15:30:41.437157Z info test_helpers::xds sending response... [INFO] [stdout] 2026-03-06T15:30:41.437164Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 189 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.436627Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.438713Z debug test_helpers::xds response... Ok(DeltaDiscoveryResponse { system_version_info: "1.0.0", resources: [Resource { name: "foo0", aliases: [], version: "0.0.1", resource: Some(Any { type_url: "type.googleapis.com/istio.security.Authorization", value: [10, 4, 102, 111, 111, 48, 18, 7, 100, 101, 102, 97, 117, 108, 116, 32, 1, 42, 7, 10, 5, 18, 3, 74, 1, 80] }), ttl: None, cache_control: None }], type_url: "type.googleapis.com/istio.security.Authorization", removed_resources: [], nonce: "KmriGSHzqmkAAAAAERUhuI6U0Mh/0Ks6" }) [INFO] [stdout] 2026-03-06T15:30:41.436677Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: ["*"], resource_names_unsubscribe: ["*"], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.437226Z info test_helpers::xds sending response... [INFO] [stdout] 2026-03-06T15:30:41.438886Z debug test_helpers::xds response... Ok(DeltaDiscoveryResponse { system_version_info: "1.0.0", resources: [Resource { name: "1.1.1.1", aliases: [], version: "0.0.1", resource: Some(Any { type_url: "type.googleapis.com/istio.workload.Address", value: [10, 51, 10, 7, 49, 46, 49, 46, 49, 46, 49, 18, 7, 100, 101, 102, 97, 117, 108, 116, 26, 4, 127, 0, 0, 1, 50, 5, 108, 111, 99, 97, 108, 58, 7, 100, 101, 102, 97, 117, 108, 116, 74, 7, 100, 101, 102, 97, 117, 108, 116, 112, 1] }), ttl: None, cache_control: None }], type_url: "type.googleapis.com/istio.workload.Address", removed_resources: [], nonce: "0UsJDSHzqmkAAAAAFs4V6DAcR3OiZqtf" }) [INFO] [stdout] 2026-03-06T15:30:41.436882Z debug proxy::pool:client{request=12} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.438851Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.438962Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.438988Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.439077Z debug proxy::pool spawning new pool conn for 127.0.0.44(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.439098Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.439109Z debug proxy::pool:client{request=12} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 12 } [INFO] [stdout] 2026-03-06T15:30:41.439377Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.439667Z debug xds::client:xds{id=1} connected established [INFO] [stdout] 2026-03-06T15:30:41.439158Z debug proxy::pool:client{request=12} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.436935Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.439309Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.439205Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(11), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.432548Z debug xds::client:xds{id=1} connected established [INFO] [stdout] 2026-03-06T15:30:41.439664Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.438922Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.439832Z info xds::client:xds{id=1} Stream established [INFO] [stdout] 2026-03-06T15:30:41.440033Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.440170Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.439870Z info xds::client:xds{id=1} Stream established [INFO] [stdout] 2026-03-06T15:30:41.439833Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(7), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.440229Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.440160Z debug xds::client:xds{id=1} received response type_url="type.googleapis.com/istio.workload.Address" size=1 removes=0 [INFO] [stdout] 2026-03-06T15:30:41.439803Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.439240Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 190 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.440346Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.442956Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 191 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.440477Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.440605Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(9), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.443050Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 192 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.440922Z debug proxy::pool didn't find a connection for key 1989956920639744297, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.440804Z debug xds:xds{id=1} handling insert [INFO] [stdout] 2026-03-06T15:30:41.441688Z debug rustls::client::hs:client{request=0} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.442097Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.440493Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.443155Z debug proxy::pool inserting conn mutex for key 1989956920639744297 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.444901Z debug rustls::client::hs:client{request=0} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.442847Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445209Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445264Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.440960Z info xds::client::tests received request: DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: ["*"], resource_names_unsubscribe: ["*"], initial_resource_versions: {}, response_nonce: "", error_detail: None } [INFO] [stdout] 2026-03-06T15:30:41.445300Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445334Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445379Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.444946Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel9 [INFO] [stdout] 2026-03-06T15:30:41.445411Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.444962Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.445493Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.444072Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.443435Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(11), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.445551Z debug proxy::pool spawning new pool conn for 127.0.0.45(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.445664Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445716Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.445829Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445464Z info xds::client::tests received request: DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None } [INFO] [stdout] 2026-03-06T15:30:41.445895Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445900Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.445933Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.446019Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.446066Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.446943Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.446998Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.447031Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.447085Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.447148Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.447204Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.447255Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.447211Z debug proxy::pool didn't find a connection for key 15364838760363527690, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.447306Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] test xds::client::tests::test_on_demand_handling ... ok [INFO] [stdout] 2026-03-06T15:30:41.443133Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 193 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.447445Z debug proxy::pool inserting conn mutex for key 15364838760363527690 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.447499Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 194 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.447619Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.447628Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 195 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.445825Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(7), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.447505Z debug xds::client:xds{id=1} received response type_url="type.googleapis.com/istio.workload.Address" size=1 removes=0 [INFO] [stdout] 2026-03-06T15:30:41.447655Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.445960Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.447760Z debug proxy::pool spawning new pool conn for 127.0.0.46(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.448054Z debug xds:xds{id=1} handling insert [INFO] [stdout] 2026-03-06T15:30:41.447813Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.448149Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.448192Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.447832Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 196 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.448299Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 197 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.448955Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.449010Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 198 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.449093Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 199 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.449099Z debug proxy::pool didn't find a connection for key 15087478472017649955, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.449162Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 200 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.448572Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.449264Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.449370Z debug proxy::pool inserting conn mutex for key 15087478472017649955 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.448322Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(9), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.449511Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.449544Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.449653Z debug proxy::pool spawning new pool conn for 127.0.0.47(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.449697Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.449789Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.449843Z debug rustls::client::hs:client{request=0} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.449876Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.449978Z debug rustls::client::tls13:client{request=0} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.450243Z debug rustls::client::tls13:client{request=0} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.443467Z debug xds::client:xds{id=1} sending response type_url="type.googleapis.com/istio.workload.Address" nonce="qySmGSHzqmkAAAAAHZcA8iBIZ5HtG9OM" type=Ack [INFO] [stdout] 2026-03-06T15:30:41.448223Z debug xds::client:xds{id=1} sending response type_url="type.googleapis.com/istio.workload.Address" nonce="0UsJDSHzqmkAAAAAFs4V6DAcR3OiZqtf" type=Ack [INFO] [stdout] 2026-03-06T15:30:41.450758Z debug proxy::pool didn't find a connection for key 13713543613876342860, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.450838Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 201 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.450824Z debug xds::client:xds{id=1} received response type_url="type.googleapis.com/istio.security.Authorization" size=1 removes=0 [INFO] [stdout] 2026-03-06T15:30:41.450832Z debug proxy::pool inserting conn mutex for key 13713543613876342860 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.450913Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 202 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.450915Z debug xds::client:xds{id=1} sending request type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.450896Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.450962Z info xds:xds{id=1} handling RBAC update foo0 [INFO] [stdout] 2026-03-06T15:30:41.450379Z debug rustls::client::hs:client{request=0} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.451183Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.451163Z debug xds::client:xds{id=1} sending response type_url="type.googleapis.com/istio.security.Authorization" nonce="KmriGSHzqmkAAAAAERUhuI6U0Mh/0Ks6" type=Ack [INFO] [stdout] 2026-03-06T15:30:41.448693Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.451392Z debug xds::client:xds{id=1} sending request type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.451097Z debug proxy::pool::test client spent 16ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.451707Z debug xds::client:xds{id=1} sending request type_url="type.googleapis.com/istio.security.Authorization" [INFO] [stdout] 2026-03-06T15:30:41.451982Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.451655Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.452461Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.452322Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.451917Z debug proxy::pool:client{request=13} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 13 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.451577Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.451551Z debug state deny policy match policy="default/foo0" [INFO] [stdout] 2026-03-06T15:30:41.452863Z debug proxy::pool:client{request=13} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.452104Z debug h2::client:client{request=0} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.452514Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.452929Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.451235Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(25) } [INFO] [stdout] 2026-03-06T15:30:41.452677Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.453009Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.452970Z debug test_helpers::xds request DeltaDiscoveryRequest { node: None, type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "0UsJDSHzqmkAAAAAFs4V6DAcR3OiZqtf", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.453003Z debug test_helpers::xds request DeltaDiscoveryRequest { node: None, type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "qySmGSHzqmkAAAAAHZcA8iBIZ5HtG9OM", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.453181Z debug pingora_pool::connection:client{request=12} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.452871Z debug state no allow policies, allow [INFO] [stdout] 2026-03-06T15:30:41.453278Z debug proxy::pool:client{request=12} connection ConnectionMeta { key: 385506954252831254, id: 12 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.451008Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 203 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.453175Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.454706Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 204 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.453480Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.454684Z debug test_helpers::xds request DeltaDiscoveryRequest { node: None, type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "KmriGSHzqmkAAAAAERUhuI6U0Mh/0Ks6", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.453631Z debug proxy::pool:client{request=13} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.454799Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 205 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.452516Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.455486Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.453522Z debug h2::client:client{request=0} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.454539Z debug proxy::pool spawning new pool conn for 127.0.0.48(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.453455Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(25) } [INFO] [stdout] 2026-03-06T15:30:41.456033Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.456388Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.455417Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 206 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.455792Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.455786Z debug h2::codec::framed_write:client{request=0} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.455540Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.458337Z debug proxy::pool:client{request=0} starting an idle timeout for connection ConnectionMeta { key: 14184913346861640675, id: 4 } [INFO] [stdout] 2026-03-06T15:30:41.456618Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.457409Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.454985Z debug proxy::pool:client{request=13} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 13 } [INFO] [stdout] 2026-03-06T15:30:41.457844Z debug proxy::pool:client{request=0} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.459924Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.457384Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 207 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.460017Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel6 [INFO] [stdout] 2026-03-06T15:30:41.458531Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.458245Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.460111Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.460203Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.458979Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.460679Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(27) } [INFO] [stdout] 2026-03-06T15:30:41.460026Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 208 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.460712Z debug proxy::pool didn't find a connection for key 12932317150835077929, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.460410Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.460322Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.460896Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.460836Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 209 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.461116Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 210 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.460979Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.461192Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 211 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.460927Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.461125Z debug proxy::pool inserting conn mutex for key 12932317150835077929 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.461565Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.461518Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.461318Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.461768Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.461668Z debug proxy::pool spawning new pool conn for 127.0.0.49(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.461369Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.462202Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.462237Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.461593Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 212 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.462356Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.463497Z debug proxy::pool didn't find a connection for key 5066239023032310054, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.462092Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.463773Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(27) } [INFO] [stdout] 2026-03-06T15:30:41.463783Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.462331Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.464007Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.464008Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.464069Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.462550Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 213 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.462722Z debug proxy::pool::test client spent 10ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.464189Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.464351Z debug proxy::pool inserting conn mutex for key 5066239023032310054 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.464371Z debug proxy::pool:client{request=14} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 14 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.464559Z debug proxy::pool:client{request=14} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.464681Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.464703Z debug proxy::pool:client{request=14} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.465053Z debug proxy::pool spawning new pool conn for 127.0.0.50(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.464223Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 214 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.464806Z debug pingora_pool::connection:client{request=13} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.464849Z debug proxy::pool:client{request=14} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 14 } [INFO] [stdout] 2026-03-06T15:30:41.464820Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.465122Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.465204Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.465245Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.465341Z debug proxy::pool:client{request=13} connection ConnectionMeta { key: 385506954252831254, id: 13 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.465711Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.465475Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.465245Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 215 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.465768Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.466117Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.467233Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.466030Z debug proxy::pool didn't find a connection for key 14377884572366250564, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.467377Z debug proxy::pool inserting conn mutex for key 14377884572366250564 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.467414Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.467475Z debug proxy::pool spawning new pool conn for 127.0.0.51(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.467020Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.467557Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.466268Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 216 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.467664Z debug h2::proto::settings:client{request=0}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.467823Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.466535Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.469233Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.469285Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.467932Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 217 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.471848Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.470158Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.471300Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.471613Z debug proxy::pool didn't find a connection for key 2245296014142536328, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.472087Z debug proxy::pool inserting conn mutex for key 2245296014142536328 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.472137Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.472237Z debug proxy::pool spawning new pool conn for 127.0.0.52(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.466653Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.472335Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.472098Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.472235Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 218 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.472478Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.472619Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(29) } [INFO] [stdout] 2026-03-06T15:30:41.472442Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473235Z debug proxy::pool didn't find a connection for key 2852495634622183130, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.473350Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473387Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473434Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473477Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473517Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473562Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473608Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.473523Z debug proxy::pool inserting conn mutex for key 2852495634622183130 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.473653Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.472792Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 219 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.473746Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.473886Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel2 [INFO] [stdout] 2026-03-06T15:30:41.473783Z debug proxy::pool spawning new pool conn for 127.0.0.53(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.474215Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 220 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.475917Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 221 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.474283Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.475981Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 222 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.475935Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.476223Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 223 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.473968Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.474105Z debug proxy::pool::test client spent 126ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.476212Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.476379Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(29) } [INFO] [stdout] 2026-03-06T15:30:41.474113Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.476410Z debug proxy::pool:client{request=1} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 5 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.476554Z debug proxy::pool::test client spent 12ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.477108Z debug proxy::pool:client{request=1} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.477264Z debug proxy::pool:client{request=1} checked out connection for ConnectionMeta { key: 14184913346861640675, id: 5 } is now at max streamcount; removing from pool [INFO] [stdout] 2026-03-06T15:30:41.476487Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 224 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.477375Z debug proxy::pool:client{request=1} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.477014Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.476899Z debug proxy::pool didn't find a connection for key 6108765714056439943, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.477642Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.477694Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.477755Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.477470Z debug pingora_pool::connection:client{request=0} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.477803Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.477808Z debug proxy::pool inserting conn mutex for key 6108765714056439943 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.477297Z debug proxy::pool:client{request=15} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 15 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.477485Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 225 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.477845Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.477921Z debug proxy::pool:client{request=0} connection ConnectionMeta { key: 14184913346861640675, id: 4 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.477696Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.477939Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.478111Z debug proxy::pool spawning new pool conn for 127.0.0.54(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.478187Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.477993Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 226 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.478244Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.478069Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.478479Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.478360Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 227 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.479067Z debug proxy::pool didn't find a connection for key 6095076894405840344, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.479166Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 228 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.479240Z debug proxy::pool inserting conn mutex for key 6095076894405840344 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.479039Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.478445Z debug proxy::pool:client{request=15} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.478404Z debug pingora_pool::connection:client{request=14} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.479245Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 229 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.479303Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.479342Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.479685Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 230 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.479677Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.479828Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 231 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.479828Z debug proxy::pool spawning new pool conn for 127.0.0.55(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.479911Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 232 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.479988Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 233 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.480037Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.480065Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 234 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.479580Z debug proxy::pool:client{request=14} connection ConnectionMeta { key: 385506954252831254, id: 14 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.480312Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] test xds::client::tests::test_that_caches_are_warm_before_unblocked ... ok [INFO] [stdout] 2026-03-06T15:30:41.482811Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.483137Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.480094Z debug proxy::pool:client{request=15} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 15 } [INFO] [stdout] 2026-03-06T15:30:41.483491Z debug proxy::pool::test client spent 7ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.479467Z debug proxy::pool:client{request=15} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.483406Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.482852Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.483760Z debug proxy::pool:client{request=2} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 6 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.484065Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484098Z debug proxy::pool:client{request=2} new connection needed for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.484198Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484234Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484276Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484311Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484344Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484378Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484412Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484445Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.480122Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.484478Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484954Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484494Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.485278Z debug proxy::pool:client{request=2} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.483288Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 235 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.485377Z debug state:client{request=2} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.484992Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.484854Z debug proxy::pool didn't find a connection for key 6197878651923755002, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.485506Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 236 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.485598Z debug proxy::pool inserting conn mutex for key 6197878651923755002 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.485678Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.485698Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 237 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.485501Z debug state:client{request=2} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.485856Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 238 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.485746Z debug proxy::pool spawning new pool conn for 127.0.0.56(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.485569Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.486256Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.486319Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.486329Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.485980Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.486661Z debug proxy::pool didn't find a connection for key 12696418002988072243, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.486379Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.487133Z debug proxy::pool inserting conn mutex for key 12696418002988072243 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.486843Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.487221Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.487242Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.486259Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 239 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.487443Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.487458Z debug proxy::pool spawning new pool conn for 127.0.0.57(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.487565Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.487493Z info test_helpers::xds sending response... [INFO] [stdout] 2026-03-06T15:30:41.487535Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.487753Z debug test_helpers::xds response... Err(Status { code: Aborted, message: "Aborting for test.", source: None }) [INFO] [stdout] 2026-03-06T15:30:41.487370Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.487651Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.487779Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.488065Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) } [INFO] [stdout] 2026-03-06T15:30:41.487354Z debug rustls::client::hs:client{request=2} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.487614Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 240 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.488123Z debug proxy::pool didn't find a connection for key 7400992445221906983, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.488748Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 241 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.488296Z debug rustls::client::hs:client{request=2} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.488216Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.488724Z debug proxy::pool inserting conn mutex for key 7400992445221906983 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.488101Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.488882Z debug h2::codec::framed_read:xds{id=1}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) } [INFO] [stdout] 2026-03-06T15:30:41.489240Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.489290Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(31) } [INFO] [stdout] 2026-03-06T15:30:41.489180Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.489360Z debug proxy::pool spawning new pool conn for 127.0.0.58(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.489436Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.489495Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.489532Z warn xds::client:xds{id=1} XDS client error: gRPC error status: Aborted, message: "Aborting for test.", retrying in 10ms [INFO] [stdout] 2026-03-06T15:30:41.489164Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.490217Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.489546Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.490326Z warn xds::client:xds{id=1} outbound stream complete [INFO] [stdout] 2026-03-06T15:30:41.490375Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(31) } [INFO] [stdout] 2026-03-06T15:30:41.489643Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.490361Z debug proxy::pool didn't find a connection for key 16981669617463701789, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.490573Z debug proxy::pool::test client spent 13ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.490606Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) } [INFO] [stdout] 2026-03-06T15:30:41.490690Z debug proxy::pool inserting conn mutex for key 16981669617463701789 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.490714Z debug proxy::pool:client{request=16} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 16 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.492895Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.493009Z debug proxy::pool spawning new pool conn for 127.0.0.59(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.490911Z debug h2::codec::framed_write:xds{id=1}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.493238Z debug pingora_pool::connection:client{request=15} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.493114Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.490834Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.493365Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.493767Z debug proxy::pool didn't find a connection for key 13359752251189155840, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.494179Z debug proxy::pool inserting conn mutex for key 13359752251189155840 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.493439Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.494260Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.493478Z debug proxy::pool:client{request=15} connection ConnectionMeta { key: 385506954252831254, id: 15 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.494346Z debug proxy::pool spawning new pool conn for 127.0.0.60(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.493077Z debug proxy::pool:client{request=16} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.493405Z debug h2::proto::connection:xds{id=1}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.494457Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.491098Z debug rustls::client::hs:client{request=2} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.494569Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.488816Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 242 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.494936Z debug proxy::pool didn't find a connection for key 12553178882492877594, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.495370Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 243 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.494750Z debug rustls::common_state:xds{id=1}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.495359Z debug proxy::pool inserting conn mutex for key 12553178882492877594 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.494349Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.495479Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.495544Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.494753Z debug rustls::client::tls13:client{request=2} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.495574Z debug proxy::pool spawning new pool conn for 127.0.0.61(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.495641Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.495680Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.495444Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 244 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.495779Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.494621Z debug proxy::pool:client{request=16} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.496588Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.495757Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.494723Z debug proxy::pool:client{request=16} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 16 } [INFO] [stdout] 2026-03-06T15:30:41.495939Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 245 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.495955Z debug rustls::client::tls13:client{request=2} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.496125Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) } [INFO] [stdout] 2026-03-06T15:30:41.500124Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 246 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.499875Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.499878Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.500818Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.500860Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.500895Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.500929Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.500962Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.500995Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.501028Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.501074Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.501116Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.501157Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.501197Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.501417Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.501595Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel5 [INFO] [stdout] 2026-03-06T15:30:41.500185Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 247 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.501847Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 248 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.501915Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 249 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.501975Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 250 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.502035Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 251 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.502093Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 252 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.502157Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 253 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.502227Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 254 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.496727Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.500581Z debug rustls::client::hs:client{request=2} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.503472Z debug h2::client:client{request=2} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.500474Z debug proxy::pool didn't find a connection for key 8972868521641010091, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.500683Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.504232Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.501227Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.503832Z debug proxy::pool inserting conn mutex for key 8972868521641010091 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.503772Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.507148Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.507484Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.502281Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 255 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.503581Z debug h2::client:client{request=2} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.504753Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.507880Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(33) } [INFO] [stdout] 2026-03-06T15:30:41.507619Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 256 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.508037Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 257 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.508110Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 258 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.507940Z debug proxy::pool spawning new pool conn for 127.0.0.62(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.504698Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.508655Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.508720Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.506103Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.508297Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.509723Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.500446Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.510185Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.508764Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.510290Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.509533Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.508412Z debug h2::codec::framed_write:client{request=2} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.510650Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(33) } [INFO] [stdout] 2026-03-06T15:30:41.511722Z debug proxy::pool:client{request=2} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.509162Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.510920Z debug proxy::pool didn't find a connection for key 7283882931036585260, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.511831Z debug proxy::pool inserting conn mutex for key 7283882931036585260 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.512042Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.510704Z info test_helpers::xds ads_server: stream over [INFO] [stdout] 2026-03-06T15:30:41.511080Z debug proxy::pool::test client spent 20ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.512087Z debug proxy::pool spawning new pool conn for 127.0.0.63(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.512049Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.512133Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.511852Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.512218Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.512473Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.511987Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.510338Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.512664Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.512713Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.512763Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.512800Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.513011Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.513067Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.513108Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.508191Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 259 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.513185Z debug proxy::pool:client{request=17} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 17 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.513263Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 260 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.513334Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 261 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.512246Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel15 [INFO] [stdout] 2026-03-06T15:30:41.514778Z debug rustls::webpki::anchors:xds{id=2} add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.514957Z debug xds::client:xds{id=2} XDS header added: header_foo=foo [INFO] [stdout] 2026-03-06T15:30:41.515114Z debug proxy::pool:client{request=17} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.515498Z debug proxy::pool:client{request=2} starting an idle timeout for connection ConnectionMeta { key: 14184913346861640675, id: 6 } [INFO] [stdout] 2026-03-06T15:30:41.515260Z debug proxy::pool:client{request=17} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.515777Z debug proxy::pool:client{request=17} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 17 } [INFO] [stdout] 2026-03-06T15:30:41.516097Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.516421Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.515048Z debug xds::client:xds{id=2} XDS header added: header_bar=bar [INFO] [stdout] 2026-03-06T15:30:41.516794Z debug hyper_util::client::legacy::connect::http:xds{id=2} connecting to 127.0.0.1:45409 [INFO] [stdout] 2026-03-06T15:30:41.513404Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 262 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.516915Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 263 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.512529Z debug proxy::pool didn't find a connection for key 9317353171421294499, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.516518Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.517294Z debug hyper_util::client::legacy::connect::http:xds{id=2} connected to 127.0.0.1:45409 [INFO] [stdout] 2026-03-06T15:30:41.517320Z debug proxy::pool inserting conn mutex for key 9317353171421294499 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.516111Z debug pingora_pool::connection:client{request=16} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.517358Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.517390Z debug proxy::pool spawning new pool conn for 127.0.0.64(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.516994Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 264 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.517452Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.517589Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 265 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.517414Z debug rustls::client::hs:xds{id=2} No cached session for IpAddress(V4(Ipv4Addr([127, 0, 0, 1]))) [INFO] [stdout] 2026-03-06T15:30:41.517766Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 266 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.516537Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.518041Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.517902Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 267 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.518369Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 268 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.516212Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.517664Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.519151Z debug pingora_pool::connection:client{request=3} idle connection is being evicted [INFO] [stdout] 2026-03-06T15:30:41.518492Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 269 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.518619Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.518893Z debug proxy::pool didn't find a connection for key 251722715159058338, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.519474Z debug pingora_pool::connection:client{request=3} evict fd: 6 from key 10407060835540538402 [INFO] [stdout] 2026-03-06T15:30:41.513174Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.518187Z debug rustls::client::hs:xds{id=2} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.518226Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.519592Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.519654Z debug proxy::pool:client{request=3} connection ConnectionMeta { key: 10407060835540538402, id: 6 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.519910Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.519489Z debug proxy::pool inserting conn mutex for key 251722715159058338 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.519379Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 270 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.520071Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.520547Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.520208Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.520641Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.520356Z debug h2::codec::framed_write:client{request=3}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.520455Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 271 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.520351Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.519910Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.520875Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 272 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.520761Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.520961Z debug proxy::pool spawning new pool conn for 127.0.0.65(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.521067Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.521208Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.521086Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 273 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.520260Z debug h2::proto::settings:client{request=2}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.521279Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.521488Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 274 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.520960Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.522409Z debug rustls::client::hs:xds{id=2} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.521779Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.521890Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 275 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.521386Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.521531Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.523280Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.517419Z debug proxy::pool:client{request=16} connection ConnectionMeta { key: 385506954252831254, id: 16 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.523429Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.523540Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.522798Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.522531Z debug rustls::client::tls13:xds{id=2} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.523189Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 276 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.520912Z debug h2::proto::connection:client{request=3}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.523762Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 277 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.523619Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.523620Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.524125Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.523926Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 278 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.523859Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.524011Z debug proxy::pool didn't find a connection for key 5835386980557015839, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.524226Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.523916Z debug rustls::client::tls13:xds{id=2} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.524511Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.524649Z debug rustls::client::hs:xds{id=2} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.524665Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.524356Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.524330Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 279 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.524467Z debug rustls::common_state:client{request=3}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.524435Z debug proxy::pool inserting conn mutex for key 5835386980557015839 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.524622Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.525016Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.524940Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.525248Z debug proxy::pool spawning new pool conn for 127.0.0.66(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.525107Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.524810Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 280 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.528807Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.529003Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.528941Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 281 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.529165Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.529092Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.529348Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.524443Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.530241Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.530344Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.527685Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.525331Z debug proxy::h2::client:client{request=3} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.529267Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 282 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.530423Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.525265Z debug h2::client:xds{id=2} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.529792Z debug proxy::pool didn't find a connection for key 6947697710077286394, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.530820Z debug h2::client:xds{id=2} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.525439Z debug h2::codec::framed_write:Connection{peer=Server} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.530777Z debug proxy::pool inserting conn mutex for key 6947697710077286394 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.530895Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.530562Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 283 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.530996Z debug proxy::pool spawning new pool conn for 127.0.0.67(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.526859Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(35) } [INFO] [stdout] 2026-03-06T15:30:41.531006Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.530656Z debug proxy::pool::test client spent 17ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.531438Z debug proxy::pool:client{request=18} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 18 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.531125Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 284 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.531597Z debug proxy::pool:client{request=18} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.531226Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.531624Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.531679Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.531046Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.532079Z debug proxy::pool didn't find a connection for key 15278217203590355181, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.532520Z debug proxy::pool inserting conn mutex for key 15278217203590355181 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.531592Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 285 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.532559Z debug pingora_pool::connection:client{request=17} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.532488Z debug proxy::pool:client{request=18} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 18 } [INFO] [stdout] 2026-03-06T15:30:41.530669Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.532868Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.532708Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 286 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.533030Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.532092Z debug proxy::pool::test inbound serve done Err(hyper::Error(Io, Kind(BrokenPipe))) [INFO] [stdout] 2026-03-06T15:30:41.533003Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 287 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.532852Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.532602Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.533212Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 288 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.531761Z debug proxy::pool:client{request=18} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.533492Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 289 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.533301Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.533374Z debug proxy::pool spawning new pool conn for 127.0.0.68(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.533176Z error proxy::pool::test server failed: hyper::Error(Io, Kind(BrokenPipe)) [INFO] [stdout] 2026-03-06T15:30:41.530906Z debug h2::codec::framed_write:xds{id=2} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.533800Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.533894Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.534011Z debug proxy::pool didn't find a connection for key 10176950471149436033, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.534120Z debug proxy::pool::test persistent client stop [INFO] [stdout] 2026-03-06T15:30:41.534073Z debug hyper_util::client::legacy::pool:xds{id=2} pooling idle connection for ("https", 127.0.0.1:45409) [INFO] [stdout] 2026-03-06T15:30:41.534118Z debug proxy::pool inserting conn mutex for key 10176950471149436033 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.534199Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.534253Z debug proxy::pool spawning new pool conn for 127.0.0.69(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.534319Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.533260Z debug proxy::pool::test client spent 49ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.534368Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.534320Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.534522Z debug proxy::pool:client{request=3} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 7 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.534327Z debug h2::codec::framed_write:xds{id=2}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.534751Z debug proxy::pool didn't find a connection for key 9982646974996013528, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.535290Z debug proxy::pool inserting conn mutex for key 9982646974996013528 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.534818Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.535361Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.535496Z debug proxy::pool spawning new pool conn for 127.0.0.70(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.535563Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.534793Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(35) } [INFO] [stdout] 2026-03-06T15:30:41.535439Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.535675Z debug pingora_pool::connection:client{request=2} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.535672Z debug h2::codec::framed_write:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.533715Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 290 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.535561Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.533689Z debug proxy::pool:client{request=17} connection ConnectionMeta { key: 385506954252831254, id: 17 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.535458Z debug xds::client:xds{id=2} sending initial request resources=1 type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.535587Z debug proxy::pool:client{request=3} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.535883Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.536031Z debug proxy::pool:client{request=3} checked out connection for ConnectionMeta { key: 14184913346861640675, id: 7 } is now at max streamcount; removing from pool [INFO] [stdout] 2026-03-06T15:30:41.536118Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.536128Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.536128Z debug proxy::pool:client{request=3} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.535788Z debug proxy::pool:client{request=2} connection ConnectionMeta { key: 14184913346861640675, id: 6 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.535825Z debug h2::proto::connection:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.535824Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 291 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.535618Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.536233Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.536381Z debug xds::client:xds{id=2} sending initial request resources=0 type_url="type.googleapis.com/istio.security.Authorization" [INFO] [stdout] 2026-03-06T15:30:41.536948Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 292 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.537342Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.536952Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.536858Z debug proxy::pool didn't find a connection for key 15292966535956639400, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.537484Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 293 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.537660Z debug h2::codec::framed_write:xds{id=2}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.537578Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.537769Z debug h2::codec::framed_write:Connection{peer=Server} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(11) } [INFO] [stdout] 2026-03-06T15:30:41.536639Z debug rustls::common_state:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.537905Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.537913Z debug h2::codec::framed_write:xds{id=2}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.540879Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.537684Z debug proxy::pool inserting conn mutex for key 15292966535956639400 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.538296Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.540901Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.541063Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.541177Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.538013Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 294 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.541423Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.538201Z debug proxy::h2::client done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.536421Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.541246Z debug proxy::pool spawning new pool conn for 127.0.0.71(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.541412Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 295 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.541596Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.541709Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.541318Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.541809Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.541803Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.541545Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.541926Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.541999Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.541644Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 296 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.541326Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.543329Z debug proxy::pool::test inbound serve done Err(hyper::Error(Io, Kind(BrokenPipe))) [INFO] [stdout] 2026-03-06T15:30:41.542087Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.543455Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.542188Z debug h2::codec::framed_read:xds{id=2}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.542476Z debug proxy::pool didn't find a connection for key 5245311096553105999, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.542956Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 12401958628566421154, id: 297 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.543638Z debug proxy::pool inserting conn mutex for key 5245311096553105999 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.543407Z error proxy::pool::test server failed: hyper::Error(Io, Kind(BrokenPipe)) [INFO] [stdout] 2026-03-06T15:30:41.543874Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.543692Z debug h2::codec::framed_write:xds{id=2}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.543772Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 230488702111320817, id: 298 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.543982Z debug proxy::pool spawning new pool conn for 127.0.0.72(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.543542Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.544227Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.544392Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.544121Z debug proxy::pool checkout - found mutex for pool key ConnectionMeta { key: 11949129260861191224, id: 299 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.544763Z debug proxy::pool didn't find a connection for key 7227994128159977244, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.544305Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.545880Z debug proxy::pool inserting conn mutex for key 7227994128159977244 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.544368Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.544053Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.545461Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.546180Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.546172Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.546287Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 } [INFO] [stdout] 2026-03-06T15:30:41.541807Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.546519Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.545955Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547041Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547128Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547195Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547235Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.546953Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.547447Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.547263Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547751Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547786Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.547823Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547923Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.547934Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.542071Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.547244Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.548002Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.548141Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.548168Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.548207Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.548297Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.548240Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(37) } [INFO] [stdout] 2026-03-06T15:30:41.546041Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.548500Z debug proxy::pool spawning new pool conn for 127.0.0.73(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.548575Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.548401Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] test proxy::pool::test::idle_eviction_with_persistent ... ok [INFO] [stdout] 2026-03-06T15:30:41.548649Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.548893Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.546809Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.549680Z debug proxy::pool::test client spent 18ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.547615Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.548719Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.548762Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.549878Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.549751Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(37) } [INFO] [stdout] 2026-03-06T15:30:41.550034Z debug proxy::pool:client{request=19} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 19 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.549986Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.549964Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.550346Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.554504Z debug proxy::pool:client{request=19} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.549752Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.550343Z debug proxy::pool didn't find a connection for key 6290856554075661095, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.554411Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.555395Z debug proxy::pool inserting conn mutex for key 6290856554075661095 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.555558Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.555633Z debug proxy::pool spawning new pool conn for 127.0.0.74(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.554853Z debug pingora_pool::connection:client{request=18} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.555726Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.555852Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.550996Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.556274Z debug proxy::pool didn't find a connection for key 14789415446694367767, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.555200Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.556706Z debug proxy::pool inserting conn mutex for key 14789415446694367767 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.555474Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.554916Z debug proxy::pool::test client spent 20ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.557119Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.555888Z debug proxy::pool:client{request=18} connection ConnectionMeta { key: 385506954252831254, id: 18 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.557296Z debug proxy::pool spawning new pool conn for 127.0.0.75(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.557411Z debug proxy::pool:client{request=4} checkout - found mutex for pool key ConnectionMeta { key: 14184913346861640675, id: 8 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.555295Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.557408Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.557507Z debug proxy::pool:client{request=4} new connection needed for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.554925Z debug proxy::pool:client{request=19} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 19 } [INFO] [stdout] 2026-03-06T15:30:41.557201Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.557626Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.558157Z debug proxy::pool didn't find a connection for key 9588630222467925752, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.557525Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.558598Z debug proxy::pool inserting conn mutex for key 9588630222467925752 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.558983Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.557723Z debug proxy::pool:client{request=4} spawning new pool conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:37661[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.556992Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.557753Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.559069Z debug proxy::pool spawning new pool conn for 127.0.0.76(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.557128Z debug h2::codec::framed_read:client{request=2}:Connection{peer=Client} received frame=Data { stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.559706Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.559617Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.559810Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.559210Z debug state:client{request=4} wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.555001Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {"1.1.1.1": ""}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.558876Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.559880Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.560726Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.559511Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.560781Z debug test_helpers::xds request DeltaDiscoveryRequest { node: Some(Node { id: "ztunnel~1.1.1.1~.~.svc.cluster.local", cluster: "", metadata: Some(Struct { fields: {"CLUSTER_ID": Value { kind: Some(StringValue("test-cluster")) }, "INSTANCE_IPS": Value { kind: Some(StringValue("1.1.1.1")) }, "NAME": Value { kind: Some(StringValue("")) }, "NAMESPACE": Value { kind: Some(StringValue("")) }, "NODE_NAME": Value { kind: Some(StringValue("")) }} }), build_version: "" }), type_url: "type.googleapis.com/istio.security.Authorization", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.560876Z debug state:client{request=4} got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.560618Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.560720Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.561027Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.561166Z debug proxy::pool didn't find a connection for key 582398938430587779, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.561784Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.562365Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.563422Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.563198Z debug proxy::pool inserting conn mutex for key 582398938430587779 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.562463Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.563569Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.562433Z debug rustls::client::hs:client{request=4} No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.563092Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.563897Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.564141Z debug rustls::client::hs:client{request=4} Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.563592Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.564171Z debug proxy::pool spawning new pool conn for 127.0.0.77(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.563963Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.564367Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.564458Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.564103Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.564280Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.564782Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.564503Z debug h2::codec::framed_read:xds{id=2}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.565600Z debug h2::proto::settings:xds{id=2}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.564673Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.564547Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.566381Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.565774Z debug h2::codec::framed_read:xds{id=2}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 } [INFO] [stdout] 2026-03-06T15:30:41.565613Z debug proxy::pool didn't find a connection for key 8816420411240732415, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.566681Z debug proxy::pool inserting conn mutex for key 8816420411240732415 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.566264Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.554710Z debug proxy::pool:client{request=19} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.566861Z debug h2::codec::framed_read:xds{id=2}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.563807Z debug proxy::pool checking in new conn for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 230488702111320817, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.566474Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.567142Z info test_helpers::xds sending response... [INFO] [stdout] 2026-03-06T15:30:41.565993Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.567130Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.565028Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.567203Z debug test_helpers::xds response... Ok(DeltaDiscoveryResponse { system_version_info: "1.0.0", resources: [], type_url: "type.googleapis.com/istio.workload.Address", removed_resources: ["127.0.0.1"], nonce: "fOwJDSHzqmkAAAAAL5c+SOBdtwz2c73T" }) [INFO] [stdout] 2026-03-06T15:30:41.567628Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.567153Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.567696Z debug xds::client:xds{id=2} connected established [INFO] [stdout] 2026-03-06T15:30:41.567840Z info xds::client:xds{id=2} Stream established [INFO] [stdout] 2026-03-06T15:30:41.567953Z debug proxy::pool spawning new pool conn for 127.0.0.78(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.567992Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.568109Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.567067Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.568451Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.568744Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.568396Z debug h2::codec::framed_read:xds{id=2}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.568822Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.568920Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.568980Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.569069Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.568943Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.568524Z debug rustls::client::hs:client{request=4} Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.569152Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.569275Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.569099Z debug xds::client:xds{id=2} received response type_url="type.googleapis.com/istio.workload.Address" size=0 removes=1 [INFO] [stdout] 2026-03-06T15:30:41.569408Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.569445Z debug xds:xds{id=2} handling delete 127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.569443Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.569548Z warn xds:xds{id=2} tried to remove service but it did not have the expected namespace/hostname format [INFO] [stdout] 2026-03-06T15:30:41.569640Z debug xds::client:xds{id=2} received delete resource type.googleapis.com/istio.workload.Address/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.568598Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.569323Z debug rustls::client::tls13:client{request=4} Not resuming [INFO] [stdout] 2026-03-06T15:30:41.569948Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.570152Z debug rustls::client::tls13:client{request=4} TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.570212Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(39) } [INFO] [stdout] 2026-03-06T15:30:41.570300Z debug rustls::client::hs:client{request=4} ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.568541Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.570587Z debug proxy::pool::test client spent 20ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.570491Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(39) } [INFO] [stdout] 2026-03-06T15:30:41.570747Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.569808Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.571094Z debug proxy::pool:client{request=20} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 20 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.568415Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.571204Z debug h2::client:client{request=4} binding client connection [INFO] [stdout] 2026-03-06T15:30:41.569336Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.571339Z debug h2::client:client{request=4} client connection bound [INFO] [stdout] 2026-03-06T15:30:41.571312Z debug pingora_pool::connection:client{request=19} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.570937Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.571336Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.571499Z debug proxy::pool:client{request=20} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.571620Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.571682Z debug proxy::pool:client{request=20} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 20 } [INFO] [stdout] 2026-03-06T15:30:41.571450Z debug proxy::pool:client{request=19} connection ConnectionMeta { key: 385506954252831254, id: 19 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.558655Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.573072Z debug proxy::pool didn't find a connection for key 5592745139803406470, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.571768Z debug proxy::pool:client{request=20} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.569750Z debug xds::client:xds{id=2} sending response type_url="type.googleapis.com/istio.workload.Address" nonce="fOwJDSHzqmkAAAAAL5c+SOBdtwz2c73T" type=Ack [INFO] [stdout] 2026-03-06T15:30:41.571862Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.573227Z debug proxy::pool inserting conn mutex for key 5592745139803406470 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.573347Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.573083Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel12 [INFO] [stdout] 2026-03-06T15:30:41.571885Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 1 } [INFO] [stdout] 2026-03-06T15:30:41.571442Z debug h2::codec::framed_write:client{request=4} send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.573438Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.573575Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.572872Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.572104Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.573825Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.574865Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.573901Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.574065Z debug xds::client:xds{id=2} sending request type_url="type.googleapis.com/istio.workload.Address" [INFO] [stdout] 2026-03-06T15:30:41.574314Z debug proxy::pool:client{request=4} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.573364Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.575549Z debug proxy::pool spawning new pool conn for 127.0.0.79(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.575643Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.575703Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.575025Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.578874Z debug proxy::pool didn't find a connection for key 8128360710451647314, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.579018Z debug proxy::pool inserting conn mutex for key 8128360710451647314 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.579111Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.579184Z debug proxy::pool spawning new pool conn for 127.0.0.80(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.575405Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.576518Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.579293Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.579872Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.580254Z debug proxy::pool didn't find a connection for key 10077195294906054576, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.571480Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.574655Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.580687Z debug proxy::pool inserting conn mutex for key 10077195294906054576 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.579364Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.581015Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.581170Z debug proxy::pool spawning new pool conn for 127.0.0.81(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.579757Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.575774Z debug h2::codec::framed_write:xds{id=2}:Connection{peer=Client} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.580807Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.580932Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.573943Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.581097Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.582120Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.582133Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.581493Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.581580Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.581247Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.582500Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.581345Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.575281Z debug proxy::pool:client{request=4} starting an idle timeout for connection ConnectionMeta { key: 14184913346861640675, id: 8 } [INFO] [stdout] 2026-03-06T15:30:41.582072Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.582375Z info test_helpers::xds received request... [INFO] [stdout] 2026-03-06T15:30:41.582694Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(41) } [INFO] [stdout] 2026-03-06T15:30:41.582272Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.582539Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.582439Z debug proxy::pool checking in new conn for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 11949129260861191224, id: 2 } [INFO] [stdout] 2026-03-06T15:30:41.582830Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.574422Z debug h2::codec::framed_write:client{request=4}:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.584580Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.583219Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585147Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.585149Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.583159Z debug test_helpers::xds request DeltaDiscoveryRequest { node: None, type_url: "type.googleapis.com/istio.workload.Address", resource_names_subscribe: [], resource_names_unsubscribe: [], initial_resource_versions: {}, response_nonce: "fOwJDSHzqmkAAAAAL5c+SOBdtwz2c73T", error_detail: None }... [INFO] [stdout] 2026-03-06T15:30:41.583608Z debug proxy::pool didn't find a connection for key 4439791217791957424, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.585320Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585360Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.584890Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.585484Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585555Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585609Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.584201Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.585656Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585719Z debug proxy::pool inserting conn mutex for key 4439791217791957424 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.583493Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.585845Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585986Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586057Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586091Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585877Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.586252Z debug proxy::pool::test client spent 15ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.586450Z debug proxy::pool spawning new pool conn for 127.0.0.82(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.586447Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.585753Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.586663Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586136Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(41) } [INFO] [stdout] 2026-03-06T15:30:41.586845Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586567Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.586952Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586687Z debug proxy::pool:client{request=21} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 21 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.587065Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586976Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.587103Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.587197Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586862Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.587326Z debug pingora_pool::connection:client{request=20} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.587251Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586721Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.587238Z debug proxy::pool:client{request=21} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.587663Z debug proxy::pool:client{request=21} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.587750Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.587408Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.587848Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.587462Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.588492Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.587407Z debug proxy::pool:client{request=20} connection ConnectionMeta { key: 385506954252831254, id: 20 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.587901Z debug proxy::pool:client{request=21} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 21 } [INFO] [stdout] 2026-03-06T15:30:41.588529Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.586315Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.587317Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.588433Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.593284Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 2 } [INFO] [stdout] 2026-03-06T15:30:41.588559Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.588699Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.591843Z debug proxy::pool didn't find a connection for key 5513795627034011899, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.592081Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.594031Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.594224Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.592976Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.594642Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.595013Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.594899Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.595110Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.593611Z debug proxy::pool inserting conn mutex for key 5513795627034011899 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.594778Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.595570Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.588408Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.587660Z debug h2::codec::framed_read:client{request=4}:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.594241Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.595222Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.595681Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.596661Z debug proxy::pool spawning new pool conn for 127.0.0.83(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.595778Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.596757Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.597172Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.597615Z debug proxy::pool didn't find a connection for key 7138405113075858830, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.598076Z debug proxy::pool inserting conn mutex for key 7138405113075858830 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.598145Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.597069Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.597198Z debug proxy::pool checking in new conn for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 12401958628566421154, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.596578Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.598205Z debug proxy::pool spawning new pool conn for 127.0.0.84(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.596528Z debug h2::codec::framed_write:client{request=4}:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.598307Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.598796Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.599012Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.598574Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.593643Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.599816Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.598687Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.598928Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.599975Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.599023Z debug h2::codec::framed_read:client{request=4}:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.600397Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.600083Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.600502Z debug h2::codec::framed_read:client{request=4}:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.600602Z debug h2::proto::settings:client{request=4}:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.600711Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.596807Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.600199Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.600787Z debug h2::codec::framed_write:client{request=4}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.599350Z debug proxy::pool didn't find a connection for key 8186286009660476952, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.600910Z debug proxy::pool inserting conn mutex for key 8186286009660476952 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.600016Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.600984Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.601282Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.601272Z debug proxy::pool spawning new pool conn for 127.0.0.85(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.601338Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.600534Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.601408Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.601317Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.602006Z debug proxy::pool didn't find a connection for key 2823648660364046677, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.602403Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.601317Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.601171Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.601516Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.602417Z debug proxy::pool inserting conn mutex for key 2823648660364046677 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.602543Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.602605Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(43) } [INFO] [stdout] 2026-03-06T15:30:41.602627Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.601631Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 0 } [INFO] [stdout] 2026-03-06T15:30:41.602666Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.602568Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.602790Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.602812Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.602983Z debug proxy::pool::test client spent 416ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.602975Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.602837Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.602603Z debug proxy::pool spawning new pool conn for 127.0.0.86(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.603249Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(43) } [INFO] [stdout] 2026-03-06T15:30:41.603181Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.603035Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.603180Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.603330Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.603673Z debug proxy::pool didn't find a connection for key 500723199682604598, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.603249Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 1 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.604172Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 4 } [INFO] [stdout] 2026-03-06T15:30:41.603591Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.604313Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.604754Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.604749Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.603495Z debug proxy::pool::test client spent 16ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.602822Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.604076Z debug proxy::pool inserting conn mutex for key 500723199682604598 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.604580Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.605435Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.605357Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.605484Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 2 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.605476Z debug proxy::pool spawning new pool conn for 127.0.0.87(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.605542Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.605564Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 5 } [INFO] [stdout] 2026-03-06T15:30:41.605253Z debug proxy::pool:client{request=22} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 22 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.605019Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.604961Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.605980Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.605746Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.606151Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.606299Z debug h2::codec::framed_read:client{request=4}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.606495Z debug h2::codec::framed_read:client{request=4}:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.605646Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.606771Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.606867Z debug proxy::pool::test client spent 49ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.606517Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.607000Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.606784Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.607115Z debug proxy::pool:client{request=22} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.605764Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.607849Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.607376Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.608006Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.608421Z debug proxy::pool:client{request=22} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.607219Z debug pingora_pool::connection:client{request=21} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.608120Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.608323Z debug proxy::pool:client{request=22} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 22 } [INFO] [stdout] 2026-03-06T15:30:41.608251Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.608874Z debug proxy::pool didn't find a connection for key 3710877805467182834, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.608403Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.607978Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.608395Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.609379Z debug proxy::pool inserting conn mutex for key 3710877805467182834 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.608578Z debug proxy::pool:client{request=21} connection ConnectionMeta { key: 385506954252831254, id: 21 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.609678Z debug h2::codec::framed_write:client{request=4}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.609299Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 0 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.609835Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 3 } [INFO] [stdout] 2026-03-06T15:30:41.609430Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.609495Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.610020Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.609630Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.610068Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.607328Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.610006Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.610167Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.610129Z debug proxy::pool spawning new pool conn for 127.0.0.88(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.610415Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.603470Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.610212Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.610332Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.610297Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.610577Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.610506Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 3 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.611054Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.610822Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.609953Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.610867Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.610963Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.610927Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 8 } [INFO] [stdout] 2026-03-06T15:30:41.611332Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.614003Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.614046Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.611402Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.614073Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 8 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.614087Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.613821Z debug proxy::pool didn't find a connection for key 4890230676712331089, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.614156Z debug proxy::pool inserting conn mutex for key 4890230676712331089 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.614118Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.614171Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.611245Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.611017Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.610693Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.614817Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.614900Z debug proxy::pool spawning new pool conn for 127.0.0.89(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.614967Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.615018Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.615323Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.615573Z debug proxy::pool::test dropped stream [INFO] [stdout] 2026-03-06T15:30:41.615608Z debug proxy::pool didn't find a connection for key 18214437302673942378, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.615646Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.615767Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(45) } [INFO] [stdout] 2026-03-06T15:30:41.614823Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.615771Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.615873Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.615945Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.615918Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 6 } [INFO] [stdout] 2026-03-06T15:30:41.616130Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.615577Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.616222Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.616292Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 6 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.616298Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(45) } [INFO] [stdout] 2026-03-06T15:30:41.616364Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 10 } [INFO] [stdout] 2026-03-06T15:30:41.616431Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.616437Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 4 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.616501Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 10 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.616113Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.620015Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.616556Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.620166Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.616054Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.620327Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.620462Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.620492Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.620636Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.619856Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 7 } [INFO] [stdout] 2026-03-06T15:30:41.620783Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.616623Z debug proxy::pool::test client spent 11ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.615674Z debug proxy::pool inserting conn mutex for key 18214437302673942378 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.618655Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.620554Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.621294Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.621339Z debug proxy::pool spawning new pool conn for 127.0.0.90(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.621416Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.620879Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.616142Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.620389Z debug h2::proto::connection:client{request=0}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.621574Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.621614Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.620792Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.621806Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.622977Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623027Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.621831Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 7 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.623063Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623100Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623135Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.622074Z debug h2::codec::framed_write:client{request=2}:Connection{peer=Client} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.623168Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623202Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623236Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623228Z debug h2::proto::connection:client{request=2}:Connection{peer=Client} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.623269Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623302Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623337Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623279Z debug proxy::pool didn't find a connection for key 15610322275496440687, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.623362Z debug rustls::common_state:client{request=2}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.623130Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.623512Z debug proxy::pool inserting conn mutex for key 15610322275496440687 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.623539Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.623516Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 5 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.621684Z debug proxy::pool:client{request=23} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 23 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.623373Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623665Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.624023Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.623702Z debug proxy::pool spawning new pool conn for 127.0.0.91(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.623760Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.624111Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.624212Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.623802Z debug proxy::pool:client{request=23} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.623941Z debug pingora_pool::connection:client{request=22} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.624065Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.624255Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.625065Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.624381Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.625145Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.625238Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.625303Z debug proxy::pool didn't find a connection for key 7248308639097295254, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.625355Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 11 } [INFO] [stdout] 2026-03-06T15:30:41.625409Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.625439Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.625481Z debug proxy::pool inserting conn mutex for key 7248308639097295254 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.625494Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 11 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.624033Z debug proxy::h2::client:client{request=2} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.625525Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.625636Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.625685Z debug proxy::pool:client{request=22} connection ConnectionMeta { key: 385506954252831254, id: 22 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.625768Z debug proxy::pool spawning new pool conn for 127.0.0.92(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.625867Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.625861Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.625962Z debug proxy::pool:client{request=23} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.625978Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.626069Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.624477Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 14 } [INFO] [stdout] 2026-03-06T15:30:41.623975Z debug rustls::common_state:client{request=0}:Connection{peer=Client} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.626189Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.625573Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.624235Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.626216Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.626296Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 9 } [INFO] [stdout] 2026-03-06T15:30:41.626355Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.626415Z debug proxy::pool:client{request=23} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 23 } [INFO] [stdout] 2026-03-06T15:30:41.626981Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.626958Z debug h2::codec::framed_write:Connection{peer=Server} send frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(3) } [INFO] [stdout] 2026-03-06T15:30:41.627606Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.627657Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.626618Z debug proxy::pool didn't find a connection for key 13318215558926261924, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.627668Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; connection error error=GoAway(b"", NO_ERROR, Library) [INFO] [stdout] 2026-03-06T15:30:41.626880Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.627098Z debug proxy::h2::client:client{request=0} done with HBONE connection handshake: Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.627812Z debug proxy::pool inserting conn mutex for key 13318215558926261924 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.629933Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.627687Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.626915Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.630548Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.630843Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.630122Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.630038Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 9 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.631125Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.627074Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 14 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.630936Z debug proxy::pool spawning new pool conn for 127.0.0.93(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.628069Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.631235Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.631250Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.631118Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.630421Z debug proxy::pool::test inbound serve done Err(hyper::Error(Io, Kind(BrokenPipe))) [INFO] [stdout] 2026-03-06T15:30:41.631646Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.631245Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 12 } [INFO] [stdout] 2026-03-06T15:30:41.631327Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.631694Z error proxy::pool::test server failed: hyper::Error(Io, Kind(BrokenPipe)) [INFO] [stdout] 2026-03-06T15:30:41.632695Z debug proxy::pool didn't find a connection for key 15905978394723455053, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.631350Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.629933Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.631531Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.631785Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.631028Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.631891Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.632706Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.631578Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.632912Z debug proxy::pool inserting conn mutex for key 15905978394723455053 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.632954Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.635143Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.626830Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.635122Z debug proxy::pool spawning new pool conn for 127.0.0.94(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.633046Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 13 } [INFO] [stdout] 2026-03-06T15:30:41.633356Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.633874Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.635303Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel7 [INFO] [stdout] 2026-03-06T15:30:41.635266Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.635347Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.635377Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.635442Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 13 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.635418Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.635703Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.633212Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 12 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.635591Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 16 } [INFO] [stdout] 2026-03-06T15:30:41.635500Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.633759Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.635915Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.636777Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 16 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.636852Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 17 } [INFO] [stdout] 2026-03-06T15:30:41.636917Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.635259Z debug h2::codec::framed_read:Connection{peer=Server} received frame=GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) } [INFO] [stdout] 2026-03-06T15:30:41.636975Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 17 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.636917Z debug proxy::pool didn't find a connection for key 3333494607164627256, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.637087Z debug rustls::common_state:Connection{peer=Server} Sending warning alert CloseNotify [INFO] [stdout] 2026-03-06T15:30:41.637118Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.637303Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.637360Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.637324Z debug proxy::pool inserting conn mutex for key 3333494607164627256 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.637479Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.637549Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.637558Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 15 } [INFO] [stdout] 2026-03-06T15:30:41.635968Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 19 } [INFO] [stdout] 2026-03-06T15:30:41.637984Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.637634Z debug proxy::pool spawning new pool conn for 127.0.0.95(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.636322Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.638127Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.636398Z debug proxy::pool::test hello world: completed request [INFO] [stdout] 2026-03-06T15:30:41.637660Z debug proxy::pool poolstate dropping, stopping all connection drivers and cancelling all outstanding eviction timeout spawns [INFO] [stdout] 2026-03-06T15:30:41.635633Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.638412Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.635847Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.638560Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.637773Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(47) } [INFO] [stdout] 2026-03-06T15:30:41.639164Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.639495Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.639688Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(47) } [INFO] [stdout] 2026-03-06T15:30:41.638776Z debug proxy::h2::client:client{request=4} draining outer HBONE connection [INFO] [stdout] 2026-03-06T15:30:41.638222Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 20 } [INFO] [stdout] 2026-03-06T15:30:41.638506Z debug pingora_pool::connection:client{request=4} idle connection is being closed [INFO] [stdout] 2026-03-06T15:30:41.639689Z debug proxy::pool didn't find a connection for key 17625645155764991053, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.640219Z debug proxy::pool::test client spent 18ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.640305Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.640042Z debug h2::proto::connection:Connection{peer=Server} Connection::poll; IO error error=UnexpectedEof [INFO] [stdout] 2026-03-06T15:30:41.640119Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.640486Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.640540Z debug proxy::pool::test inbound serve done Ok(()) [INFO] [stdout] 2026-03-06T15:30:41.640319Z debug proxy::pool inserting conn mutex for key 17625645155764991053 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.640612Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.638676Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.640886Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 15 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.640971Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.640315Z debug pingora_pool::connection:client{request=4} evict fd: 8 from key 14184913346861640675 [INFO] [stdout] 2026-03-06T15:30:41.640723Z debug proxy::pool:client{request=24} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 24 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.639995Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 18 } [INFO] [stdout] 2026-03-06T15:30:41.640341Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.641050Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 19 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.641161Z debug proxy::pool:client{request=4} connection ConnectionMeta { key: 14184913346861640675, id: 8 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.640958Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.641498Z debug pingora_pool::connection:client{request=23} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.641429Z debug proxy::pool spawning new pool conn for 127.0.0.96(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.641588Z debug proxy::pool:client{request=23} connection ConnectionMeta { key: 385506954252831254, id: 23 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.641229Z debug proxy::pool:client{request=24} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.641338Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.641577Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.641243Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.641875Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.642150Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.641872Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.641759Z debug proxy::pool:client{request=24} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.641413Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 22 } [INFO] [stdout] 2026-03-06T15:30:41.641781Z debug proxy::pool:client{request=24} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 24 } [INFO] [stdout] 2026-03-06T15:30:41.642453Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.642646Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.642806Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.643115Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] test proxy::pool::test::connection_limits ... ok [INFO] [stdout] 2026-03-06T15:30:41.643368Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.641452Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.643593Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.643320Z debug proxy::pool didn't find a connection for key 16801032447682948002, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.643727Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.643693Z debug proxy::pool inserting conn mutex for key 16801032447682948002 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.643782Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.643860Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.643893Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.643943Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.643954Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.644031Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.644147Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.644216Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.642556Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 20 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.644288Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.644298Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 23 } [INFO] [stdout] 2026-03-06T15:30:41.644385Z debug proxy::pool spawning new pool conn for 127.0.0.97(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.644319Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.644493Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.644530Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.644638Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.644527Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.644699Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.644845Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.644847Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.642324Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.644214Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.643666Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 18 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.645049Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.645648Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 21 } [INFO] [stdout] 2026-03-06T15:30:41.644767Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.645726Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.645815Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.645180Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.645878Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.645974Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.645548Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.646061Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.646207Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(49) } [INFO] [stdout] 2026-03-06T15:30:41.646123Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.646335Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.646453Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.646240Z debug proxy::pool didn't find a connection for key 1323289863600362684, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.646531Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.646615Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.646614Z debug proxy::pool inserting conn mutex for key 1323289863600362684 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.646688Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(49) } [INFO] [stdout] 2026-03-06T15:30:41.646720Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.646006Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.647061Z debug proxy::pool spawning new pool conn for 127.0.0.98(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.646956Z debug proxy::pool::test client spent 6ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.647127Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.645780Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.647108Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.647448Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.646911Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.647555Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.647247Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.647605Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.647781Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.647572Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.647912Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.647283Z debug proxy::pool:client{request=25} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 25 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.648631Z debug proxy::pool:client{request=25} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.647671Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.648713Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 22 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.648065Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.649124Z debug proxy::pool didn't find a connection for key 10338413320475016937, making sure lockmap has entry [INFO] [stdout] 2026-03-06T15:30:41.649138Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.648835Z debug pingora_pool::connection:client{request=24} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.648626Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.648880Z debug proxy::pool:client{request=25} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 25 } [INFO] [stdout] 2026-03-06T15:30:41.648942Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.648827Z debug proxy::pool:client{request=25} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.649321Z debug proxy::pool inserting conn mutex for key 10338413320475016937 into lockmap [INFO] [stdout] 2026-03-06T15:30:41.649792Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.650133Z debug proxy::pool nothing else is creating a conn and we won the lock, make one [INFO] [stdout] 2026-03-06T15:30:41.650218Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.650323Z debug proxy::pool spawning new pool conn for 127.0.0.99(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.650419Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.650554Z info test_helpers::xds sending response... [INFO] [stdout] 2026-03-06T15:30:41.650650Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.649243Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 25 } [INFO] [stdout] 2026-03-06T15:30:41.650487Z debug state wait for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.650765Z debug state got sub, waiting for workload wl=default.ns (source-workload) [INFO] [stdout] 2026-03-06T15:30:41.650832Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.650687Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.650098Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.651032Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.650907Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.651789Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.651871Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.651947Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.652031Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.650899Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 23 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.652941Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 26 } [INFO] [stdout] 2026-03-06T15:30:41.651163Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.652106Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.650644Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.653778Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.653366Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.653907Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.654037Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.654146Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.654125Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.654255Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.654340Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.654422Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.654501Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.654505Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.654649Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(51) } [INFO] [stdout] 2026-03-06T15:30:41.654611Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.655067Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.654791Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.655267Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(51) } [INFO] [stdout] 2026-03-06T15:30:41.655212Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.655474Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.655394Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.655622Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.655769Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.655884Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.655596Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.649825Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.656311Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.656433Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.649363Z debug proxy::pool:client{request=24} connection ConnectionMeta { key: 385506954252831254, id: 24 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.656002Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.659318Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.656630Z debug proxy::pool::test client spent 9ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.657145Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.659426Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.659608Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.655995Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.659704Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.659804Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.659879Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.656543Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.659954Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.659868Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.660045Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660124Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660197Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.659666Z debug proxy::pool:client{request=26} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 26 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.660033Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 26 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.660268Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660387Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660460Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.660465Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660566Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660214Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.660645Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660723Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660823Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.660783Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.660910Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.661007Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.661062Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.661101Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.661231Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.661230Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.661336Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.661422Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.657579Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.661538Z debug pingora_pool::connection:client{request=25} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.661567Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.661643Z debug proxy::pool:client{request=25} connection ConnectionMeta { key: 385506954252831254, id: 25 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.660933Z debug proxy::pool:client{request=26} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.661888Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.661940Z debug proxy::pool:client{request=26} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.660608Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.662406Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.662175Z debug proxy::pool:client{request=26} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 26 } [INFO] [stdout] 2026-03-06T15:30:41.662510Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.662575Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.662073Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.662324Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.685975Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.662713Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.685932Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.685287Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.662645Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.686170Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.686342Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.686244Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.686337Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.686452Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.686564Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.685600Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.662876Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.686414Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.686639Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 21 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.687020Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.687055Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.687201Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.687273Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.687475Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.687597Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.687645Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.687760Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.687832Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.688034Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.688103Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.688205Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.688219Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.688489Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.688554Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.689170Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.689318Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.689319Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.689420Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.689536Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 24 } [INFO] [stdout] 2026-03-06T15:30:41.689622Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.689668Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 24 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.689770Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.689805Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.689888Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.689921Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.690095Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.690239Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.690308Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.690372Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.690650Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.690797Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.690800Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.691064Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.688960Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.689024Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.691254Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.691411Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.691462Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.691513Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.691565Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.691622Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.691660Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.691623Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.691710Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.688824Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.691808Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.691998Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.692145Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.692255Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.692465Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.692568Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.692676Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.692725Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.692787Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.692973Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.693138Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.693171Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.693248Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.693472Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.693473Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.693520Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.693374Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.691325Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.690922Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(53) } [INFO] [stdout] 2026-03-06T15:30:41.687968Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.693646Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.693893Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.696011Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.696409Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.696564Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.696804Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.696959Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.697049Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.697299Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.697310Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.697466Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.697727Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.697834Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.697849Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.698072Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.698147Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.698221Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.698405Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.698457Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.698533Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.698653Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.697050Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.698726Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.698868Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel10 [INFO] [stdout] 2026-03-06T15:30:41.698876Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.698955Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(53) } [INFO] [stdout] 2026-03-06T15:30:41.699050Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.699295Z debug proxy::pool::test client spent 39ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.699264Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.699451Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.699495Z debug proxy::pool:client{request=27} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 27 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.699645Z debug proxy::pool:client{request=27} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.699819Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.699830Z debug proxy::pool:client{request=27} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.699918Z debug proxy::pool:client{request=27} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 27 } [INFO] [stdout] 2026-03-06T15:30:41.699941Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.695777Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.695509Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.700844Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.701015Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.701100Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.701151Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.701253Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.701358Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.701456Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.701491Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.701535Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.700789Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.700401Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.700306Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 32 } [INFO] [stdout] 2026-03-06T15:30:41.700078Z debug pingora_pool::connection:client{request=26} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.698673Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.701678Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.701879Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.701934Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.702618Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.702886Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.702399Z debug proxy::pool:client{request=26} connection ConnectionMeta { key: 385506954252831254, id: 26 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.702953Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.702306Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.703194Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 32 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.703292Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.703357Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 25 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.703365Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.703433Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 30 } [INFO] [stdout] 2026-03-06T15:30:41.703520Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.703537Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.703585Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 30 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.703850Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.702930Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.703829Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.703999Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.704147Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.704313Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.704428Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.704567Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.704674Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.704788Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.704835Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.703638Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 28 } [INFO] [stdout] 2026-03-06T15:30:41.704916Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.704980Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.705051Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 28 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.705126Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 34 } [INFO] [stdout] 2026-03-06T15:30:41.705186Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.702231Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.705298Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.705414Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.705470Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.705518Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.705525Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.705226Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 34 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.705793Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.705900Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 29 } [INFO] [stdout] 2026-03-06T15:30:41.705982Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.705988Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.706054Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 29 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.706107Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 35 } [INFO] [stdout] 2026-03-06T15:30:41.706172Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.706235Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 35 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.706297Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 27 } [INFO] [stdout] 2026-03-06T15:30:41.706356Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.702127Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.706411Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.706550Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.706777Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.706858Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.706920Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.707136Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.707258Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.707286Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(55) } [INFO] [stdout] 2026-03-06T15:30:41.707347Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.707644Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.707656Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.707759Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.707837Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(55) } [INFO] [stdout] 2026-03-06T15:30:41.701774Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.708098Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.708158Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.708176Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.708276Z debug proxy::pool::test client spent 8ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.708448Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.708586Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.708908Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.709040Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.709047Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.705566Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.706407Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 27 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.709360Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.709240Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 33 } [INFO] [stdout] 2026-03-06T15:30:41.709457Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.709509Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.709425Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.709563Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 33 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.709371Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.709810Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.709930Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.709987Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.710044Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.710093Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.710099Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.710144Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.708438Z debug proxy::pool:client{request=28} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 28 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.710369Z debug proxy::pool:client{request=28} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.710453Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.710517Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.710541Z debug proxy::pool:client{request=28} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.710649Z debug pingora_pool::connection:client{request=27} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.710863Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.710929Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.710976Z debug proxy::pool:client{request=28} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 28 } [INFO] [stdout] 2026-03-06T15:30:41.711172Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.711287Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.711352Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.711536Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.711679Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.711698Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.711823Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.711871Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.710763Z debug proxy::pool:client{request=27} connection ConnectionMeta { key: 385506954252831254, id: 27 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.710213Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.709616Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 31 } [INFO] [stdout] 2026-03-06T15:30:41.712156Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.712272Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.712334Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 31 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.712388Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.712398Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 37 } [INFO] [stdout] 2026-03-06T15:30:41.712494Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.712553Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 37 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.712679Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.712780Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.712854Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.713241Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.713288Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.713388Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.713602Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.713601Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.713688Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.713826Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.713957Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.714122Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.714262Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.714382Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.714484Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.714589Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.715007Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.715138Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.712087Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.715174Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.715266Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.715301Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.715342Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.705762Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.715367Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.715511Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.712831Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.715660Z debug proxy::pool checking in new conn for 127.0.0.61(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 12553178882492877594, id: 61 } [INFO] [stdout] 2026-03-06T15:30:41.715899Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 40 } [INFO] [stdout] 2026-03-06T15:30:41.715995Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.715980Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.716098Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.716161Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.716378Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.716510Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.716520Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.716674Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.713528Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.717008Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.717091Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.717254Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.717446Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.717504Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.717599Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.717895Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.717914Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.715370Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.718004Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.714663Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.718060Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(57) } [INFO] [stdout] 2026-03-06T15:30:41.715969Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.718602Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.718759Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 40 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.718801Z debug state fetch workload addr=/127.0.0.1 [INFO] [stdout] 2026-03-06T15:30:41.718410Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.718889Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.718904Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.716727Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12553178882492877594, id: 61 } [INFO] [stdout] 2026-03-06T15:30:41.718979Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.719010Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.719516Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.719612Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.718499Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.718840Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 38 } [INFO] [stdout] 2026-03-06T15:30:41.719706Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] test xds::client::tests::test_add_abort_remove ... ok [INFO] [stdout] 2026-03-06T15:30:41.719716Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.719783Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.720519Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.720619Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.720659Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.720705Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.720761Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.720806Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.720802Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.720272Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.720184Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.720847Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.720455Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.720967Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 38 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.721144Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.721520Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.721599Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.721275Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.721863Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.722011Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.722140Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.722326Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.722382Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.722466Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.722723Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.722795Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.722874Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.723096Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.723223Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.723270Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.723495Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.723499Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.723645Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.723652Z debug proxy::pool::test client spent 515ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.721353Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.723862Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.723907Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.723933Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.723972Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724002Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724030Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.724097Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724133Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724176Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.721026Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 41 } [INFO] [stdout] 2026-03-06T15:30:41.724200Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.724270Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.724328Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 41 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.724202Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724517Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.724614Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724664Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724705Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724785Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724830Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724868Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724901Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.723753Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.724934Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.725304Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.724385Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 36 } [INFO] [stdout] 2026-03-06T15:30:41.725396Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.725451Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 36 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.725523Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 39 } [INFO] [stdout] 2026-03-06T15:30:41.725586Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.725643Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 39 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.725791Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.725940Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.726158Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 43 } [INFO] [stdout] 2026-03-06T15:30:41.726308Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.725028Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.725103Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.723021Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.728471Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.724961Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.726864Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.731022Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.727394Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.726631Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.733956Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.734025Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.729034Z debug proxy::pool::test client spent 20ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.726540Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.734626Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.735214Z debug proxy::pool:client{request=29} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 29 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.734071Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.735326Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.735560Z debug proxy::pool:client{request=29} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.728985Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(57) } [INFO] [stdout] 2026-03-06T15:30:41.731224Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.735832Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.728780Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.735357Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.734918Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 44 } [INFO] [stdout] 2026-03-06T15:30:41.736714Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.731802Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.737012Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.736873Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.737428Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.737393Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.737582Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.737683Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.735889Z debug pingora_pool::connection:client{request=28} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.737842Z debug proxy::pool:client{request=29} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.737991Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.738109Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.738119Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.738285Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.738436Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.738457Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.738589Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.738592Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.738669Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.738725Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.738792Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.738945Z debug proxy::pool:client{request=29} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 29 } [INFO] [stdout] 2026-03-06T15:30:41.738933Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.739100Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.739165Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.739427Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.739433Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 42 } [INFO] [stdout] 2026-03-06T15:30:41.738822Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.739539Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.737851Z debug proxy::pool:client{request=28} connection ConnectionMeta { key: 385506954252831254, id: 28 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.740102Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.739797Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.740208Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.740223Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.734284Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.739963Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.739665Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.740828Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.740548Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.740864Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.740918Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.740986Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.741051Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.740753Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.741248Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.741404Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.741430Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.741558Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.741084Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.741770Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.741772Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.741815Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.741905Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.741936Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.741918Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.741963Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.739873Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.742251Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742222Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.742281Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742372Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.742373Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742399Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.742560Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742599Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742646Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742687Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742744Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742746Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.742787Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.742860Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.740569Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.742492Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.743108Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.743278Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.742180Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 46 } [INFO] [stdout] 2026-03-06T15:30:41.743296Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.743389Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.743453Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 46 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.743536Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.743599Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 43 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.743722Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.743720Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.743853Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.744103Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.744669Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.744792Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.744868Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 44 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.745033Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.745181Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.745199Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.745336Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.742868Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.745544Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.745596Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.745628Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.745594Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.745704Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.745837Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.741278Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.746096Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.746182Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.746284Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.746596Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.746768Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.746864Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.747028Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.747145Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.747107Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.747275Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.747287Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.747404Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.747539Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.747665Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.747798Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.747842Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.748125Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.748236Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.747674Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.744919Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 45 } [INFO] [stdout] 2026-03-06T15:30:41.748357Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.748479Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.748667Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.748680Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.748315Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.748979Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.748984Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.749142Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.749383Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.749527Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.749771Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.745668Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.749793Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.749901Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.749950Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.749995Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.750050Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.750092Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.750094Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.750131Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.744456Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.749586Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.749389Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.750228Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.750437Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.750788Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.750897Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.751154Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.751250Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.750454Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(59) } [INFO] [stdout] 2026-03-06T15:30:41.751474Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.751656Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.750259Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.750619Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.751811Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.751975Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.752086Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.752109Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.752222Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.752264Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.752485Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.752572Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.752661Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.745470Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.752782Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(59) } [INFO] [stdout] 2026-03-06T15:30:41.752861Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel16 [INFO] [stdout] 2026-03-06T15:30:41.751339Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.753079Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.753215Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.753288Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.753545Z debug proxy::pool::test client spent 18ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.753554Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.751850Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.753658Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.753766Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.753830Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.752354Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.753991Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.754144Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.754212Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.754263Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.754372Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.754475Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.754589Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.751500Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 45 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.755208Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.755310Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 47 } [INFO] [stdout] 2026-03-06T15:30:41.755387Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.755441Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 47 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.755505Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.755573Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.755709Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.755719Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.753866Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.755918Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.755955Z debug proxy::pool checking in new conn for 127.0.0.11(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 11228261214770619285, id: 11 } [INFO] [stdout] 2026-03-06T15:30:41.756287Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.753999Z debug proxy::pool:client{request=30} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 30 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.757139Z debug proxy::pool:client{request=30} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.757289Z debug proxy::pool:client{request=30} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.756701Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.757491Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.757527Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.757683Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11228261214770619285, id: 11 } [INFO] [stdout] 2026-03-06T15:30:41.757956Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.758056Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.758291Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.758390Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.759276Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.759376Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.759696Z debug pingora_pool::connection:client{request=29} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.756168Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.759832Z debug proxy::pool:client{request=29} connection ConnectionMeta { key: 385506954252831254, id: 29 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.759964Z debug proxy::pool:client{request=30} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 30 } [INFO] [stdout] 2026-03-06T15:30:41.760156Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.760253Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.760492Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.757567Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.760642Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.760693Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.760794Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.760851Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.760903Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.760942Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.760947Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.759578Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.761106Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.761356Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.761453Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.761491Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.761587Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.761710Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.761928Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 49 } [INFO] [stdout] 2026-03-06T15:30:41.761945Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.754681Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.762169Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.762205Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.762333Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.762455Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.762575Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.762627Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.762697Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.762851Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.762894Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.762965Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.763087Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.763198Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.763311Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.763363Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.763424Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.763530Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.763595Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.763639Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.760994Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.763823Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.763877Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.763917Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.763975Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.764023Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.760501Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.764235Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.764340Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.756544Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.764679Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.764709Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.764832Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.764928Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.765028Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.765129Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.765232Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.765340Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(15) } [INFO] [stdout] 2026-03-06T15:30:41.765493Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.765644Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.765806Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.765934Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.766087Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.766231Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.766448Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.766582Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.766681Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.764936Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.766806Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.766898Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.767219Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.767248Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.767420Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.767557Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.767923Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.768205Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.768452Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.768522Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.768932Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.769045Z debug proxy::pool::test client spent 15ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.769159Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.769231Z debug proxy::pool:client{request=31} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 31 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.769386Z debug proxy::pool:client{request=31} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.769565Z debug proxy::pool:client{request=31} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.769605Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.769769Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.769897Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.769897Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.770019Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.770149Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.770264Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.770375Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.768927Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(61) } [INFO] [stdout] 2026-03-06T15:30:41.770621Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.762012Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.759450Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.770934Z debug proxy::pool checking in new conn for 127.0.0.12(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 6357918293284060487, id: 12 } [INFO] [stdout] 2026-03-06T15:30:41.771044Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.771213Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.771261Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.771300Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.771514Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.771626Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.771784Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.764034Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.778996Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.770621Z debug pingora_pool::connection:client{request=30} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.770722Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 49 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.783020Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.771619Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.767569Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.779087Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.771904Z debug proxy::pool:client{request=31} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 31 } [INFO] [stdout] 2026-03-06T15:30:41.779257Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.779313Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.783132Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 42 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.770384Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.791200Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.791342Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.791457Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.791565Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.788407Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.790887Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 50 } [INFO] [stdout] 2026-03-06T15:30:41.790930Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.792125Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.791666Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.783314Z debug proxy::pool:client{request=30} connection ConnectionMeta { key: 385506954252831254, id: 30 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.792489Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.787707Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.792772Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.792381Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.792904Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.792176Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 50 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.790397Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(61) } [INFO] [stdout] 2026-03-06T15:30:41.795886Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.789898Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.796310Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.796363Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.793077Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.793317Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.796567Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.796822Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.792610Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel0 [INFO] [stdout] 2026-03-06T15:30:41.796726Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.796399Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.791830Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.799075Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.793249Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 6357918293284060487, id: 12 } [INFO] [stdout] 2026-03-06T15:30:41.797663Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.801083Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.792303Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.801308Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.799347Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.799365Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.803868Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.801440Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.799280Z debug proxy::pool checking in new conn for 127.0.0.13(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 10222465753960659478, id: 13 } [INFO] [stdout] 2026-03-06T15:30:41.797865Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 48 } [INFO] [stdout] 2026-03-06T15:30:41.802098Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.804897Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.804987Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.803919Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.805468Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.804205Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.801880Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.805601Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.799538Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.805916Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 48 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.805809Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.806459Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.806625Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 61 } [INFO] [stdout] 2026-03-06T15:30:41.805408Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.806107Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(63) } [INFO] [stdout] 2026-03-06T15:30:41.806238Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.806826Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.805231Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.807242Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.804152Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.807026Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.808225Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.809348Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.808022Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.807543Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 10222465753960659478, id: 13 } [INFO] [stdout] 2026-03-06T15:30:41.808383Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.808534Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.810392Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(63) } [INFO] [stdout] 2026-03-06T15:30:41.808639Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 61 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.810058Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.810544Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 52 } [INFO] [stdout] 2026-03-06T15:30:41.809522Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.809312Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.811095Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.810426Z debug proxy::pool::test client spent 41ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.811349Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.810805Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.811422Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 52 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.810757Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.811598Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.811315Z debug proxy::pool:client{request=32} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 32 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.811790Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.809795Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.812033Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.812160Z debug pingora_pool::connection:client{request=31} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.812216Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.811512Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 60 } [INFO] [stdout] 2026-03-06T15:30:41.812399Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.811663Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.812379Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.812498Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.812607Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 60 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.812640Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.812258Z debug proxy::pool:client{request=31} connection ConnectionMeta { key: 385506954252831254, id: 31 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.813275Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.812459Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.813421Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.811099Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 62 } [INFO] [stdout] 2026-03-06T15:30:41.812822Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.813403Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.813001Z debug proxy::pool:client{request=32} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.813508Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.813580Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.813111Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 53 } [INFO] [stdout] 2026-03-06T15:30:41.813906Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.813502Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.813949Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.813867Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.814147Z debug proxy::pool:client{request=32} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 32 } [INFO] [stdout] 2026-03-06T15:30:41.814138Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.814342Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.814525Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.814107Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 62 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.813757Z debug proxy::pool:client{request=32} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.812962Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.814258Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 53 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.814341Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.815271Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 64 } [INFO] [stdout] 2026-03-06T15:30:41.814912Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.814797Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.815600Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.815494Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.815140Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.815370Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(65), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.814636Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.816266Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.816159Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.816072Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 64 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.816611Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(65), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.816069Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.815903Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.816928Z debug proxy::pool checking in new conn for 127.0.0.14(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 13141878506853951134, id: 14 } [INFO] [stdout] 2026-03-06T15:30:41.817148Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 51 } [INFO] [stdout] 2026-03-06T15:30:41.817059Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.817018Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.817978Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.818093Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.817926Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.817917Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.817524Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.818629Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.818377Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 13141878506853951134, id: 14 } [INFO] [stdout] 2026-03-06T15:30:41.818711Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.819570Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.819653Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.819034Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 51 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.819648Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.819855Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.819233Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.819185Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.818524Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.820223Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.819928Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.819357Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(65), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.819768Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 65 } [INFO] [stdout] 2026-03-06T15:30:41.820694Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.820111Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.821217Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.820376Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.820244Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.821900Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.821399Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(65) } [INFO] [stdout] 2026-03-06T15:30:41.822134Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.821529Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.820786Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(65), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.822220Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.822526Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.821304Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 63 } [INFO] [stdout] 2026-03-06T15:30:41.822686Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.821846Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.821320Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.820952Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.822785Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.822533Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.823283Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.823405Z debug proxy::pool::test client spent 12ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.823491Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 63 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.823189Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 65 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.823349Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.823144Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.823901Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.822916Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(65) } [INFO] [stdout] 2026-03-06T15:30:41.824207Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.823093Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.824504Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.824614Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.825700Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.825791Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.824344Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 55 } [INFO] [stdout] 2026-03-06T15:30:41.824714Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.825791Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.826061Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.825911Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.824102Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.825916Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.824251Z debug proxy::pool:client{request=33} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 33 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.826529Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.826058Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.826545Z debug proxy::pool:client{request=33} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.826149Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 55 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.826358Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.826827Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.826693Z debug pingora_pool::connection:client{request=32} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.827150Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 67 } [INFO] [stdout] 2026-03-06T15:30:41.826874Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.827527Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.827011Z debug proxy::pool:client{request=33} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 33 } [INFO] [stdout] 2026-03-06T15:30:41.827479Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.827375Z debug proxy::pool:client{request=32} connection ConnectionMeta { key: 385506954252831254, id: 32 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.826376Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.827424Z debug proxy::pool:client{request=33} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.828369Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.827264Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.828462Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.827629Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 67 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.828638Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.827667Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.828184Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.825579Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.828765Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 56 } [INFO] [stdout] 2026-03-06T15:30:41.829151Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.829000Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.829260Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.828621Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.829609Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.829215Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.829290Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.828870Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(67), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.829716Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.829803Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.829931Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.829916Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.830724Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.829850Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 56 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.830298Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.830279Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.831045Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.830593Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.830922Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.830433Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(67), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.830816Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.831807Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.830910Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 68 } [INFO] [stdout] 2026-03-06T15:30:41.831341Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.831937Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.831165Z debug proxy::pool checking in new conn for 127.0.0.15(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 11992319344839659518, id: 15 } [INFO] [stdout] 2026-03-06T15:30:41.832007Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.831888Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.831357Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.832242Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 68 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.832682Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11992319344839659518, id: 15 } [INFO] [stdout] 2026-03-06T15:30:41.832939Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 54 } [INFO] [stdout] 2026-03-06T15:30:41.833124Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.833123Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.832632Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.832522Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.831910Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.831671Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.832439Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.832808Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.833426Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.833514Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.833436Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.834250Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.834058Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.833900Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.834202Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.833355Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 54 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.834924Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 66 } [INFO] [stdout] 2026-03-06T15:30:41.834659Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.834824Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.835030Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.834995Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.835147Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.835334Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.834461Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.834684Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.835186Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.835235Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 66 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.835878Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.835398Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.836124Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.835512Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.833907Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(67), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.835692Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.836931Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(67) } [INFO] [stdout] 2026-03-06T15:30:41.835716Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.835943Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.836093Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.836216Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.837326Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(67), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.837718Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.837889Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.837235Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.837490Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.838148Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.838002Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.838344Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.838422Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.838411Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.837613Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.838656Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.838479Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.839195Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.838250Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.839438Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.839557Z debug proxy::pool::test client spent 15ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.839454Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(67) } [INFO] [stdout] 2026-03-06T15:30:41.838637Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.839349Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.838923Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.839909Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.839835Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.840594Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.840890Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.840941Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.840493Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.841269Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.840375Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.841094Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.840213Z debug proxy::pool:client{request=34} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 34 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.840769Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.841429Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.841643Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.841566Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.842231Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.841450Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.841796Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.842150Z debug pingora_pool::connection:client{request=33} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.843022Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.842080Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.842293Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.842757Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.843126Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.842347Z debug proxy::pool:client{request=34} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.843271Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.842639Z debug proxy::pool checking in new conn for 127.0.0.16(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 1496265549087212537, id: 16 } [INFO] [stdout] 2026-03-06T15:30:41.842836Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.843010Z debug proxy::pool:client{request=33} connection ConnectionMeta { key: 385506954252831254, id: 33 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.843427Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.843302Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(15) } [INFO] [stdout] 2026-03-06T15:30:41.844186Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 1496265549087212537, id: 16 } [INFO] [stdout] 2026-03-06T15:30:41.843660Z debug proxy::pool:client{request=34} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 34 } [INFO] [stdout] 2026-03-06T15:30:41.845194Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.845430Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.844137Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.844336Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.844105Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.844585Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.844182Z debug proxy::pool:client{request=34} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.845675Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.845971Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.846021Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.845713Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.845716Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.846482Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.846526Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.846655Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.846339Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(69), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.846554Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.847128Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.847175Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.847447Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.847299Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.846230Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 70 } [INFO] [stdout] 2026-03-06T15:30:41.847690Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.847724Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.847593Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.847788Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 70 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.847792Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.847921Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.847486Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(69), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.847863Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.847222Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.848155Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.847991Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 69 } [INFO] [stdout] 2026-03-06T15:30:41.848382Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.848409Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.847988Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 71 } [INFO] [stdout] 2026-03-06T15:30:41.849558Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.848292Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.849690Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 71 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.849468Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.850083Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.849312Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.849469Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.851337Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(69), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.851705Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.849964Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.851866Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.851965Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.852022Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.852087Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.852144Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.851831Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.852265Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 69 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.852201Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.852204Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.852798Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.852874Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.852935Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.853000Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.853079Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.853294Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(69) } [INFO] [stdout] 2026-03-06T15:30:41.853339Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.853466Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.852629Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(69), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.851787Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.853170Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.854184Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.852800Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 58 } [INFO] [stdout] 2026-03-06T15:30:41.853592Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.851135Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 73 } [INFO] [stdout] 2026-03-06T15:30:41.854508Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.854090Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.854328Z debug proxy::pool::test client spent 14ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.854471Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.854704Z debug proxy::pool checking in new conn for 127.0.0.17(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 8068171329476349730, id: 17 } [INFO] [stdout] 2026-03-06T15:30:41.854805Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.853920Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.855098Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 73 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.854976Z debug proxy::pool:client{request=35} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 35 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.853941Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(69) } [INFO] [stdout] 2026-03-06T15:30:41.855710Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.855308Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.855709Z debug proxy::pool:client{request=35} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.854616Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.854930Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.856088Z debug pingora_pool::connection:client{request=34} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.856826Z debug proxy::pool:client{request=34} connection ConnectionMeta { key: 385506954252831254, id: 34 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.856319Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 8068171329476349730, id: 17 } [INFO] [stdout] 2026-03-06T15:30:41.855983Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 74 } [INFO] [stdout] 2026-03-06T15:30:41.854301Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.856715Z debug proxy::pool:client{request=35} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 35 } [INFO] [stdout] 2026-03-06T15:30:41.855962Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.857111Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.857377Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 74 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.857215Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.857485Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.856605Z debug proxy::pool:client{request=35} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.849529Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.856727Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 58 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.857689Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 72 } [INFO] [stdout] 2026-03-06T15:30:41.857648Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.856888Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.856441Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.858053Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel1 [INFO] [stdout] 2026-03-06T15:30:41.858036Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 76 } [INFO] [stdout] 2026-03-06T15:30:41.858244Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.857590Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.858469Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.858660Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.858811Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 76 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.858310Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(71), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.858077Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.858213Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.858546Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.859262Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 72 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.859538Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.858876Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 59 } [INFO] [stdout] 2026-03-06T15:30:41.859348Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.859808Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.858993Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.859710Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 77 } [INFO] [stdout] 2026-03-06T15:30:41.859932Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.859139Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(71), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.860773Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.859714Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.860784Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.861245Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 77 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.861106Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.861349Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.861001Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 59 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.861850Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 79 } [INFO] [stdout] 2026-03-06T15:30:41.861932Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.861997Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 79 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.862020Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(71), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.862167Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.862327Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.862484Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.861751Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 75 } [INFO] [stdout] 2026-03-06T15:30:41.862647Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.862621Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.862798Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.862781Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 75 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.862926Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.862899Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.863065Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.863175Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(71), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.860893Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.861754Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.861659Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.860621Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.863087Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(71) } [INFO] [stdout] 2026-03-06T15:30:41.863692Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.863755Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.864163Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.864224Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.863876Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.864342Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.864274Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.864412Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.864467Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.863415Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.864518Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.864694Z debug proxy::pool::test client spent 9ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.864750Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.864332Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.864834Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(71) } [INFO] [stdout] 2026-03-06T15:30:41.865183Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.864619Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.865222Z debug proxy::pool:client{request=36} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 36 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.864949Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.865587Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.865825Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.865097Z debug proxy::pool checking in new conn for 127.0.0.18(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 12160162382132988306, id: 18 } [INFO] [stdout] 2026-03-06T15:30:41.865323Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(15) } [INFO] [stdout] 2026-03-06T15:30:41.866353Z debug proxy::pool:client{request=36} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.866424Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.866467Z debug pingora_pool::connection:client{request=35} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.866004Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.866664Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.866335Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12160162382132988306, id: 18 } [INFO] [stdout] 2026-03-06T15:30:41.866884Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.866684Z debug proxy::pool:client{request=36} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 36 } [INFO] [stdout] 2026-03-06T15:30:41.865899Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.866977Z debug proxy::pool:client{request=36} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.866908Z debug proxy::pool:client{request=35} connection ConnectionMeta { key: 385506954252831254, id: 35 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.867336Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.866283Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.868091Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.867607Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.867534Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.867024Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.868277Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.867903Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(73), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.868259Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.866921Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.868512Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.868852Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(73), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.869514Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.868406Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.868991Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.868619Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.868138Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.869126Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 80 } [INFO] [stdout] 2026-03-06T15:30:41.869709Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.870559Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.869949Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(73), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.869778Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.870681Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(73) } [INFO] [stdout] 2026-03-06T15:30:41.870812Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.870868Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.870106Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.870200Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.871107Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(73), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.871250Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 80 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.871498Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.871351Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.871617Z debug proxy::pool::test client spent 6ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.870903Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.870418Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.871490Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(73) } [INFO] [stdout] 2026-03-06T15:30:41.870664Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.872013Z debug proxy::pool:client{request=37} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 37 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.872167Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.872279Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.872406Z debug pingora_pool::connection:client{request=36} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.872289Z debug proxy::pool:client{request=37} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.872659Z debug proxy::pool:client{request=37} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.872980Z debug proxy::pool:client{request=37} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 37 } [INFO] [stdout] 2026-03-06T15:30:41.872904Z debug proxy::pool:client{request=36} connection ConnectionMeta { key: 385506954252831254, id: 36 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.873184Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(75), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.873675Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(75), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.873860Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.873924Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.873997Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.874187Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.871924Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.874316Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.874446Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.874699Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.874575Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(75), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.871686Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 78 } [INFO] [stdout] 2026-03-06T15:30:41.874991Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.872400Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.875248Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(75) } [INFO] [stdout] 2026-03-06T15:30:41.875413Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.875121Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(75), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.875672Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.875061Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 78 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.875778Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.875890Z debug proxy::pool::test client spent 3ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.875936Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 82 } [INFO] [stdout] 2026-03-06T15:30:41.875533Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.875292Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.872000Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.876109Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.875972Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(75) } [INFO] [stdout] 2026-03-06T15:30:41.876415Z debug proxy::pool:client{request=38} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 38 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.876489Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.876587Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.876644Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.876690Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.876750Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.876791Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.876848Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.876944Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 82 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.876914Z debug proxy::pool:client{request=38} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.876995Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.876870Z debug pingora_pool::connection:client{request=37} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.877152Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 83 } [INFO] [stdout] 2026-03-06T15:30:41.877209Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.877122Z debug proxy::pool:client{request=37} connection ConnectionMeta { key: 385506954252831254, id: 37 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.877502Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.877319Z debug proxy::pool:client{request=38} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 38 } [INFO] [stdout] 2026-03-06T15:30:41.876186Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.876884Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.877483Z debug proxy::pool:client{request=38} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.877715Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.877893Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.877951Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878004Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878057Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878108Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878141Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878099Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(77), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.878171Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878234Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.877389Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.876393Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.878465Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(77), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.878270Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878749Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878809Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878856Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878905Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.878953Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.879086Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.879368Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.879505Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.879539Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(77), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.879642Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.879773Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.879773Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(77) } [INFO] [stdout] 2026-03-06T15:30:41.880027Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.880136Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.880192Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(77), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.880241Z debug h2::codec::framed_read:Connection{peer=Client} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.880399Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.880416Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(77) } [INFO] [stdout] 2026-03-06T15:30:41.880494Z debug proxy::pool::test client spent 4ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.878990Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.877607Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.881068Z debug proxy::pool:client{request=39} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 39 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.881243Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.878365Z debug proxy::pool checking in new conn for 127.0.0.19(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 12204414564819847720, id: 19 } [INFO] [stdout] 2026-03-06T15:30:41.877256Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 83 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.880840Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.880494Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.881358Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.881488Z debug pingora_pool::connection:client{request=38} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.881965Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.881710Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.881823Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.881567Z debug proxy::pool:client{request=39} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.882622Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 81 } [INFO] [stdout] 2026-03-06T15:30:41.882381Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.882084Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.882290Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12204414564819847720, id: 19 } [INFO] [stdout] 2026-03-06T15:30:41.882712Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.882138Z debug proxy::pool:client{request=38} connection ConnectionMeta { key: 385506954252831254, id: 38 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.882910Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.882876Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 85 } [INFO] [stdout] 2026-03-06T15:30:41.883201Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 81 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.882630Z debug proxy::pool:client{request=39} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 39 } [INFO] [stdout] 2026-03-06T15:30:41.881395Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.882979Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.883027Z debug proxy::pool:client{request=39} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.883528Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.883708Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.883816Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(79), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.883947Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.883957Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.884098Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 85 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.883386Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.884346Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(79), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.883891Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.884019Z debug h2::proto::settings:Connection{peer=Server} received settings ACK; applying Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.885655Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.883585Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.886009Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.886078Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.886118Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.886153Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.885515Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 86 } [INFO] [stdout] 2026-03-06T15:30:41.886358Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.886367Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(79), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.886431Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 86 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.883848Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.886572Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(79) } [INFO] [stdout] 2026-03-06T15:30:41.885289Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.887070Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.887207Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.887394Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 84 } [INFO] [stdout] 2026-03-06T15:30:41.885822Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.887338Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.886852Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(79), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.886986Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.885637Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 57 } [INFO] [stdout] 2026-03-06T15:30:41.887951Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.886197Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.888165Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.888007Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 57 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.887211Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.887598Z debug proxy::pool checking in new conn for 127.0.0.20(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 4875247413344461511, id: 20 } [INFO] [stdout] 2026-03-06T15:30:41.887985Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(79) } [INFO] [stdout] 2026-03-06T15:30:41.887828Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.887706Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.888560Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.888966Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.889119Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.888073Z debug proxy::pool::test client spent 7ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.889479Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.888404Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.889333Z debug proxy::pool:client{request=40} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 40 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.889809Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.889905Z debug proxy::pool::test client spent 628ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.889970Z debug pingora_pool::connection:client{request=39} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.890100Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 84 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.888516Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 88 } [INFO] [stdout] 2026-03-06T15:30:41.890170Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.889665Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.890216Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 88 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.890154Z debug proxy::pool:client{request=40} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.888270Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.890078Z debug proxy::pool:client{request=39} connection ConnectionMeta { key: 385506954252831254, id: 39 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.890785Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.890669Z debug proxy::pool:client{request=40} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 40 } [INFO] [stdout] 2026-03-06T15:30:41.890965Z debug proxy::pool:client{request=40} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.888207Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.890295Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.889995Z debug proxy::pool::test client spent 677ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.891598Z debug proxy::pool::test client spent 670ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.891671Z debug proxy::pool::test client spent 663ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.891185Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.891358Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.891908Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.889621Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.891385Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 89 } [INFO] [stdout] 2026-03-06T15:30:41.892053Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.891938Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(81), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.892116Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 89 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.892553Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.892679Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.892381Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(81), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.892916Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 4875247413344461511, id: 20 } [INFO] [stdout] 2026-03-06T15:30:41.890669Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.893481Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.892276Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(29), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.891726Z debug proxy::pool::test client spent 662ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.893231Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.893690Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.894115Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.894266Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.893886Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(81), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.894410Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.894294Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.894557Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.894656Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(81) } [INFO] [stdout] 2026-03-06T15:30:41.892986Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.895096Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.895150Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.895248Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.894950Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(81), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.895305Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.895178Z debug proxy::pool checking in new conn for 127.0.0.21(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 10261372043896398520, id: 21 } [INFO] [stdout] 2026-03-06T15:30:41.894084Z debug proxy::pool::test client spent 653ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.895795Z debug proxy::pool::test client spent 639ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.891956Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.895994Z debug proxy::pool::test client spent 6ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.895882Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(81) } [INFO] [stdout] 2026-03-06T15:30:41.896371Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.895644Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.896559Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.895890Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.896500Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.895520Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(31), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.895389Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.896786Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.896574Z debug proxy::pool:client{request=41} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 41 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.897166Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.896959Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.897498Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.897342Z debug pingora_pool::connection:client{request=40} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.896443Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.897296Z debug h2::proto::settings:Connection{peer=Client} received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.897248Z debug proxy::pool:client{request=41} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.897536Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.897599Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.897071Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(33), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.897655Z debug proxy::pool:client{request=40} connection ConnectionMeta { key: 385506954252831254, id: 40 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.898910Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(35), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.896889Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.898674Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.899052Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.899156Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.899175Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.899224Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.899271Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.898183Z debug proxy::pool:client{request=41} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 41 } [INFO] [stdout] 2026-03-06T15:30:41.899576Z debug proxy::pool checking in new conn for 127.0.0.22(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 769168371185029786, id: 22 } [INFO] [stdout] 2026-03-06T15:30:41.898270Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.899996Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.900127Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.900206Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.897868Z debug proxy::pool:client{request=41} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.894692Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.900346Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.900559Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.900558Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.899263Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.898113Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.900287Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(37), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.900823Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.901163Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(39), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.901309Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(41), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.901464Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(43), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.901603Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.901683Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.901865Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(83), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.901897Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.902140Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.902173Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(83), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.900983Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.900860Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.902432Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.902848Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.899482Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.902356Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.902283Z debug h2::codec::framed_read:Connection{peer=Client} received frame=Data { stream_id: StreamId(15) } [INFO] [stdout] 2026-03-06T15:30:41.901622Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(45), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.902984Z debug proxy::pool checking in new conn for 127.0.0.23(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 17918551596502831353, id: 23 } [INFO] [stdout] 2026-03-06T15:30:41.903576Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(47), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.903250Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.903264Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.903703Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(49), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.903406Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.903906Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.904074Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.903062Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.904153Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.904265Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.903423Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(83), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.904649Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(51), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.904245Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.904800Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(53), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.904939Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(55), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.903805Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.904608Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(83), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.904771Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.905234Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.905321Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.905271Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.904931Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(83) } [INFO] [stdout] 2026-03-06T15:30:41.905080Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.904491Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 10261372043896398520, id: 21 } [INFO] [stdout] 2026-03-06T15:30:41.904105Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.906075Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.906192Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.906239Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.906372Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(65), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.905640Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.906271Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.906500Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(67), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.906652Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.905607Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.905074Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(57), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.907610Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(83) } [INFO] [stdout] 2026-03-06T15:30:41.907835Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(59), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.906543Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.907942Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.905856Z debug proxy::pool::test client spent 9ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.908027Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908093Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908152Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908216Z debug proxy::pool:client{request=42} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 42 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.908216Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908279Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908337Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908400Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908474Z debug pingora_pool::connection:client{request=41} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.908465Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908543Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.906756Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 87 } [INFO] [stdout] 2026-03-06T15:30:41.908607Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.906810Z debug proxy::pool::test client spent 650ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.908677Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908826Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.908683Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.908820Z debug proxy::pool::test client spent 763ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.907020Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.908914Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.909011Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 87 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.908579Z debug proxy::pool:client{request=41} connection ConnectionMeta { key: 385506954252831254, id: 41 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.909241Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.907985Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(61), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.908342Z debug proxy::pool:client{request=42} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.909366Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 91 } [INFO] [stdout] 2026-03-06T15:30:41.909845Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.909126Z debug proxy::pool::test client spent 699ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.909926Z debug proxy::pool:client{request=42} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.909925Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 91 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.909344Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.908945Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.910068Z debug proxy::pool::test client spent 691ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.910340Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.910527Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.907554Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.910750Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.910837Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(65), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.910460Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.910888Z debug proxy::pool::test client spent 688ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.909685Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.911034Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.911534Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(63), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.911534Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.911617Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.909982Z debug proxy::pool:client{request=42} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 42 } [INFO] [stdout] 2026-03-06T15:30:41.911362Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(85), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.911301Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(67), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.911689Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.911440Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.912193Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(85), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.911695Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(27), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.912005Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(15), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.912575Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.911834Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.912667Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.911243Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.912134Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.912897Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.912551Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.913008Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.912913Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.913138Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.912387Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 17918551596502831353, id: 23 } [INFO] [stdout] 2026-03-06T15:30:41.913390Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 769168371185029786, id: 22 } [INFO] [stdout] 2026-03-06T15:30:41.912785Z debug proxy::pool checking in new conn for 127.0.0.24(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 1895892462465854785, id: 24 } [INFO] [stdout] 2026-03-06T15:30:41.913024Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(5), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.913235Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.913710Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.913644Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(7), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.913561Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.914372Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(9), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.903601Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.914648Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(85), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.914214Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.914791Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel8 [INFO] [stdout] 2026-03-06T15:30:41.913824Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(7), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.914474Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.914963Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(9), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.914930Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.914785Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(85) } [INFO] [stdout] 2026-03-06T15:30:41.913526Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.915055Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.914595Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(11), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.915163Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.914785Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.915061Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(11), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.915185Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.915695Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.915408Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.915283Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(13), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.915748Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(13), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.914059Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.915612Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(85), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.915896Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.915550Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 1895892462465854785, id: 24 } [INFO] [stdout] 2026-03-06T15:30:41.916627Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(15), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.916794Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(17), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.916498Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 92 } [INFO] [stdout] 2026-03-06T15:30:41.917090Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(85) } [INFO] [stdout] 2026-03-06T15:30:41.916018Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.916516Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.917200Z debug proxy::pool::test client spent 9ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.916467Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.916902Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(19), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.917708Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.917613Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.917015Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.918370Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.917713Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.916269Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.918094Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.918295Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.918637Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.918239Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(21), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.918467Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.919001Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(23), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.918536Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.919115Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(25), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.918692Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.917943Z debug proxy::pool:client{request=43} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 43 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.918748Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 92 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.919108Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.920224Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.920366Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.920462Z debug pingora_pool::connection:client{request=42} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.920447Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.920539Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.919215Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(27) } [INFO] [stdout] 2026-03-06T15:30:41.920275Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.920609Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.920878Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.920993Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.918841Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.921496Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.920564Z debug proxy::pool:client{request=42} connection ConnectionMeta { key: 385506954252831254, id: 42 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.921678Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.919319Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.920239Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 96 } [INFO] [stdout] 2026-03-06T15:30:41.921858Z debug proxy::pool::test client spent 693ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.921677Z debug proxy::pool:client{request=43} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.920098Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.921207Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.921315Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.922231Z debug proxy::pool:client{request=43} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.922052Z debug proxy::pool::test client spent 681ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.922530Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(87), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.922488Z debug proxy::pool::test client spent 681ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.922372Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.922921Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(87), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.922808Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.922686Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.921638Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.922294Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.923363Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 100 } [INFO] [stdout] 2026-03-06T15:30:41.923054Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.922403Z debug proxy::pool:client{request=43} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 43 } [INFO] [stdout] 2026-03-06T15:30:41.923184Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.922002Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.923476Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.923696Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 96 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.923388Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.923274Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.923844Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 90 } [INFO] [stdout] 2026-03-06T15:30:41.924325Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.924377Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 90 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.923447Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.924521Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 100 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.924520Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.924646Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.924710Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 101 } [INFO] [stdout] 2026-03-06T15:30:41.923718Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.923168Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.924415Z debug proxy::pool checking in new conn for 127.0.0.25(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 5602206303248432968, id: 25 } [INFO] [stdout] 2026-03-06T15:30:41.924253Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.924906Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.925270Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.924792Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.924896Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.925373Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.925282Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.925521Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.925589Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.925656Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.924199Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.925412Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.925185Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(87), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.925952Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 99 } [INFO] [stdout] 2026-03-06T15:30:41.925718Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.926011Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 101 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.925898Z debug hyper_util TLS handshake succeeded [INFO] [stdout] 2026-03-06T15:30:41.926226Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.925204Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.926300Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.926215Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.926431Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 103 } [INFO] [stdout] 2026-03-06T15:30:41.926097Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.926467Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.926789Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 99 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.925930Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.926580Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.926287Z debug proxy::pool::test server stream started [INFO] [stdout] 2026-03-06T15:30:41.926624Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.927131Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 103 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.926389Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.926692Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(87) } [INFO] [stdout] 2026-03-06T15:30:41.927029Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), max_concurrent_streams: 200, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 65536 } [INFO] [stdout] 2026-03-06T15:30:41.926844Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 94 } [INFO] [stdout] 2026-03-06T15:30:41.927400Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.926896Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.927562Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.927327Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(69), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.927097Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.926752Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.927722Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 94 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.927945Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(71), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.927917Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.928132Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.928052Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.928294Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(87), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.928459Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(87) } [INFO] [stdout] 2026-03-06T15:30:41.927823Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 104 } [INFO] [stdout] 2026-03-06T15:30:41.928038Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(15), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.929282Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.929524Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.928982Z debug proxy::pool::test client spent 11ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.929494Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.928247Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.928893Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.929896Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.929982Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930062Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930123Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930186Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.929809Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.930182Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930543Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930628Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.930616Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930682Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930763Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930827Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.930889Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.929719Z debug proxy::pool:client{request=44} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 44 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.930438Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.931081Z debug proxy::pool:client{request=44} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.930943Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.931208Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.931256Z debug proxy::pool:client{request=44} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.931355Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.931392Z debug proxy::pool:client{request=44} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 44 } [INFO] [stdout] 2026-03-06T15:30:41.930919Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 104 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.931515Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 95 } [INFO] [stdout] 2026-03-06T15:30:41.931409Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.931592Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.931672Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 95 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.931519Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.931807Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(69), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.929246Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.931929Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(71), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.932037Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(73), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.932170Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.932521Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(89), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.931850Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.932254Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.931345Z debug pingora_pool::connection:client{request=43} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.932652Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.932995Z debug proxy::pool:client{request=43} connection ConnectionMeta { key: 385506954252831254, id: 43 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.932989Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.929435Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(1), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.933095Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.933219Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(3), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.933296Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(5), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.933213Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.933390Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(7), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.933392Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.933465Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(9), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.932874Z debug proxy::pool checking in new conn for 127.0.0.26(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 13114846752334268514, id: 26 } [INFO] [stdout] 2026-03-06T15:30:41.933530Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(11), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.930871Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.932222Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 93 } [INFO] [stdout] 2026-03-06T15:30:41.933935Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.933926Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.934056Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.933457Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.934318Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.934389Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.934480Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.934465Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.934535Z debug state Service svc1.ns.svc.cluster.local has no healthy endpoints [INFO] [stdout] 2026-03-06T15:30:41.934228Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.932880Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(89), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.934613Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.933800Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.933985Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 93 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.934882Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Reset { stream_id: StreamId(13), error_code: CANCEL } [INFO] [stdout] 2026-03-06T15:30:41.935229Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.935101Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.935141Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.935750Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.935531Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.934766Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.938977Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.935888Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 106 } [INFO] [stdout] 2026-03-06T15:30:41.939985Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.940086Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.940152Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 106 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.940235Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.940361Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.940452Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(89), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.940632Z debug proxy::pool checking in new conn for 127.0.0.27(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 9945544581471895133, id: 27 } [INFO] [stdout] 2026-03-06T15:30:41.934870Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.941090Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(89), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.941252Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.941541Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.940235Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 97 } [INFO] [stdout] 2026-03-06T15:30:41.941722Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 5602206303248432968, id: 25 } [INFO] [stdout] 2026-03-06T15:30:41.939969Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.941441Z debug proxy::pool::test client spent 11ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.942027Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.941852Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.942223Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.942532Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.939633Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.942335Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 97 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.942714Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.940893Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(89) } [INFO] [stdout] 2026-03-06T15:30:41.942966Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 13114846752334268514, id: 26 } [INFO] [stdout] 2026-03-06T15:30:41.943234Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.943377Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.942865Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 107 } [INFO] [stdout] 2026-03-06T15:30:41.943653Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.943608Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.935616Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 102 } [INFO] [stdout] 2026-03-06T15:30:41.943938Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.943831Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.941011Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.943302Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(89) } [INFO] [stdout] 2026-03-06T15:30:41.942406Z debug proxy::pool:client{request=45} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 45 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.944135Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 9945544581471895133, id: 27 } [INFO] [stdout] 2026-03-06T15:30:41.944438Z debug proxy::pool:client{request=45} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.944624Z debug proxy::pool:client{request=45} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.944251Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.944949Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.945069Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(91), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.945075Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.945317Z debug proxy::pool:client{request=45} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 45 } [INFO] [stdout] 2026-03-06T15:30:41.945574Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(91), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.944629Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.945777Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.945862Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.946148Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(91), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.946530Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.946828Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.946958Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.947246Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.942596Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.949065Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.944881Z debug pingora_pool::connection:client{request=44} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.949256Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.949535Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.948924Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.943987Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 102 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.949398Z debug proxy::pool:client{request=44} connection ConnectionMeta { key: 385506954252831254, id: 44 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.949961Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.950115Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.947413Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(91) } [INFO] [stdout] 2026-03-06T15:30:41.950313Z debug proxy::pool checking in new conn for 127.0.0.28(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 17527473550082483247, id: 28 } [INFO] [stdout] 2026-03-06T15:30:41.943772Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.950669Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.949824Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.949294Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.951006Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.949801Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.950792Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 107 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.951169Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.951562Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.950696Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(91), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.951127Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.951679Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(65), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.951991Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.952000Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(67), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.950860Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.950888Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 105 } [INFO] [stdout] 2026-03-06T15:30:41.952311Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(91) } [INFO] [stdout] 2026-03-06T15:30:41.951838Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.952156Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(69), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.952392Z debug proxy::pool::test client spent 10ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.952768Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(71), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.952594Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.951758Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.952752Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.952889Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(73), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.952433Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.953344Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(75), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.953292Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 105 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.952950Z debug proxy::pool:client{request=46} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 46 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.953493Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.953904Z debug proxy::pool:client{request=46} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.953133Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.954062Z debug proxy::pool:client{request=46} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.953567Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.954682Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.954945Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.955663Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 110 } [INFO] [stdout] 2026-03-06T15:30:41.955660Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.953712Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.955422Z debug pingora_pool::connection:client{request=45} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.953725Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 109 } [INFO] [stdout] 2026-03-06T15:30:41.956157Z debug proxy::pool:client{request=45} connection ConnectionMeta { key: 385506954252831254, id: 45 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.956160Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.956377Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(93), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.955985Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.956771Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.955816Z debug proxy::pool:client{request=46} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 46 } [INFO] [stdout] 2026-03-06T15:30:41.955316Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.955760Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.958888Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.956990Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.955503Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.959007Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 110 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.959620Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.956768Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(93), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.959776Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.956270Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 109 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.959776Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 108 } [INFO] [stdout] 2026-03-06T15:30:41.960106Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 98 } [INFO] [stdout] 2026-03-06T15:30:41.960200Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.960214Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.960033Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.960303Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 98 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.959959Z debug proxy::pool checking in new conn for 127.0.0.29(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 12249195444314390149, id: 29 } [INFO] [stdout] 2026-03-06T15:30:41.960642Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(93), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.959450Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.960458Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(73), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.959229Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.960273Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 108 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.960964Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(75), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.961523Z debug rustls::client::hs No cached session for IpAddress(V4(Ipv4Addr([0, 0, 0, 0]))) [INFO] [stdout] 2026-03-06T15:30:41.960816Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.961831Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.961233Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 113 } [INFO] [stdout] 2026-03-06T15:30:41.961778Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.962106Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.961938Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.962208Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 113 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.962303Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 115 } [INFO] [stdout] 2026-03-06T15:30:41.962468Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(75), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.960799Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(93) } [INFO] [stdout] 2026-03-06T15:30:41.960135Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.962719Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(77), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.963006Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 111 } [INFO] [stdout] 2026-03-06T15:30:41.961188Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 112 } [INFO] [stdout] 2026-03-06T15:30:41.963129Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 116 } [INFO] [stdout] 2026-03-06T15:30:41.963230Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.962956Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.961605Z debug rustls::client::hs Not resuming any session [INFO] [stdout] 2026-03-06T15:30:41.963443Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(93), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.964932Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.965190Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.965374Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.965354Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.962524Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.965482Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.965640Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.967406Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.967389Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.965057Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 112 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.965863Z debug proxy::pool::test client spent 13ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.968507Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.967528Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.964688Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.968897Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.969015Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.968722Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.965760Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(93) } [INFO] [stdout] 2026-03-06T15:30:41.969555Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(77), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.968570Z debug proxy::pool:client{request=47} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 47 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.967925Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.969135Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.969355Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.969769Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(79), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.968291Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.972569Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.967223Z debug xds handling insert [INFO] [stdout] 2026-03-06T15:30:41.971833Z debug pingora_pool::connection:client{request=46} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.969772Z debug proxy::pool:client{request=47} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.972424Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.972065Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.972148Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 17527473550082483247, id: 28 } [INFO] [stdout] 2026-03-06T15:30:41.970309Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.972711Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.974182Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.974062Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 114 } [INFO] [stdout] 2026-03-06T15:30:41.974594Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.974239Z debug proxy::pool:client{request=47} starting an idle timeout for connection ConnectionMeta { key: 385506954252831254, id: 47 } [INFO] [stdout] 2026-03-06T15:30:41.974684Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 114 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.972195Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.974424Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.966477Z debug rustls::webpki::anchors add_parsable_certificates processed 1 valid and 0 invalid certs [INFO] [stdout] 2026-03-06T15:30:41.975131Z debug identity::manager certificate fetch succeeded id=spiffe://cluster.local/ns/istio-system/sa/ztunnel17 [INFO] [stdout] 2026-03-06T15:30:41.974637Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.974810Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.974426Z debug proxy::pool:client{request=46} connection ConnectionMeta { key: 385506954252831254, id: 46 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.974956Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.974911Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 111 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.974042Z debug proxy::pool:client{request=47} initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.975368Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(77), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.975126Z debug proxy::pool checking in new conn for 127.0.0.30(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 16687963005405547643, id: 30 } [INFO] [stdout] 2026-03-06T15:30:41.976074Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(79), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.976438Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.976522Z debug rustls::client::tls13 Not resuming [INFO] [stdout] 2026-03-06T15:30:41.975868Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.976688Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.976881Z debug h2::codec::framed_write:client{request=0}:Connection{peer=Client} send frame=Headers { stream_id: StreamId(95), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.976330Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.977197Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.977262Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(95), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.975248Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.978822Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 115 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.978957Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.979214Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(79), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.976944Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12249195444314390149, id: 29 } [INFO] [stdout] 2026-03-06T15:30:41.979002Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.975837Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.979683Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.979874Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.980195Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.977613Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 118 } [INFO] [stdout] 2026-03-06T15:30:41.980377Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.980469Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.980523Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 118 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.979395Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(81), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.976792Z debug rustls::client::tls13 TLS1.3 encrypted extensions: ServerExtensions { selected_protocol: SingleProtocolName(ProtocolName(6832)), unknown_extensions: {}, .. } [INFO] [stdout] 2026-03-06T15:30:41.980900Z debug rustls::client::hs ALPN protocol is Some(b"h2") [INFO] [stdout] 2026-03-06T15:30:41.980939Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 16687963005405547643, id: 30 } [INFO] [stdout] 2026-03-06T15:30:41.980795Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.979329Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.980324Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.982123Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 119 } [INFO] [stdout] 2026-03-06T15:30:41.980777Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.981139Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.982767Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 116 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.982765Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.982236Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Headers { stream_id: StreamId(95), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.982899Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(1) } [INFO] [stdout] 2026-03-06T15:30:41.983080Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Data { stream_id: StreamId(95) } [INFO] [stdout] 2026-03-06T15:30:41.983012Z debug proxy::pool connection ConnectionMeta { key: 11949129260861191224, id: 119 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.982934Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 117 } [INFO] [stdout] 2026-03-06T15:30:41.983787Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.982628Z debug rustls::server::hs Chosen ALPN protocol ProtocolName(6832) [INFO] [stdout] 2026-03-06T15:30:41.983940Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 121 } [INFO] [stdout] 2026-03-06T15:30:41.983619Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.984048Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.981898Z debug h2::client binding client connection [INFO] [stdout] 2026-03-06T15:30:41.984175Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 11949129260861191224, id: 122 } [INFO] [stdout] 2026-03-06T15:30:41.984262Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.984452Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(81), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.984592Z debug h2::client client connection bound [INFO] [stdout] 2026-03-06T15:30:41.984656Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.983785Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.984864Z debug proxy::pool connection ConnectionMeta { key: 12401958628566421154, id: 117 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.985151Z debug h2::codec::framed_write send frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.984702Z debug proxy::pool re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.985036Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 12401958628566421154, id: 120 } [INFO] [stdout] 2026-03-06T15:30:41.983491Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Settings { flags: (0x0), enable_push: 0, initial_window_size: 4194304, max_frame_size: 1048576, max_header_list_size: 16384 } [INFO] [stdout] 2026-03-06T15:30:41.985468Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.986181Z debug h2::codec::framed_write:Connection{peer=Server} send frame=Settings { flags: (0x1: ACK) } [INFO] [stdout] 2026-03-06T15:30:41.983426Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Headers { stream_id: StreamId(95), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.985923Z debug pingora_pool::connection idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.986316Z debug h2::codec::framed_read:Connection{peer=Server} received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.986477Z debug proxy::pool::test client spent 18ms waiting for conn [INFO] [stdout] 2026-03-06T15:30:41.986756Z debug h2::codec::framed_read:Connection{peer=Server} received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.985301Z debug proxy::pool checking in new conn for 127.0.0.31(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:36925[spiffe://cluster.local/ns/istio-system/sa/ztunnel] with pk ConnectionMeta { key: 11463953853427422205, id: 31 } [INFO] [stdout] 2026-03-06T15:30:41.986833Z debug proxy::pool connection ConnectionMeta { key: 230488702111320817, id: 121 } was removed/checked out/timed out of the pool [INFO] [stdout] 2026-03-06T15:30:41.986985Z debug h2::codec::framed_write:Connection{peer=Server} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.986210Z debug proxy::pool re-using connection for 127.0.0.2(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.985748Z debug h2::codec::framed_write:Connection{peer=Client} send frame=Headers { stream_id: StreamId(81), flags: (0x4: END_HEADERS) } [INFO] [stdout] 2026-03-06T15:30:41.986383Z debug h2::codec::framed_read:client{request=0}:Connection{peer=Client} received frame=Data { stream_id: StreamId(95) } [INFO] [stdout] 2026-03-06T15:30:41.987439Z debug proxy::pool::test hello world: received request [INFO] [stdout] 2026-03-06T15:30:41.986813Z debug proxy::pool:client{request=48} checkout - found mutex for pool key ConnectionMeta { key: 385506954252831254, id: 48 }, waiting for writelock [INFO] [stdout] 2026-03-06T15:30:41.986034Z debug rustls::server::hs decided upon suite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.988315Z debug h2::codec::framed_write:Connection{peer=Client} send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 16711681 } [INFO] [stdout] 2026-03-06T15:30:41.987313Z debug proxy::pool initial attempt - found existing conn, done [INFO] [stdout] 2026-03-06T15:30:41.988190Z debug pingora_pool::connection:client{request=47} idle connection is being picked up [INFO] [stdout] 2026-03-06T15:30:41.987127Z debug rustls::client::hs Using ciphersuite TLS13_AES_256_GCM_SHA384 [INFO] [stdout] 2026-03-06T15:30:41.987610Z debug proxy::pool starting an idle timeout for connection ConnectionMeta { key: 230488702111320817, id: 124 } [INFO] [stdout] 2026-03-06T15:30:41.988877Z debug proxy::pool re-using connection for 127.0.0.0(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:42837[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.988654Z debug proxy::pool:client{request=48} re-using connection for 127.0.0.1(spiffe://cluster.local/ns/istio-system/sa/ztunnel)->127.0.0.1:38427[spiffe://cluster.local/ns/istio-system/sa/ztunnel] [INFO] [stdout] 2026-03-06T15:30:41.989376Z debug proxy::pool initial attempt - found existing conn, done [WARN] too much data in the log, truncating it