Pingora doesn't use upstream's "max concurrent send streams" when doing H2 to upstream

[pborzenkov]

Describe the bug

It looks like Pingora doesn't respect upstream's "max concurrent send streams" setting sent in the SETTINGS frame and sticks with whatever is configured for a peer. If peer's configuration has a higher value than what the upstream sends, new streams can't be opened (upstream rejects them) and new connections to the upstreams are not opened because Pingora believes the existing connection still has available streams.

Pingora info

Please include the following information about your environment:

Pingora version: 0.3.0
Rust version: 1.73
Operating system version: NixOS 24.05

Steps to reproduce

I've created a small project that reproduces the problem: https://github.com/pborzenkov/pingora-h2-streams

Expected results

New connections are opened to the upstream once existing connections no longer have available streams.

Observed results

New connections are not opened to the upstream.

Additional context

It looks like the problem is here -

pingora/pingora-core/src/connectors/http/v2.rs

Lines 421 to 427 in 8ae4ebb

	.handshake(stream)
	.await
	.or_err(HandshakeError, "during H2 handshake")?;
	debug!("H2 handshake to server done.");
	let ping_timeout_occurred = Arc::new(AtomicBool::new(false));
	let ping_timeout_clone = ping_timeout_occurred.clone();
	let max_allowed_streams = std::cmp::min(max_streams, connection.max_concurrent_send_streams());

According to the docs, the handshake future doesn't wait for the SETTINGS frame from the server:

The handshake future does not wait for the initial settings frame from the server.