Excessive per-connection memory cause by channel

[howardjohn]

This is not a bug or anything, or even really behaving wrong, just an optimization potential.

Each HTTP/1.1 channel allocates an obvious 8k from INIT_BUFFER_SIZE: usize = 8192; this is fine and expected (although perhaps could use some configuration for "many connections with small payloads/less stringent performance requirements").

However, there is another ~8kb allocated as well per connection that I don't think is desirable. Each connection gets an unbounded dispatch channel. This in turn creates a channel that has 32 blocks (https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/mod.rs#L140 https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/block.rs#L47).

Each http request is ~250b, giving us another ~8kb for the channel here.

While in theory the SendRequest API does allow pushing a bunch of requests at once, practically speaking most are using this in front of a pooling client (hyper_util or otherwise) which is going to only send requests that are ready. So I would think we don't need for most cases unbounded, nor a 32 block buffer.

A theoretical nice option here, that allows the strange case of sending multiple requests but optimizes the common case, would be to just have the channel not use 32 blocks (which I imagine is for performance reasons not relevant to us?). However, tokio doesn't expose any such option.

[seanmonstar]

Very good point. We really only need one or two slots on the http1 dispatch channel. (The http2 could need more, since its SendRequest is clone). However, even if we set mpsc::channel(2), it would still allocate with the block size hard coded.

Another option is to use futures::mpsc, which only allocates a slot on demand as a linked list. The downside, and I think it was noticeable in benchmarks (such as end_to_end), is that means an allocation on every single send.

[howardjohn]

Yeah I found the same dilemma. It seems there could be a theoretical channel that achieves exactly what we need but I didn't find one at least in mainstream crates (crossbeam included; it uses the same 32 chunks). Probably not worth trying to find a less mature crate that meets this (or implementing)

[seanmonstar]

Somewhat related, just linking them up: #3309