Is your feature request related to a problem? Please describe.
I'm always frustrated when we have limits in the system that are hard to explain/configure/have any meaningful defaults for. Shard pools are exactly that, they limit concurrency artificially, this limit has no relation to real system capabilities (for example, SSDs used for storage) and product technologies (FSTree combined writer can handle a lot of objects, but we're only giving it like eight).
Describe the solution you'd like
Have a deadline for write operation. Push data into shard as it goes. Collect real operation times observed in a sliding window manner (clustered by size, potentially). Predict operation time based on statistics collected. Start dropping some requests with ErrBusy once predicted time is >80% (60-70-90, depends) of the target one, drop more of them as time grows.
Describe alternatives you've considered
None.
Additional context
Related to #2479 (can also take context deadline into account)?
See https://en.wikipedia.org/wiki/CoDel for inspiration.
Is your feature request related to a problem? Please describe.
I'm always frustrated when we have limits in the system that are hard to explain/configure/have any meaningful defaults for. Shard pools are exactly that, they limit concurrency artificially, this limit has no relation to real system capabilities (for example, SSDs used for storage) and product technologies (FSTree combined writer can handle a lot of objects, but we're only giving it like eight).
Describe the solution you'd like
Have a deadline for write operation. Push data into shard as it goes. Collect real operation times observed in a sliding window manner (clustered by size, potentially). Predict operation time based on statistics collected. Start dropping some requests with ErrBusy once predicted time is >80% (60-70-90, depends) of the target one, drop more of them as time grows.
Describe alternatives you've considered
None.
Additional context
Related to #2479 (can also take context deadline into account)?
See https://en.wikipedia.org/wiki/CoDel for inspiration.