feat: configurable timeouts and concurrency with lenient defaults for high-burst workloads

[VWang1111]

Summary

• Expose connectionTimeout, connectionAcquireTimeout, subscriptionTimeout, and maxConcurrency on SageMakerConfig.Builder. Each is validated > 0 / non-null at build time. Public DEFAULT_* constants on SageMakerConfig document the chosen defaults.
• Wire them through SageMakerTransportFactory (Netty connectionTimeout / connectionAcquisitionTimeout / maxConcurrency) and SageMakerTransport (input-publisher subscription wait at ensureConnected()).
• Move defaults to values tuned for high-burst workloads instead of inheriting the AWS Netty defaults, which fail-fast in a way that's actively wrong for SageMaker bidi streaming under bursts.

Knob	Old default	New default	Why
connectionTimeout	AWS Netty default = 2 s	30 s	Cold-start endpoints under a 200–500-stream burst can't accept TLS handshakes in 2 s
connectionAcquireTimeout	AWS Netty default = 10 s	60 s	A 400-stream burst drains the acquire pool past 10 s
subscriptionTimeout	(was hardcoded 30 s)	60 s	Slow first-stream subscribe under burst
maxConcurrency	(was hardcoded 500)	500 (now tunable)	No change — just made it tunable

Why this matters

Empirically validated against a 400-concurrent-stream burst test on a 10× ml.g6.2xlarge endpoint, replicating a customer report:

Error category	Before (Maven Central 0.1.2 + AWS defaults)	After (this PR)
Connection pool exhausted (Acquire took longer)	3,230	~0
TCP connect timed out	706	~0
TCP Connection reset	3,528	sharply down

Out of the box, callers no longer have to know that AWS's general-purpose Netty defaults bite hard on the SageMaker streaming path; the transport ships sane-for-its-use-case defaults. Anyone who wants fail-fast behavior can tighten the knobs explicitly:

SageMakerConfig config = SageMakerConfig.builder()
    .endpointName("my-deepgram-endpoint")
    .region("us-east-2")
    .connectionTimeout(Duration.ofSeconds(5))
    .connectionAcquireTimeout(Duration.ofSeconds(15))
    .build();

Scope and what this does NOT fix

This PR addresses the transport layer only. The SDK-level retry storm in com.deepgram:deepgram-java-sdk (the hardcoded connectionFuture.get(4000, MILLISECONDS) in ReconnectingWebSocketListener.connect() plus the listener's default maxRetries = Integer.MAX_VALUE) is not touched here and was empirically observed to remain even with this PR's transport fixes — the bottleneck simply moves inward to the SDK ceiling.

The full burst-handling fix requires pairing this PR with the corresponding change in deepgram-java-sdk that auto-disables ReconnectingWebSocketListener when a custom transport is configured (ClientOptions.reconnect(false) + auto-set in DeepgramClientBuilder.transportFactory(...)). With both layers in place, end-to-end error counts drop ~97 % at 400 concurrent streams.

Backwards compatibility

Adds new builder methods only; no existing API is removed or modified. Existing callers using SageMakerConfig.builder().endpointName(...).region(...).build() get the new defaults automatically — same calling convention, more lenient runtime behavior. No code changes required for existing applications.

Test plan

• ./gradlew :sagemaker-transport:test — all tests pass (18/18; 4 new tests for defaults, custom values, and validation of non-positive arguments).
• Local build of the patched JAR consumed by a load-test harness against a real SageMaker endpoint.
• Validation run vs. baseline measured a Connection pool exhausted / TCP connect timed out reduction to ~0.
• CI passes.

🤖 Generated with Claude Code

[lukeocodes]

appreciate the burst-test work, those AWS-side numbers going to ~0 are good to see. but on its own this PR doesn't fix the real problem, and deepgram/deepgram-java-sdk#45 on the SDK side is going the wrong way trying to compensate. I've left the detail over there.

the storm exists because every transient AWS error here goes straight to transport.onError(...) at SageMakerTransport.java:100-110 and the SDK happily reconnect-wraps it into Throttling-on-Throttling. the plugin already owns the AWS HTTP/2 client, the bidi stream, the publisher buffering, the lot. it should be the layer that decides what's recoverable and what's terminal. SageMaker should be invisible to the SDK.

what I'd like in here:

1. retry-with-backoff around the bidi stream in ensureConnected() (SageMakerTransport.java:77-129). wrap invokeEndpointWithBidirectionalStream(...) with a budget + classification, so ThrottlingException, connection-pool-exhausted, and transient connect failure all get retried internally and never fire errorListeners.
2. reuse StreamPublisher.pending (:239-290) across reconnect cycles so messages queued during an internal reset are drained on the new stream instead of dropped.
3. only propagate to listeners on terminal errors: auth, protocol, exhausted budget. the handler at :100-110 is the right gate.
4. while in here, inputPublisher.awaitSubscription(...) ignores its boolean return at :121-128. the new subscriptionTimeout doesn't actually fail fast today, it just waits and continues. either honour the return or drop the knob.

once the plugin absorbs storm handling internally, the SDK-side change becomes tiny. basically just the listener fixes I asked for on #45 (configurable connect timeout + clean maxRetries(0) semantics). no reconnect(boolean) flag, no auto-disable heuristic, no edits to generated WS clients. that keeps transportFactory() as the only thing the SDK knows about SageMaker, which is where I'd like to land.

the 30s / 60s / 60s / 500 defaults look fine. if anything the missing config surface here is retry budget + backoff, not looser timeouts.

approving the netty wiring + new knobs in principle, but I'd like the retry/classification work landed in this PR (or one stacked on it) before we touch #45.

[VWang1111]

Added robust error connection / buffer handling. Tested at high concurrency.