Skip to content

Latest commit

 

History

History
169 lines (119 loc) · 4.59 KB

CONFIGURATION.md

File metadata and controls

169 lines (119 loc) · 4.59 KB

Configuration Guide

This document describes the available configuration options for the Coveo Push API Java Client.

Batch Size Configuration

The batch size controls how much data is accumulated before creating a file container and pushing to Coveo. The default is 5 MB. The maximum allowed is 256 MB (Stream API limit).

Configuration Methods

There are two ways to configure the batch size:

1. System Property (Runtime Configuration)

Set the coveo.push.batchSize system property to configure the default batch size globally for all service instances:

Java Command Line:

java -Dcoveo.push.batchSize=134217728 -jar your-application.jar

Within Java Code:

// Set before creating any service instances
System.setProperty("coveo.push.batchSize", "134217728"); // 128 MB in bytes

Maven/Gradle Build:

<!-- pom.xml -->
<properties>
    <argLine>-Dcoveo.push.batchSize=134217728</argLine>
</properties>
// build.gradle
test {
    systemProperty 'coveo.push.batchSize', '134217728'
}

Example Values:

  • 5242880 = 5 MB (default)
  • 268435456 = 256 MB (maximum)
  • 134217728 = 128 MB
  • 67108864 = 64 MB
  • 33554432 = 32 MB
  • 10485760 = 10 MB

2. Constructor Parameter (Per-Instance Configuration)

Pass the maxQueueSize parameter when creating service instances:

// UpdateStreamService with custom 128 MB batch size
UpdateStreamService service = new UpdateStreamService(
    catalogSource,
    backoffOptions,
    null,  // userAgents (optional)
    128 * 1024 * 1024  // 128 MB in bytes
);

// PushService with custom batch size
PushService pushService = new PushService(
    pushEnabledSource,
    backoffOptions,
    128 * 1024 * 1024  // 128 MB
);

// StreamService with custom batch size
StreamService streamService = new StreamService(
    streamEnabledSource,
    backoffOptions,
    null,  // userAgents (optional)
    128 * 1024 * 1024  // 128 MB
);

Configuration Priority

When both methods are used:

  1. Constructor parameter takes precedence (if specified)
  2. System property is used as default (if set)
  3. Built-in default of 5 MB is used otherwise

Validation Rules

All batch size values are validated:

  • ✅ Maximum: 256 MB (268,435,456 bytes) - API limit
  • ✅ Minimum: Greater than 0
  • ❌ Values exceeding 256 MB will throw IllegalArgumentException
  • ❌ Invalid or negative values will throw IllegalArgumentException

Examples

Example 1: Using System Property

// Configure globally via system property
System.setProperty("coveo.push.batchSize", "134217728"); // 128 MB

// All services will use 128 MB by default
UpdateStreamService updateService = new UpdateStreamService(catalogSource, backoffOptions);
PushService pushService = new PushService(pushEnabledSource, backoffOptions);
StreamService streamService = new StreamService(streamEnabledSource, backoffOptions);

Example 2: Override Per Service

// Set global default to 128 MB
System.setProperty("coveo.push.batchSize", "134217728");

// Update service uses global default (128 MB)
UpdateStreamService updateService = new UpdateStreamService(catalogSource, backoffOptions);

// Push service overrides with 64 MB
PushService pushService = new PushService(pushEnabledSource, backoffOptions, 64 * 1024 * 1024);

// Stream service uses global default (128 MB)
StreamService streamService = new StreamService(streamEnabledSource, backoffOptions);

When to Adjust Batch Size

Use smaller batches (32-64 MB) when:

  • Network bandwidth is limited
  • Memory is constrained
  • Processing many small documents
  • You want more frequent progress updates

Use larger batches (128-256 MB) when:

  • Network bandwidth is high
  • Processing large documents or files
  • You want to minimize API calls
  • Maximum throughput is needed

Keep default (5 MB) when:

  • You're unsure
  • Memory is a concern
  • You want predictable, frequent pushes

Configuration Property Reference

Property Name Description Default Value Valid Range
coveo.push.batchSize Default batch size in bytes 5242880 (5 MB) 1 to 268435456

Additional Configuration

Environment Variables

The following environment variables can be used for general configuration:

  • COVEO_API_KEY - API key for authentication
  • COVEO_ORGANIZATION_ID - Organization identifier
  • COVEO_PLATFORM_URL - Custom platform URL (if needed)

Refer to the Coveo Platform documentation for complete environment configuration options.