Improve error recovery for QLever query timeouts and parse failures

[ddeboer]

Problem

Running the full dataset-knowledge-graph pipeline reveals gaps in error handling that cause unnecessary failures. Updated with findings from the 2026-03-21 DKG run.

1. QLever query timeouts (HTTP 429) — increase --default-query-timeout

QLever's default query timeout is 30s. On the deployment, heavy VoID aggregation queries on large datasets (78M+ triples) exceed this — locally the same queries complete in ~25s, so a higher timeout is sufficient.

The @lde/sparql-qlever Server class now accepts a queryTimeout option (passed as --default-query-timeout to qlever-server). DKG should set this to a generous value (e.g. '120s').

Retrying 429s would not help here — the same query would just time out again.

119 failed SPARQL stages in the 2026-03-21 run hit HTTP 429, containing:

• 113 "Operation timed out" exceptions
• 52 "Tried to allocate X, but only Y available" (QLever memory limit, --memory-max-size 6G)
• 6 "Sort operation was canceled" (time estimate exceeded remaining time)

Affected datasets (all openarchieven.nl — large datasets):
dataset_aal, dataset_ade, dataset_arg, dataset_bhi, dataset_bor, dataset_brd, dataset_cod, dataset_dar, dataset_den, dataset_dev, dataset_eal, dataset_elo, dataset_eem, dataset_ell, dataset_ens, dataset_frl, dataset_gld, dataset_gmb, dataset_gra, dataset_hco, dataset_hga, dataset_hua, dataset_nha, dataset_raa, dataset_rad, dataset_rar, dataset_rat, dataset_rhe, dataset_rhl, dataset_saa, dataset_sha, dataset_smh, dataset_srt, dataset_wba, dataset_wfa, dataset_zar

2. QLever --parse-parallel false retry too narrow

The retry in packages/sparql-qlever/src/importer.ts only catches the multiline string literal error and only for format === 'ttl'. Other QLever parallel parsing errors also need --parse-parallel false but aren't caught.

The format === 'ttl' restriction should be removed: while the multiline retry was originally motivated by TTL files, the same QLever parallel parsing errors can occur with any format.

Note: the two datasets originally listed here (Krantenpaginas from deventit.coda-apeldoorn.nl, LOD Beelddocumenten from studiezaal.nijmegen.nl) are not fixable by retry logic — both servers serve RDF/XML regardless of the requested format (.ttl/.nt/.nq URL extensions are ignored; content-disposition confirms filename=*.rdf(.gz)). This is a data quality issue at the source, not a QLever parsing problem.

No known datasets currently need this broader retry, but it's still a correctness improvement.

3. --parse-parallel false retry passes flag incorrectly — FIXED in #287

Fixed by replacing the hardcoded -p true + conditional --parse-parallel false with a single --parse-parallel ${parallel}.

2026-03-21 DKG run error summary (baseline)

328 datasets processed, 266 completed successfully, 62 skipped.

Error	Count
Failed SPARQL stages (HTTP 429)	119
— of which: query timeout	113
— of which: memory allocation	52
— of which: sort cancelled	6
Import failed (total)	29
— of which: no supported import format	46 (skipped)
— of which: indexed 0 triples	2
— of which: QLever parse error (RDF/XML as TTL)	2
— of which: --parse-parallel flag error	1
— of which: NQ parse error	1
Network/download timeouts	11

Proposed fixes

1. Add status === 429 to isTransientError(). Not useful — same query would time out again. Instead, expose queryTimeout on Server so DKG can increase the timeout (done in fix(sparql-qlever): fix conflicting --parse-parallel flags on import retry #287).
2. Broaden the QLever retry to also match marks the end of a statement and remove the format === 'ttl' restriction.
3. Fix the --parse-parallel false flag passing. Done in fix(sparql-qlever): fix conflicting --parse-parallel flags on import retry #287.

[ddeboer]

2026-03-23 DKG run analysis

HTTP 429s eliminated

The queryTimeout increase from #287 is working — 0 HTTP 429 errors in this run (down from 119 in the 2026-03-21 baseline).

QLever memory allocation failures (newly visible)

With 429s gone, the local QLever OOM errors are now the main blocker for large datasets. On dataset_bhi (686M triples, --memory-max-size already set to 12G in DKG):

Stage	Needs	Available	Pattern
subjects.rq	4.3 GB	3.4 GB	COUNT(DISTINCT ?s) full scan
object-literals.rq	4.3 GB	3.4 GB	COUNT(DISTINCT ?o) FILTER ISLITERAL full scan
entity-properties.rq	5.5 GB	787 MB	COUNT(DISTINCT ?s), COUNT(DISTINCT ?o) GROUP BY ?p

Plus 6 stages aborted as cascading failures after the OOMs (datatypes.rq, class-properties-subjects.rq, class-properties-objects.rq, class-property-datatypes.rq, class-property-object-classes.rq, class-property-languages.rq).

Query pattern: all failing queries are ungrouped full-scan COUNT(DISTINCT) operations. These require QLever to materialize and deduplicate entire columns in memory. The queries that succeeded on the same dataset (properties.rq, triples.rq, object-uris.rq) either don't use DISTINCT or have fewer unique values to deduplicate. Per-class queries (class-properties-*.rq) use GROUP BY ?class which limits per-group scan size — they didn't run here (aborted after OOM) but would likely succeed.

Memory budget breakdown: the index for 686M triples consumes ~8.6 GB of the 12 GB budget, leaving only 3.4 GB for queries. Minimum needed: 8.6 GB (index) + 5.5 GB (largest query) = ~14.1 GB. Setting --memory-max-size to 14G should work given the 16 Gi container limit (~2 GB left for Node.js/pipeline, which uses ~555 MB RSS).

Remote SPARQL endpoint failures (26)

These were likely present before but masked by the 429s:

Endpoint	Status	Count
colonialcollections.nl nmvw/collection-archives	504	14
data.bibliotheken.nl	503	6
data.bibliotheken.nl	504	1
colonialcollections.nl data-hub/knowledge-graph	504	2
colonialcollections.nl data-hub/knowledge-graph	503	1
druid.datalegend.net IISG (Virtuoso deadlock)	500	1
Other	400	1

Updated error summary (datasets 1–240/328, log truncated)

Error	2026-03-21	2026-03-23
HTTP 429 (QLever timeout)	119	0
QLever OOM (local)	masked	3 (+6 cascading aborts)
Failed SPARQL stages (remote)	—	26
Import failed: no supported format	29	23
Import failed: QLever parse error	3	2
Import failed: indexed 0 triples	2	2
Network/download timeouts	11	11
Datasets skipped	62	58 (so far)