Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
17d150d
initial chkpoint of implementation state
PhilipBAdams Feb 9, 2026
6e6d237
implementation of SQPoll and more testing
PhilipBAdams Feb 9, 2026
c051711
some simplifying refactoring
PhilipBAdams Feb 9, 2026
c197e56
fix overflow and total order problems
PhilipBAdams Feb 9, 2026
338f1dd
wait completions, filtering, relaxed mono drain
PhilipBAdams Feb 10, 2026
7bb9191
Queue-based ExpandBeam: unified pipelined search via submit_expand/ex…
PhilipBAdams Feb 10, 2026
3d1ade7
Add node cache integration and IO tracking to PipelinedDiskAccessor
PhilipBAdams Feb 10, 2026
585bfd6
Fix slot overflow and heap corruption in pipelined IO
PhilipBAdams Feb 10, 2026
ef02641
Expose relaxed_monotonicity_l for UnifiedPipeSearch
PhilipBAdams Feb 10, 2026
47e9413
Make adaptive_beam_width configurable for UnifiedPipeSearch
PhilipBAdams Feb 10, 2026
f0ae5af
Add up_to parameter to expand_available for pipeline granularity
PhilipBAdams Feb 10, 2026
0a46791
Pool io_uring reader and PQ scratch across queries
PhilipBAdams Feb 10, 2026
375500b
Non-blocking expand, ranked expansion, spin_loop hint
PhilipBAdams Feb 11, 2026
a3c50e8
Add SearchTrace: per-query tracing and profiling infrastructure
PhilipBAdams Feb 11, 2026
39b7ba7
Wire DISKANN_TRACE env var for per-query profiling
PhilipBAdams Feb 11, 2026
08b4f88
Decouple IO submission from node visitation
PhilipBAdams Feb 11, 2026
bbce453
Fix decoupled submission: is_pipelined flag, drop stale nodes
PhilipBAdams Feb 11, 2026
cfae612
Match PipeANN recall: full_retset reranking, send-1 pacing, FIFO comp…
PhilipBAdams Feb 11, 2026
8ad3f01
Optimize pipelined search: batch completions, rank-based expansion, I…
PhilipBAdams Feb 11, 2026
4c499c1
Clean up: consolidate PipeSearch into unified implementation
PhilipBAdams Feb 11, 2026
24a2d32
Add SIFT1M benchmark script for ablation testing
PhilipBAdams Feb 11, 2026
a030ae7
Fix benchmark script: add search_phase to build config, thread sweep …
PhilipBAdams Feb 11, 2026
c9abe71
more ablation
PhilipBAdams Feb 11, 2026
6f3d4bb
Revert "more ablation"
PhilipBAdams Feb 11, 2026
65655ef
track PQ preprocess time in pipelined path
PhilipBAdams Feb 11, 2026
80c5ff1
Eliminate per-query allocations in pipelined search
PhilipBAdams Feb 11, 2026
1ca067b
Pool LoadedNode instances to eliminate per-IO Vec allocations
PhilipBAdams Feb 11, 2026
028e215
Remove old standalone PipelinedSearcher implementation
PhilipBAdams Feb 11, 2026
03acde7
remove search tracing
PhilipBAdams Feb 11, 2026
948c5c9
cleanup unused
PhilipBAdams Feb 11, 2026
61c8049
Waste-based adaptive beam width matching PipeANN
PhilipBAdams Feb 11, 2026
b5b2eea
Make ABW convergence gate configurable (abw_convergence_hops, default 5)
PhilipBAdams Feb 11, 2026
b538f74
Fix relaxed monotonicity to use max_marker convergence matching PipeANN
PhilipBAdams Feb 11, 2026
3cd317d
Expose ABW and relaxed monotonicity for BeamSearch
PhilipBAdams Feb 11, 2026
91ccb52
Add ABW and RM variants to sift1m benchmark script
PhilipBAdams Feb 11, 2026
d1be70d
Fix review findings: slot safety, stale CQEs, cursor optimization
PhilipBAdams Feb 11, 2026
0d1c4ea
remove benchmarking files
PhilipBAdams Feb 11, 2026
2ee89bc
Fix clippy warnings and fmt cleanup
PhilipBAdams Feb 11, 2026
554ece9
Remove adaptive beam width and relaxed monotonicity
PhilipBAdams Feb 12, 2026
3925956
Remove peek_best_unsubmitted_with_position
PhilipBAdams Feb 12, 2026
aaab5ca
Unify search_internal into a single loop path
PhilipBAdams Feb 12, 2026
a27d719
Replace unused id_scratch with neighbors/submitted buffers in SearchS…
PhilipBAdams Feb 12, 2026
ec24282
Tri-state NodeState queue: eliminate submitted HashSet, fix silent drop
PhilipBAdams Feb 12, 2026
3cd039c
Review fixes: fuse peek+mark, propagate IO errors, eliminate allocations
PhilipBAdams Feb 12, 2026
664f0c1
fix(pipelined_reader): resolve io_uring safety issues from PR #769 re…
PhilipBAdams Feb 13, 2026
7ff0962
fmt
PhilipBAdams Feb 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -333,4 +333,4 @@ target/
*.info

# ignore VS Code local history
.history/
.history/
17 changes: 14 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

79 changes: 79 additions & 0 deletions diskann-benchmark/example/pipe-search.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
{
"search_directories": [
"test_data/disk_index_search"
],
"jobs": [
{
"type": "disk-index",
"content": {
"source": {
"disk-index-source": "Load",
"data_type": "float32",
"load_path": "test_data/disk_index_search/disk_index_sift_learn_R4_L50_A1.2_truth_search"
},
"search_phase": {
"queries": "disk_index_sample_query_10pts.fbin",
"groundtruth": "disk_index_10pts_idx_uint32_truth_search_res.bin",
"search_list": [10, 20, 40, 80],
"beam_width": 4,
"recall_at": 10,
"num_threads": 1,
"is_flat_search": false,
"distance": "squared_l2",
"vector_filters_file": null,
"search_mode": {
"mode": "BeamSearch"
}
}
}
},
{
"type": "disk-index",
"content": {
"source": {
"disk-index-source": "Load",
"data_type": "float32",
"load_path": "test_data/disk_index_search/disk_index_sift_learn_R4_L50_A1.2_truth_search"
},
"search_phase": {
"queries": "disk_index_sample_query_10pts.fbin",
"groundtruth": "disk_index_10pts_idx_uint32_truth_search_res.bin",
"search_list": [10, 20, 40, 80],
"beam_width": 4,
"recall_at": 10,
"num_threads": 1,
"is_flat_search": false,
"distance": "squared_l2",
"vector_filters_file": null,
"search_mode": {
"mode": "PipeSearch"
}
}
}
},
{
"type": "disk-index",
"content": {
"source": {
"disk-index-source": "Load",
"data_type": "float32",
"load_path": "test_data/disk_index_search/disk_index_sift_learn_R4_L50_A1.2_truth_search"
},
"search_phase": {
"queries": "disk_index_sample_query_10pts.fbin",
"groundtruth": "disk_index_10pts_idx_uint32_truth_search_res.bin",
"search_list": [10, 20, 40, 80],
"beam_width": 4,
"recall_at": 10,
"num_threads": 1,
"is_flat_search": false,
"distance": "squared_l2",
"vector_filters_file": null,
"search_mode": {
"mode": "PipeSearch"
}
}
}
}
]
}
Loading
Loading