⚡️ Speed up function find_last_node by 3,916%#264
Closed
codeflash-ai[bot] wants to merge 1 commit intomainfrom
Closed
⚡️ Speed up function find_last_node by 3,916%#264codeflash-ai[bot] wants to merge 1 commit intomainfrom
find_last_node by 3,916%#264codeflash-ai[bot] wants to merge 1 commit intomainfrom
Conversation
The optimized code achieves a **39x speedup** by replacing the O(N×M) nested iteration pattern with an O(N+M) algorithm using set-based lookups. **Key optimization:** The original code uses a generator expression with nested `all()` that iterates through ALL edges for EVERY node, resulting in quadratic complexity. For each of the N nodes, it checks all M edges to see if any edge has that node as a source - this is O(N×M). The optimized version builds a set of all source IDs in a single pass through the edges (O(M)), then checks each node against this set in O(1) time (O(N) total). This reduces the overall complexity from O(N×M) to O(N+M). **Performance characteristics based on test results:** 1. **Small graphs (few nodes/edges):** The optimization shows minor slowdowns (0-48% slower) due to the overhead of set construction and the isinstance check. The original's lazy evaluation is more efficient when early-exit is likely. 2. **Large graphs:** Massive speedups are achieved: - 500-node chain: **137x faster** (4.42ms → 31.9μs) - 100-node chain: **21x faster** (200μs → 9.08μs) - Dense 50-node graph: **47x faster** (1.44ms → 29.4μs) 3. **Sparse graphs with early matches:** Moderate speedups (23-45%) as the first node often has no outgoing edges, benefiting less from set lookup. **Special handling:** The code preserves the original's semantics for edge cases: - When `edges` is empty, it returns the first node without accessing `n["id"]` (avoiding KeyError on nodes missing the 'id' key) - When `edges` is an Iterator (single-pass), it falls back to the original algorithm to avoid consuming the iterator for set construction **Impact:** This optimization is particularly valuable in graph analysis pipelines where `find_last_node` is called repeatedly on graphs with hundreds of nodes/edges, which appears common based on the test suite's focus on large graph scenarios.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📄 3,916% (39.16x) speedup for
find_last_nodeinsrc/algorithms/graph.py⏱️ Runtime :
7.62 milliseconds→190 microseconds(best of162runs)📝 Explanation and details
The optimized code achieves a 39x speedup by replacing the O(N×M) nested iteration pattern with an O(N+M) algorithm using set-based lookups.
Key optimization: The original code uses a generator expression with nested
all()that iterates through ALL edges for EVERY node, resulting in quadratic complexity. For each of the N nodes, it checks all M edges to see if any edge has that node as a source - this is O(N×M).The optimized version builds a set of all source IDs in a single pass through the edges (O(M)), then checks each node against this set in O(1) time (O(N) total). This reduces the overall complexity from O(N×M) to O(N+M).
Performance characteristics based on test results:
Small graphs (few nodes/edges): The optimization shows minor slowdowns (0-48% slower) due to the overhead of set construction and the isinstance check. The original's lazy evaluation is more efficient when early-exit is likely.
Large graphs: Massive speedups are achieved:
Sparse graphs with early matches: Moderate speedups (23-45%) as the first node often has no outgoing edges, benefiting less from set lookup.
Special handling: The code preserves the original's semantics for edge cases:
edgesis empty, it returns the first node without accessingn["id"](avoiding KeyError on nodes missing the 'id' key)edgesis an Iterator (single-pass), it falls back to the original algorithm to avoid consuming the iterator for set constructionImpact: This optimization is particularly valuable in graph analysis pipelines where
find_last_nodeis called repeatedly on graphs with hundreds of nodes/edges, which appears common based on the test suite's focus on large graph scenarios.✅ Correctness verification report:
🌀 Click to see Generated Regression Tests
To edit these changes
git checkout codeflash/optimize-find_last_node-mkq33xmoand push.