perf: tune badger defaults and add db bench #2950
Conversation
|
The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).
|
|
Claude finished @tac0turtle's task in 2m 38s —— View job Code Review CompleteI've reviewed the BadgerDB optimization and benchmarking tool changes. Overall, this is a well-implemented performance improvement with good documentation and a useful benchmarking tool. Here's my detailed feedback: SummaryStrengths:
Areas for Improvement:
1. BadgerDB Options Tuning (
|
Summary of ChangesHello @tac0turtle, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request enhances the performance of the BadgerDB key-value store by introducing a set of finely-tuned default options tailored for the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces performance tuning for BadgerDB by providing a set of default options tailored for the application's write patterns. It also adds a new benchmark tool, db-bench, to validate these performance improvements. The changes are well-structured and the new options are correctly applied. The benchmark tool is a valuable addition. I have provided a couple of suggestions to improve the implementation of the benchmark tool for better error handling and code clarity.
| for _, p := range selected { | ||
| profileDir := filepath.Join(baseDir, p.name) | ||
| if cfg.reset { | ||
| _ = os.RemoveAll(profileDir) |
There was a problem hiding this comment.
The error returned by os.RemoveAll is ignored. If removing the directory fails (e.g., due to permissions), the benchmark might run with old data or fail at a later stage, leading to confusing results. It would be better to handle this error, for instance by logging a warning to the user.
| _ = os.RemoveAll(profileDir) | |
| if err := os.RemoveAll(profileDir); err != nil { | |
| fmt.Fprintf(os.Stderr, "Warning: failed to remove profile directory %s: %v\n", profileDir, err) | |
| } |
| kv, err := p.open(dir) | ||
| if err != nil { | ||
| return result{}, fmt.Errorf("failed to open db: %w", err) | ||
| } | ||
|
|
||
| ctx := context.Background() | ||
| start := time.Now() | ||
|
|
||
| batch, err := kv.Batch(ctx) | ||
| if err != nil { | ||
| _ = kv.Close() | ||
| return result{}, fmt.Errorf("failed to create batch: %w", err) | ||
| } |
There was a problem hiding this comment.
The manual calls to kv.Close() on various execution paths are repetitive and can be simplified using a defer statement. This ensures the database connection is always closed, making the code cleaner and more robust.
This suggestion adds defer kv.Close() and cleans up the first error handling block. Please also remove the other manual kv.Close() calls throughout this function for consistency (on lines 157, 164, 169, 178, 184, and the block on 188-190).
| kv, err := p.open(dir) | |
| if err != nil { | |
| return result{}, fmt.Errorf("failed to open db: %w", err) | |
| } | |
| ctx := context.Background() | |
| start := time.Now() | |
| batch, err := kv.Batch(ctx) | |
| if err != nil { | |
| _ = kv.Close() | |
| return result{}, fmt.Errorf("failed to create batch: %w", err) | |
| } | |
| kv, err := p.open(dir) | |
| if err != nil { | |
| return result{}, fmt.Errorf("failed to open db: %w", err) | |
| } | |
| defer kv.Close() | |
| ctx := context.Background() | |
| start := time.Now() | |
| batch, err := kv.Batch(ctx) | |
| if err != nil { | |
| return result{}, fmt.Errorf("failed to create batch: %w", err) | |
| } |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2950 +/- ##
==========================================
- Coverage 59.15% 57.66% -1.49%
==========================================
Files 90 92 +2
Lines 8632 8900 +268
==========================================
+ Hits 5106 5132 +26
- Misses 2944 3179 +235
- Partials 582 589 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Overview
Tune badgerdb for our write heavy usecase, this is a small change i dont believe we need to go beyond this
closes #2605