Skip to content

s3 suggestions + widget cache#1987

Merged
sawka merged 3 commits intomainfrom
sawka/s3-suggestions
Feb 18, 2025
Merged

s3 suggestions + widget cache#1987
sawka merged 3 commits intomainfrom
sawka/s3-suggestions

Conversation

@sawka
Copy link
Member

@sawka sawka commented Feb 18, 2025

No description provided.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 18, 2025

Walkthrough

The changes introduce a new mechanism for disposing of suggestions throughout the codebase. In the frontend, a new method is added to handle disposal commands, and several components update their event handling functions. Specifically, method signatures for onSelect have been altered to return a boolean, and error handling has been enhanced with additional null checks and revised control flow in suggestion-related components. In the suggestion package, a caching system for directory listings is implemented using new cache structures, constants, and functions, with adjustments made to directory listing functions to utilize this cache and manage concurrency. Furthermore, new RPC command functions and methods are added across multiple packages (wshclient, wshremote, wshrpctypes, and wshserver) to support the disposal functionality. Additionally, a logging statement in the router has been modified to report the correct route identifier.

✨ Finishing Touches
  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@sawka sawka merged commit da1f8de into main Feb 18, 2025
7 of 8 checks passed
@sawka sawka deleted the sawka/s3-suggestions branch February 18, 2025 23:15
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
pkg/suggestion/filewalk.go (2)

7-19: Consider making cache ticker interval configurable.
The new imports and additions for LRU caching, singleflight, and the ticker-based cleanup look good. However, consider optionally exposing the ticker interval (currently hard-coded to 60 seconds) through a configuration parameter or environment variable. This would allow tuning for different load or latency requirements.


46-54: Potential for an optional shutdown mechanism.
The init() function spawns a background goroutine that runs indefinitely. If the application context ends or the process needs a graceful shutdown, consider providing a way to stop the ticker so resources aren’t left running.

pkg/wshrpc/wshremote/wshremote.go (1)

871-874: Consider improving error handling and documentation.

The implementation could be enhanced in the following ways:

  1. Add error handling for potential errors from DisposeSuggestions
  2. Add documentation comments explaining the method's purpose and behavior

Consider applying this diff:

+// DisposeSuggestionsCommand handles the disposal of suggestions for a specific widget.
+// It cleans up any cached suggestions associated with the given widgetId.
 func (*ServerImpl) DisposeSuggestionsCommand(ctx context.Context, widgetId string) error {
-    suggestion.DisposeSuggestions(ctx, widgetId)
-    return nil
+    if err := suggestion.DisposeSuggestions(ctx, widgetId); err != nil {
+        return fmt.Errorf("failed to dispose suggestions: %w", err)
+    }
+    return nil
 }
pkg/wshrpc/wshserver/wshserver.go (1)

964-967: Consider handling potential errors from DisposeSuggestions.

The function ignores any potential errors from suggestion.DisposeSuggestions. While it may be intentional, it's good practice to at least log errors for debugging purposes.

func (ws *WshServer) DisposeSuggestionsCommand(ctx context.Context, widgetId string) error {
-    suggestion.DisposeSuggestions(ctx, widgetId)
-    return nil
+    if err := suggestion.DisposeSuggestions(ctx, widgetId); err != nil {
+        log.Printf("error disposing suggestions for widget %s: %v", widgetId, err)
+    }
+    return nil
}
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7c25ebf and ad59438.

📒 Files selected for processing (12)
  • frontend/app/store/wshclientapi.ts (1 hunks)
  • frontend/app/suggestion/suggestion.tsx (2 hunks)
  • frontend/app/view/preview/directorypreview.tsx (1 hunks)
  • frontend/app/view/preview/preview.tsx (2 hunks)
  • frontend/app/view/webview/webview.tsx (1 hunks)
  • pkg/suggestion/filewalk.go (1 hunks)
  • pkg/suggestion/suggestion.go (3 hunks)
  • pkg/wshrpc/wshclient/wshclient.go (1 hunks)
  • pkg/wshrpc/wshremote/wshremote.go (1 hunks)
  • pkg/wshrpc/wshrpctypes.go (1 hunks)
  • pkg/wshrpc/wshserver/wshserver.go (1 hunks)
  • pkg/wshutil/wshrouter.go (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • pkg/wshutil/wshrouter.go
⏰ Context from checks skipped due to timeout of 90000ms (3)
  • GitHub Check: Analyze (javascript-typescript)
  • GitHub Check: Analyze (go)
  • GitHub Check: Build for TestDriver.ai
🔇 Additional comments (21)
pkg/suggestion/filewalk.go (10)

24-28: Cache size and TTL seem reasonable.
The defined maxCacheEntries of 20 and a default cacheTTL of 60s provide a decent balance for small-scale caching. This is for typical usage scenarios. If the workload grows, you may want to allow dynamic adjustments or increase sizes.


30-35: Ensure expiration is updated whenever value is refreshed.
The cacheEntry struct is clear. Just make sure it’s always updated consistently when using setCache (which you do). This avoids stale data issues.


37-44: Locking is correctly protecting LRU operations.
Using a single mutex (cacheMu) to protect both the map and the linked list is correct. This code is free of obvious data race hazards in the presence of concurrent readers and writers.


56-66: cleanCache efficiently removes expired data.
The implementation is straightforward: it checks expiration, removes from cacheLRU, and deletes entries from cache. This is correct for an LRU-based approach, so .


68-84: Verify potential overhead of frequent time.Now() calls.
getCache calls time.Now() to check expiration for every read. Although probably negligible at current scale, consider if a large volume of cache lookups might benefit from a coarser approach or caching the current time in the calling code.


86-115: Eviction logic for setCache looks good.
You properly evict the oldest item when the cache is full, remove it from both cacheLRU and cache, and then insert the new entry. This seems well-structured and should maintain LRU correctness.


117-128: cacheDispose for partial invalidation is useful.
Clearing all entries that match widgetId + "|" is straightforward and meets the requirement to free resources on a per-widget basis. Ensure your usage patterns don’t lead to partial key collisions (e.g., widget “abc” and “abc1”).


135-153: Pre-cached results for S3 in listS3Directory.
Fetching directory entries for S3, caching them, and returning a channel is a neat approach. To confirm, ensure that any S3 listing errors fail fast and do not inadvertently populate stale cache data.


155-191: Singleflight group usage prevents duplicate S3 listings.
Wrapping the listing operation in group.Do(key, fn) effectively avoids race conditions and wasted network calls. The code flows logically, returning the cached results if available.


193-261: Consolidated caching for local directories.
Like S3, you use singleflight in listDirectory to avoid redundant I/O for the same path. This should reduce overhead in multi-request or high-concurrency scenarios.

frontend/app/suggestion/suggestion.tsx (2)

15-15: Method signature now returning boolean.
Changing onSelect to return a boolean is a clear improvement, allowing conditional flows such as closing the suggestion panel on success. Ensure all callers implement a meaningful return value.


259-265: Conditionally closing on ENTER.
When the user presses ENTER, you capture the boolean return value from onSelect. If it’s true, you call onClose(). This is an elegant approach, but confirm that returning false in other scenarios (e.g., partial suggestion) is handled correctly in the logic (like leaving the suggestion open when incomplete).

pkg/suggestion/suggestion.go (2)

135-137: DisposeSuggestions is simple and effective.
Forwarding directly to cacheDispose(widgetId) helps keep disposal logic localized in filewalk.go. This is for clarity and straightforward usage.


360-384: Context-based directory fetch logic is correct.
The updated fetchFileSuggestions checks whether data.FileConnection starts with aws: and delegates to listS3Directory or listDirectory accordingly. Using context.WithCancel is a good measure to allow cancellation downstream. No issues found.

frontend/app/store/wshclientapi.ts (1)

125-128: LGTM! Well-structured RPC command implementation.

The new DisposeSuggestionsCommand follows the established pattern for RPC commands, with proper parameter types and error handling.

pkg/wshrpc/wshclient/wshclient.go (1)

157-161: LGTM! Consistent RPC command implementation.

The new DisposeSuggestionsCommand maintains consistency with other RPC commands in the file, using the standard helper function and error handling pattern.

pkg/wshrpc/wshrpctypes.go (1)

208-208: LGTM! Well-placed interface method addition.

The new DisposeSuggestionsCommand method is appropriately placed in the interface, maintaining a logical grouping with related suggestion commands.

frontend/app/view/webview/webview.tsx (1)

619-622: LGTM! Consistent return value handling.

The changes ensure that onSelect always returns a boolean value, aligning with the updated suggestion handling interface. This makes the behavior more predictable and consistent.

frontend/app/view/preview/directorypreview.tsx (1)

815-818: LGTM! Robust null check for fileInfo.name.

The added null check prevents potential runtime errors and includes helpful logging for debugging. This is a good example of defensive programming.

frontend/app/view/preview/preview.tsx (2)

1103-1111: LGTM! Improved route handling and disposal support.

The changes add proper route handling for connections and support for disposing suggestions. The code correctly handles blank connections and AWS connections by setting the route to null.


1144-1156: LGTM! Enhanced selection handling with query string support.

The updated handleSelect function now properly handles:

  1. Null suggestions with blank queries by closing the modal
  2. Null suggestions with non-blank queries by attempting to open the file
  3. Valid suggestions by opening the corresponding file

xxyy2024 pushed a commit to xxyy2024/waveterm_aipy that referenced this pull request Jun 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant