Add Graphic Walker integration for interactive data exploration#409
Open
Edwardvaneechoud wants to merge 10 commits into
Open
Add Graphic Walker integration for interactive data exploration#409Edwardvaneechoud wants to merge 10 commits into
Edwardvaneechoud wants to merge 10 commits into
Conversation
Replaces the AG Grid pass-through for explore_data nodes in the Pyodide-based browser build with a full Graphic Walker interactive visualization panel, mirroring the existing flowfile_frontend integration. - Add execute_explore_data Python function (pyodide-store.ts) that materialises up to 100k rows inside Pyodide, builds the MutField list from Polars dtypes (quantitative/temporal/nominal), and ISO-serialises temporal values so they survive Pyodide dict conversion. - Persist saved chart specs on the node settings (specList rehydrates on reload via the new updateNodeSettingsSilent store helper, which updates settings without invalidating caches or triggering re-execution). - Add ExploreData.vue host component + near-verbatim copy of the flowfile_frontend VueGraphicWalker.vue React-in-Vue wrapper, both lazy-loaded via defineAsyncComponent so React + GraphicWalker never enter the initial bundle (confirmed: separate chunk at dist/assets/graphic-walker.es-*.js). - Wire the theme store into Graphic Walker's appearance prop so light/ dark/system mode follows the rest of the wasm app. - Show a truncation banner when datasets exceed the 100k row cap. - Skip the AG Grid bottom preview fetch for explore_data nodes since Graphic Walker is now their canonical view.
✅ Deploy Preview for flowfile-wasm canceled.
|
The explore_data node was still labeled "Preview" with the binoculars (view.png) icon from when it was a pass-through preview, which made it hard to discover as the Graphic Walker visualization node. Rename it to "Explore Data", wire up the existing explore_data.png icon asset, and update the help text to describe the Graphic Walker integration.
propagateSchemas() runs after executeFlow() and, for nodes whose schema can be inferred from their input (including explore_data, which just passes the schema through), rebuilds the NodeResult with a whitelist of fields. graphic_walker_input and row_info weren't in that whitelist, so the Graphic Walker payload produced by execute_explore_data was wiped immediately after execution - the UI rendered "Data loaded, but the dataset is empty or has no fields" every time. Preserve both fields when the inferred-schema branch rebuilds the result.
Two related fixes for the explore_data node: 1. Chart specs were not persisted on flow save. Graphic Walker stores chart edits in its internal React store, not in node settings, so unless the user clicked Save chart beforehand the exported YAML had an empty specList. Fixed with a before-export hook system: flow-store exposes registerBeforeExportHook, downloadFlowfile is now async and awaits all registered hooks before serialising, and ExploreData.vue registers a hook on mount that flushes the live Graphic Walker spec into node settings via updateNodeSettingsSilent. handleSaveFlow in Canvas.vue now awaits the async download. 2. No fullscreen mode. The Graphic Walker canvas inside the DraggablePanel was too cramped to use in practice. Added a fullscreen toggle that teleports the GW component to a viewport-covering overlay, with an Escape-key exit. Toggling fullscreen re-mounts the GW component, so saveCurrentSpec runs first (flushing the React-side spec into node settings) and the re-mount reads specList from settings - which also means specList now reactively updates from settings rather than the result payload, keeping fullscreen toggles and save clicks in sync.
The inline Graphic Walker in the node settings panel was cramped and showed a bright-white panel in dark mode, and the custom buttons had hardcoded colors that didn't respect the theme. Rework the panel so it is fullscreen-only: - Side panel no longer mounts Graphic Walker at all. Instead it shows a compact summary (rows / columns / saved charts count) plus a single primary "Open in fullscreen" button. All styling goes through the design-token CSS variables so it adapts automatically to light/dark. - Use the existing global .btn / .btn-primary / .btn-secondary classes rather than bespoke inline styles, so the buttons match the rest of the app and flip cleanly with the theme. - Graphic Walker itself only mounts inside the Teleport overlay, which is the only place that needs the full viewport. The overlay declares both light- and dark-mode design tokens locally (the Teleport target is outside the .flowfile-editor-root scope, so the editor.css tokens would otherwise be unresolved), and reads the effective theme from theme-store to pick the right set. - closeFullscreen awaits saveCurrentSpec before unmounting so the chart survives the toggle, and Escape still exits fullscreen. - Removed the Save chart button; auto-flushing on unmount, close, and before-flow-export covers every save path.
FlowNode.vue had its own iconMap separate from the sidebar's nodeCategories. The sidebar was updated to explore_data.png (chart icon) in 23388e5, but FlowNode.vue:99 still mapped explore_data → view.png (binoculars). This meant the sidebar showed the correct icon while the canvas node kept showing the old binoculars.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a new
explore_datanode type that integrates Graphic Walker, an interactive visualization and analytics tool, enabling users to explore and visualize data directly within the flow without leaving the application.Key Changes
New Components
Backend Integration (Pyodide)
Type System
GraphicWalkerInput,GWDataModel,GWMutField, andRowInfotypes to match flowfile_core schemasNodeResultto includegraphic_walker_inputandrow_infofieldsExploreDataSettingsinterface for persisting saved chart specificationsStore Updates
execute_explore_data()instead ofexecute_preview()for explore_data nodesBundle Optimization
UI/UX Features
Notable Implementation Details
https://claude.ai/code/session_01HCz9DF1sNDyrKbb8Zh1Wdt