Skip to content

feat(compiler): mirror memo output paths to Python source modules#6457

Open
FarhanAliRaza wants to merge 5 commits intoreflex-dev:mainfrom
FarhanAliRaza:memoize-file-path-mirror
Open

feat(compiler): mirror memo output paths to Python source modules#6457
FarhanAliRaza wants to merge 5 commits intoreflex-dev:mainfrom
FarhanAliRaza:memoize-file-path-mirror

Conversation

@FarhanAliRaza
Copy link
Copy Markdown
Contributor

@FarhanAliRaza FarhanAliRaza commented May 5, 2026

Memos now compile into a single JSX file per user module at a path that mirrors the module's dotted name, instead of one file per memo under . The page-side import surface matches the source
layout, which makes debugging easier and lets Vite group co-defined memos in the same chunk.

Memos without a captured source module keep the legacy per-name files and index. A manifest in records emitted
paths so stale files from previous compiles get pruned.

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

New Feature Submission:

  • Does your submission pass the tests?
  • Have you linted your code locally prior to submission?

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

closes #6218

Memos now compile into a single JSX file per user module at a path that
mirrors the module's dotted name, instead of one file per memo under
. The page-side import surface matches the source
layout, which makes debugging easier and lets Vite group co-defined
memos in the same chunk.

Memos without a captured source module keep the legacy per-name files
and  index. A manifest in  records emitted
paths so stale files from previous compiles get pruned.
@FarhanAliRaza FarhanAliRaza requested a review from a team as a code owner May 5, 2026 09:56
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 5, 2026

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 2 skipped benchmarks1


Comparing FarhanAliRaza:memoize-file-path-mirror (1a13ca0) with main (a84e29b)

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 5, 2026

Greptile Summary

This PR changes how memo-defined JSX files are written to .web: instead of one file per memo under utils/components/, memos that carry a captured Python source module are now compiled into a single combined JSX file whose path mirrors the module's dotted name. Legacy memos (no captured module) keep the old per-name layout and index re-export. A manifest-driven pruning step removes files from previous compiles that are no longer emitted.

  • New memo_paths utility (reflex_base.utils.memo_paths) captures fn.__module__, validates each segment for path safety, and translates module names to mirrored $/a/b/c import specifiers; a @functools.cache on the package-vs-module check can return stale results during hot reload if a module's type changes between saves.
  • Grouping in _compile_memo_components accumulates all memos sharing a source module into a _MemoGroup and renders them to one file, with framework imports merged in separately for both the output file and the aggregate page-level import set.
  • Manifest-based stale cleanup (prune_stale_memo_files) writes .memo-manifest.json atomically after each compile and deletes files from the previous run that are no longer emitted, also removing empty parent directories up to (but not including) .web.

Confidence Score: 3/5

Safe to merge for production builds, but hot-reload development workflows can silently produce wrong JSX paths when a module's package/regular-module status changes between saves.

The core compilation path is well-structured and the legacy fallback is preserved, but module_to_mirrored_segments permanently caches the find_spec-based package/module classification for the process lifetime. In Reflex's hot-reload mode — which reruns compilation inside the same process — converting myapp/widgets/__init__.py to myapp/widgets.py between saves will keep returning ("myapp", "widgets", "index") segments, so the JSX is written to the wrong path and page imports resolve to a nonexistent file until the dev server restarts. This is a silent mismatch, not a crash, which makes it harder to diagnose.

packages/reflex-base/src/reflex_base/utils/memo_paths.py — the @functools.cache on module_to_mirrored_segments is the key area to review. reflex/utils/memo_paths.py omits library_specifier_for from its re-export list.

Important Files Changed

Filename Overview
packages/reflex-base/src/reflex_base/utils/memo_paths.py New module implementing source-module capture and mirrored path translation; @functools.cache on module_to_mirrored_segments can return stale package/module classification during hot reload.
reflex/compiler/utils.py Adds manifest-based stale-file pruning with atomic rename; minor fd-leak corner case if os.fdopen raises after mkstemp.
reflex/compiler/compiler.py Refactors memo compilation to group by mirrored source-module segments; legacy path preserved for memos without a captured module.
reflex/utils/memo_paths.py Convenience re-export shim; omits library_specifier_for from its public surface while re-exporting every other helper.
reflex/experimental/memo.py Threads source_module through all definition factories and component-class builders; no logic issues found.
packages/reflex-base/src/reflex_base/components/component.py Adds _source_module field to CustomComponent and propagates it through custom_component decorator and _register_custom_component; import specifier wiring looks correct.
tests/units/compiler/test_stale_cleanup.py New test file with good coverage of the manifest-based pruning logic including missing manifest, corrupt manifest, and empty-dir cleanup cases.
tests/units/utils/test_memo_paths.py New tests covering segment safety, package detection, framework filtering, and frame-walking; reasonable coverage for the new utility module.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[memo decorator / custom_component] --> B{capture_source_module}
    B -->|user module| C[source_module set]
    B -->|None / framework| D[source_module = None]

    C --> E[module_to_mirrored_segments]
    E -->|package __init__.py| F["segments + 'index'"]
    E -->|regular module| G[segments as-is]
    F & G --> H[library_specifier_for\n'$/a/b/c']

    D --> I[legacy per-name path\n'$/utils/components/Name']

    H --> J[_compile_memo_components\ngroups by segments]
    I --> K[legacy_files + index]

    J --> L[single merged JSX\nper source module]
    K --> M[utils/components/Name.jsx\n+ index re-export]

    L & M --> N[prune_stale_memo_files\nreads .memo-manifest.json]
    N --> O[deletes stale JSX files\n& empty directories]
    N --> P[writes new manifest]
Loading

Reviews (1): Last reviewed commit: "feat(compiler): mirror memo output paths..." | Re-trigger Greptile

Comment thread packages/reflex-base/src/reflex_base/utils/memo_paths.py Outdated
Comment thread reflex/utils/memo_paths.py
Comment thread reflex/compiler/utils.py
Identical memoizable subtrees on pages from different user modules
produce the same wrapper tag. The auto-memo registry was keyed by tag
alone, so the second registration overwrote the first — only one of
the source modules got a mirrored memo file emitted, and the other
page imported the tag from a JSX file that never declared it. Vite
failed the prod build with MISSING_EXPORT.

Key the registry by (tag, source_module) so each module's mirrored
file gets its own definition, and add an integration test that builds
two pages from distinct user modules sharing a memoizable subtree.
 was d, so once a
module had been resolved its mirrored path was frozen for the
process. A user toggling a module between a regular  and a
package () during dev reload kept the original origin
and emitted memo files to the stale path.

Drop the cache and read  from  first, falling
back to  only when the module isn't loaded —
is rebound on reload, while a cached spec wouldn't be. Also tighten
 to close the mkstemp fd up front so it can't
leak if reopening raises, and re-export  from
 for parity with the rest of the surface.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compiler: Track provenance of rx.memo components and output to mirrored Python paths

1 participant