perf(studio): eliminate N duplicate getPkgVersionMeta calls on load#197
Open
jameswillis99 wants to merge 4 commits intomasterfrom
Open
perf(studio): eliminate N duplicate getPkgVersionMeta calls on load#197jameswillis99 wants to merge 4 commits intomasterfrom
jameswillis99 wants to merge 4 commits intomasterfrom
Conversation
On every studio open, the client was firing N parallel requests to
`GET /api/v1/pkgs/:pkgId/versions-without-data` (one per direct
dependency) to populate "update available" badges in the sidebar. Each
request acquired its own DB connection, ran a full permission chain, and
saturated the pool alongside all the other concurrent startup requests.
This commit eliminates those calls on initial load by including the
latest published version for every direct dependency in the
`getSiteInfo` response, and parallelises the remaining sequential
queries in `getProjectRev`.
Changes by area:
server/routes/projects.ts + AppServer.ts
- Remove `withNext` (outer transaction) from `GET /api/v1/projects/:id`
so queries use pool connections and can run concurrently.
- Restructure `getProjectRev` into three parallel rounds:
Round 1 – 9 independent queries in Promise.all (branch, project, rev,
perms, modelVersion, hostlessVersion, latestRevisionSynced,
appAuthConfig, allowedDataSources).
Round 2 – migratedBundle, owner fetch, workspaceTutorialDbs in
parallel (depend on project/rev from Round 1).
Round 3 – loadDepPackages → getLatestPkgVersionsForPkgIds (depend on
migratedBundle).
- `claimPublicProject` (write path) gets its own explicit
startTransaction since the outer withNext is removed.
- Add `latestDepPkgVersions` to the response payload.
server/db/DbMgr.ts
- Add `getLatestPkgVersionsForPkgIds`: single WHERE IN query fetching
the latest main-branch PkgVersion per pkgId, excluding the heavy
`model` column. No permission check — called inside getProjectRev
which has already verified access, same trust level as
loadDepPackages.
- Refactor `checkPkgPerms` to return the Pkg entity it already fetches,
removing a redundant `getPkgById` call in `publishPkgVersion`.
- Remove redundant `checkPkgPerms` call from `listPkgVersionsRaw` (the
endpoint-level auth check in `listPkgVersions` is sufficient).
server/db/bundle-migration-utils.ts
- Replace the 10-minute TTL cache invalidation for hostless data with an
explicit flag (`_hostlessCacheStale`) that is set on startup and
cleared only after a successful reload. `invalidateHostlessCache()` is
called by `bumpHostlessVersion` so the cache is always fresh after a
publish, without polling the DB on every request.
- Remove `hostlessVersionCount` from the cached payload (no longer
needed since the cache is invalidated explicitly rather than by
comparing version counts).
server/db/BundleMigrator.ts
- Hoist the `bundleHasStaleHostlessDeps` call before the
`isExpectedBundleVersion` check so both conditions are evaluated
before branching, avoiding a potential double-await.
shared/ApiSchema.ts
- Add `latestDepPkgVersions: Record<string, PkgVersionInfoMeta>` to
`GetProjectResponse`.
client/db.ts
- Add `latestDepPkgVersions` to `DbCtxArgs` / `DbCtx` so the
server-provided map flows through to StudioCtx.
client/init-ctx.tsx
- Destructure `latestDepPkgVersions` from the getSiteInfo response and
pass it to DbCtx.
- Simplify `checkDepPkgHosts`: it already has the full PkgVersionInfo
from `depPkgs` so there's no need to fire N extra `getPkgVersionMeta`
calls — use the already-loaded data directly.
client/ProjectDependencyManager.ts
- Add `seedLatestVersionMeta`: pre-populates latestPkgVersionMeta in
_dependencyMap from the server-provided map. Since
`_fetchLatestVersionMeta` skips the API call when the field is already
set, this eliminates all N badge-fetch calls on first load.
- Add `setPlumePkgFetch`: allows the Plume package to be fetched in
parallel with project load by accepting a pre-started Promise.
- Split `_fetchData` into `_fetchLatestVersionMeta` (spawned, non-
blocking) and `_fetchPlumeSite` (awaited, required before sync), so
badge updates do not block studio startup.
- Change `refreshDeps()` to accept `{ forceVersionMeta?: boolean }`
(default false) so the initial load does not bypass pre-seeded data.
client/studio-ctx/StudioCtx.tsx
- Call `seedLatestVersionMeta` immediately after constructing
ProjectDependencyManager.
client/components/sidebar/ProjectDependencies.tsx
- Pass `{ forceVersionMeta: true }` to `refreshDeps` when the user
clicks "Check for updates", so manual refresh still hits the API.
client/components/studio/studio-initializer.tsx
- Fire `getPlumePkg()` before `initStudioCtx` and hand the in-flight
Promise to `setPlumePkgFetch`, so the Plume fetch runs in parallel
with the project load rather than serially after it.
client/api.ts
- Remove the now-unnecessary `listPkgVersionsWithoutData` override from
`filteredApi` (the function is no longer called by the client in a
context that requires it to be filtered).
…idation Revert the _hostlessCacheStale flag approach back to the original 10-minute TTL + hostlessVersionCount comparison. The cache change is out of scope for this PR and can be revisited separately.
Keep only the removal of N duplicate getPkgVersionMeta API calls in checkDepPkgHosts. The data (hostUrl, pkg name/id) is already present on the depPkgs returned in the getSiteInfo response, so there is no need to re-fetch it. All server-side changes and broader client optimisations are reverted for a separate PR.
jameswillis99
changed the title
The previous commit changed checkDepPkgHosts from async (ProjectDependency[])
to sync (PkgVersionInfo[]) but missed two other call sites that still passed
ProjectDependency[], causing TypeScript compilation failures.
- ProjectDependencyManager.ts: pass [latest, ...depPkgs] (PkgVersionInfo[])
from the getPkgVersion response; drop the unused depPkgVersions destructure
from unbundleProjectDependency
- StudioCtx.tsx: pass depPkgs (PkgVersionInfo[]) directly; simplify the
unbundleSite destructure to { site } since depPkgVersions is unused;
remove now-unused isKnownProjectDependency import
- All three call sites: remove spawn() wrapper since checkDepPkgHosts is now
synchronous (spawn expects PromiseLike, not void)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On every studio open,
checkDepPkgHostsfired N parallelgetPkgVersionMetaAPI calls (one per dependency) to retrievehostUrl,pkg.name, andpkg.projectIdfor the host-mismatch warning. This data is already present on thedepPkgsarray returned in the samegetSiteInforesponse that just completed — there is no need to re-fetch it.For the current EP storefront template this halves the number of DB-read-heavy dependency calls on studio load from 8 to 4.
Change
client/init-ctx.tsx— simplifycheckDepPkgHosts:ProjectDependency[]toPkgVersionInfo[]getPkgVersionMetacalls; iteratedepPkgsdirectly using thehostUrl,pkg.name, andpkg.projectIdfields already present on each entrydepPkgs(the API response objects) tocheckDepPkgHostsinstead of the unbundledProjectDependencymodel objectsProjectDependencyimport; addPkgVersionInfoasync)Verification
/getPkgVersionMetacalls on studio load🤖 Generated with Claude Code