Rework daily cron jobs to hit the API instead of working directly on the Registry #715
+1,462
−508
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
f-f
commented
Jan 11, 2026
f-f
commented
Jan 11, 2026
f-f
commented
Jan 11, 2026
thomashoneyman
requested changes
Jan 13, 2026
Member
There was a problem hiding this comment.
Noticed a few things that need to be resolved before we merge. Also, I went ahead and removed the daily jobs workflow from the registry repository since this replaces it:
purescript/registry#525
f-f
changed the title
f-f
commented
Jan 17, 2026
f-f
commented
Jan 17, 2026
thomashoneyman
force-pushed
the
f-f/cronjobs-in-job-queue
branch
from
92b322a to
4a7ce6d
Compare
thomashoneyman
approved these changes
Jan 18, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR integrates into the Job Queue framework the three "Registry scripts" that we run daily to keep data tidy: the package transferrer, the package set upgrades, and the legacy importer - as detailed hereThe patch is simpler than I though it would be, and I think that's mainly because we only deal with packages that are already in the registry, rather than packages that could possibly be missing. As a result, the package transfer script and the package set updater are deprecated, while the legacy importer stays in place since it's the only way we have to bootstrap the registry from scratch at the moment.EDIT: the above approach didn't work because of the complex race conditions that come from running more code interacting with the registry concurrently with the jobs (that's why we have the queue, right!). So we are going to keep the daily cron jobs in separate scripts, but wire them up so that they hit the API instead of directly editing the Registry.
Somewhere in the middle of this PR I noticed that the matrix tests for detecting a compiler at the start were discarded, so I added them back.