Conversation
✅ Deploy Preview for cockroachdb-interactivetutorials-docs canceled.
|
Files changed:
|
✅ Deploy Preview for cockroachdb-api-docs canceled.
|
❌ Deploy Preview for cockroachdb-docs failed. Why did it fail? →
|
… molt-redux-phase-2
…draft diagram, improved readability
Migration doc rework phase 2
Jeremyyang920
requested review from
crash-overdrive,
noelcrl,
ryanluu12345 and
tuansydau
ryanluu12345
left a comment
ryanluu12345
left a comment
There was a problem hiding this comment.
Very good refactor. I think it's clear to me when and why I should use approaches. I also like that the information deep dives where it needs to and provides links where it's part of a larger flow.
I'm going to defer to the rest of the team's review and @Jeremyyang920's stamp here
| @@ -0,0 +1,335 @@ | |||
| A [*Phased Bulk Load Migration*]({% link molt/migration-approach-phased-bulk-load.md %}) involves [migrating data to CockroachDB]({% link molt/migration-overview.md %}) in several phases. Data can be sliced per tenant, per service, per region, or per table to suit the needs of the migration. In this approach, you stop application traffic to the source database _only_ for the tables in a particular slice of data. You then migrate that phase of data to the target cluster using [MOLT Fetch]({% link molt/molt-fetch.md %}) during a **downtime window**. Application traffic is then cut over to those target tables after schema finalization and data verification. This process is repeated for each phase of data. | |||
There was a problem hiding this comment.
I recognize this is an entirely different pattern than was covered before, but has a lot of shared steps with just a simple monolithic migration for bulk load only. This is more of a nit, but any way we can reuse content from the above? I see we do references already, but wondering if there is more we can do to consolidate and not have to maintain as much content.
There was a problem hiding this comment.
Let me pass this question along to the doc reviewer, as this is something that we've been talking about in recent docs team conversations.
There was a problem hiding this comment.
Sounds good! Not a fix you need to do now, but just to make it easier for you to maintain in the future.
There was a problem hiding this comment.
Overall, looks solid! Wish I could've given this a more comprehensive review but it's hard to do that without turning this into a full review of the whole Migrations docs, since it's hard to see the delta here.
I like the addition of examples, diagrams, and concrete examples for users to grasp and use to help visualize their migration strategy, as well as listing and giving walkthroughs of certain migration strategies! Good stuff - mainly just small comments about wordings and a question to my TL and manager. Also seems like two flags may have been dropped, and the flags are necessary to forward replication to pg
|
FYI this linked section doesn't resolve: https://deploy-preview-21245--cockroachdb-docs.netlify.app/docs/molt/migration-considerations-phases |
|
|
||
| If performing a [phased migration]({% link molt/migration-considerations-granularity.md %}), you can use MOLT Verify's `--schema-filter` and `--table-filter` flags to specify specific schemas or tables to run the validations on. | ||
|
|
||
| If using [continuous replication]({% link molt/migration-considerations-replication.md %}), you can use MOLT Verify's `--continuous` and `--live` flags to enable continuous verification. |
There was a problem hiding this comment.
Note that we no longer support --continuous or --live mode. I think the preferred way to do this type of "fast checking" would be to use the selective verification. Which is the doc @taroface has been working on. This is recommended to do a smaller data set of spot checking now.
There was a problem hiding this comment.
Okay nice, I'll remove this line for now, and once @taroface has merged his PR I'll fold it into this one and add the link to the "Verify a subset of data" heading.
3 participants
This is a chunky PR, so here's a high-level description of the changes that have been made, so you can stay oriented.
Note that the line count that github provides for this PR (10000+ lines added) is super misleading due to how github counts .svg files (which are basically just images but which for some reason github interprets as many hundreds of new lines). The real number is far less than the one provided.
Phase 1: New guidance documentation
This is all of the content under the Migrate > Migration Considerations section of the nav. That includes:
I also moved Migration Strategies to Migrate > Migration Best Practices. It's mostly the same, but I moved some of its contents into the other guidance docs.
These pages will need the most significant tech review (though probably not from eng, probably from people in the field?) as I want to make sure that all of this strategic guidance is accurate.
Phase 2: Re-organizing the MOLT tooling docs
This is all of the content under Migrate > MOLT Tools, primarily under Fetch and Replicator. I broke up the contents by task, concept, and reference documentation. I also merged from main to include all of the Replicator userscript documentation. I did a lot of refining of the main Fetch and Replicator guides, including a new graphic in the Fetch guide. All of it is more readable and navigable now.
Phase 3: Source-db specific, step-by-step walkthroughs broken up by strategy
This is the content under Migrate > Common Migration Approaches. Mainly:
Note that each of these pages is an overview, that links to three source-DB-specific walkthroughs.
These pages re-work the content that was originally under "Migration Flows", but is more rigorously task-based, is broken up by different source DB types, tries to reduce info duplication, includes some graphics, and offers greater specificity about things like application downtime.
Other changes
A lot of files required changes to links, due to the other changes here. Various includes were removed/renamed. Therefore, many random files have minor, invisible changes (like links).