CNDB-8756: Keep all the SAI components of a sstable in the same version

[adelapena]

What is the issue

Fixes https://github.com/riptano/cndb/issues/8756:

SAI indexes are mostly independent, so you can legitimately have index X with version A and index Y with version B for the same sstable.

However, all indexes share a single PrimaryKeyMap, so an index that assumes the PKM is row-based will not be compatible with one that uses a partition-based PMK.

Here is an example of the issues we can have if a sstable ends up with two indexes with incompatible formats:

createTable("CREATE TABLE %s (k int, c int, a int, b int, PRIMARY KEY (k, c))");
execute("INSERT INTO %s (k, c, a, b) VALUES (?, ?, ?, ?)", 0, 0, 0, 0);
flush();

SAIUtil.setCurrentVersion(Version.EARLIEST);
createIndex("CREATE CUSTOM INDEX ON %s(a) USING 'StorageAttachedIndex'");

SAIUtil.setCurrentVersion(Version.LATEST);
createIndex("CREATE CUSTOM INDEX ON %s(b) USING 'StorageAttachedIndex'");

execute("SELECT * FROM %s WHERE b = 0"); // java.lang.AssertionError: PrimaryKey TokenAwarePrimaryKey

Querying the second index fails because its per-index components use Version.LATEST, which is row-aware, while its per-sstable components use Version.EARLIEST, which is partition-aware.

What does this PR fix and why was it fixed

New per-index components for a sstable will use the version of any existing per-sstable components in that same sstable. If there aren't any previous per-sstable components then the configured current version will be used. A sstable can have per-sstable components if they have been added by another index. New sstables will always use the current version.

This means that all the index components of a sstable will always use the same index version, even though different sstables can have different versions.

Rebuilding indexes after changing the current version won't change the version of the existing sstable indexes, although new sstables will use the new current version. However, sstableupgrade will rewrite the tables with the new version. Thus, the procedure to upgrade indexes is first setting the current version and then running sstableupgrade.

Creating a new index will fail if there is any sstable in the indexed table using a version that doesn't support the type of index we are creating. For example, creating a vector index will fail if there is still some sstable using the aa format, even in the current version supports indexes. If an index creation fails for this reason, the index will be added to the schema but marked as failed. Running sstableupgrade will upgrade the index version, but it won't mark the index as queryable, so it will be needed to also rebuild the index so it becomes queryable.

[github-actions]

Checklist before you submit for review

• This PR adheres to the Definition of Done
• Make sure there is a PR in the CNDB project updating the Converged Cassandra version
• Use NoSpamLogger for log lines that may appear frequently in the logs
• Verify test results on Butler
• Test coverage for new/modified code is > 80%
• Proper code formatting
• Proper title for each commit staring with the project-issue number, like CNDB-1234
• Each commit has a meaningful description
• Each commit is not very long and contains related changes
• Renames, moves and reformatting are in distinct commits
• All new files should contain the DataStax copyright header instead of the Apache License one

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
90.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
90.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[cassci-bot]

❌ Build ds-cassandra-pr-gate/PR-2243 rejected by Butler

---

3 regressions found
See build details here

---

Found 3 new test failures

Test	Explanation	Runs	Upstream
o.a.c.distributed.test.AbortedQueryLoggerTest.testLogsReadMetrics	REGRESSION	🔴🔴	0 / 30
o.a.c.index.sai.cql.VectorCompaction100dTest.testOneToManyCompaction[version=ec enableNVQ=true]	REGRESSION	🔴⚪	0 / 30
o.a.c.index.sai.cql.VectorSiftSmallTest.testRerankKZeroOrderMatchesFullPrecisionSimilarity[version=ed enableNVQ=true enableFused=false]	REGRESSION	🔴⚪	0 / 30

---

Found 7 known test failures