[CASSANDRA-21083][trunk] Optimize memtable flush logic

[netudima]

No description provided.

[netudima]

I've done a retesting for the current version

            if (cell.getClass() == NativeCell.class)
            {
                valueSize = cell.valueSize();
                hasValue = valueSize > 0;
                isDeleted = cell.isTombstone();
                isExpiring = cell.isExpiring();
                cellTimestamp = cell.timestamp();
                localDeletionTime = cell.localDeletionTime();
                ttl = cell.ttl();
                value = cell.value();
                accessor = cell.accessor();
            }
            else
            {
                valueSize = cell.valueSize();
                hasValue = valueSize > 0;
                isDeleted = cell.isTombstone();
                isExpiring = cell.isExpiring();
                cellTimestamp = cell.timestamp();
                localDeletionTime = cell.localDeletionTime();
                ttl = cell.ttl();
                value = cell.value();
                accessor = cell.accessor();
            }

is not equivalent to

            if (cell.getClass() == ArrayCell.class)
            {
                valueSize = cell.valueSize();
                hasValue = valueSize > 0;
                isDeleted = cell.isTombstone();
                isExpiring = cell.isExpiring();
                cellTimestamp = cell.timestamp();
                localDeletionTime = cell.localDeletionTime();
                ttl = cell.ttl();
                value = cell.value();
                accessor = cell.accessor();
            }
            else
            {
                valueSize = cell.valueSize();
                hasValue = valueSize > 0;
                isDeleted = cell.isTombstone();
                isExpiring = cell.isExpiring();
                cellTimestamp = cell.timestamp();
                localDeletionTime = cell.localDeletionTime();
                ttl = cell.ttl();
                value = cell.value();
                accessor = cell.accessor();
            }

while direct calls like cell.timestamp() as expected are inlined in both cases

there is a difference in inlining for 2nd level in cases like cell.localDeletionTime(); -> cell.localDeletionTimeAsUnsignedInt(), it looks like inlining does not work on the 2nd level for bimorphic calls:

I'll try and compare possible options, such as two if blocks if (cell.getClass() == ArrayCell.class) + if (cell.getClass() == NativeCell.class) and switch to it if it shows better results.

[netudima]

CPU flush time per partition (get_flush_cpu_time.sh), microseconds (per partition metric is used to normalize data, because heap and offheap flushing triggers on different thresholds, so SSTable size is is different):

Option	offheap_objects	heap_buffers
1if NativeCell	10.433	12.451, no inlining for transitive Cell calls
1if ArrayCell	11.112, no inlining for transitive Cell calls	12.044
2 ifs ArrayCell+NativeCell	10.369	12.277
2 ifs NativeCell+ArrayCell	10.147	12.394

note: for the option 2 ifs NativeCell+ArrayCell, I've enforced inlining in MinMax(Int/Long)Tracker

so, it looks like 2 ifs NativeCell+ArrayCell option is the safest one (while in most cases the results are actually close/within a error margin, except some combinations)