Skip to content

WIP On SortVerbose#805

Open
jodavies wants to merge 2 commits intoform-dev:masterfrom
jodavies:sortverbose
Open

WIP On SortVerbose#805
jodavies wants to merge 2 commits intoform-dev:masterfrom
jodavies:sortverbose

Conversation

@jodavies
Copy link
Collaborator

@jodavies jodavies commented Mar 3, 2026

The first commit just cleans up WriteStats by using snprintf to sort out the field alignment, rather than the deeply nested if statements and MesPrint.

The second adds an On SortVerbose; mode for the sort statistics printing, which looks like:

Time =       0.06 sec    Generated terms =   33554432
           test1         Terms in output =          0
                         Bytes used      =          4
                         Comparisons     =  195165224
                         Small Buffer    =    0,   29
                         Large Buffer    =    0,    0

The numbers are, across all threads,

  • the total number of comparisons made
  • the number of times the small buffer was sorted due to TermsInSmall
  • the number of times the small buffer was sorted due to SmallSize
  • the number of times the large buffer was sorted due to LargePatches
  • the number of times the large buffer was sorted due to LargeSize

The motivation is to make it easier for the user to configure sorting buffer sizes, since they are able to see what is the bottleneck. For example, increasing SmallSize is not useful if TermsInSmall was the reason for your small buffer sorts, etc. Printing the comparison count is more useful from a development perspective, if one is working on the sorting systems.

I don't see any performance degradation due to collecting the extra information.

Any comments? Any extra useful information we could add here?

The deeply-nested conditions on the length of the sizes is due to
MesPrint truncating numbers which are too long for positioned
fields, rather than loosening the positioning. snprintf doesn't
truncate, so is much easier to align.
Add information to the final sort summaries (per thread and master)
including the number of comparisons made, and the number of times
the small and large buffers were sorted due to their capacities.
@coveralls
Copy link

Coverage Status

coverage: 58.093% (+0.06%) from 58.031%
when pulling 69dad23 on jodavies:sortverbose
into e33ef78 on form-dev:master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants