what are the approaches to accelerate the extraction process for big designs?

[pei108]

Magic is kind of slow compared to industrial tool Calibre which can run in a multi-threading mode. Any thoughts on accelerating Magic on big designs?

[RTimothyEdwards]

In the works, and planned to be worked on over the next three months (sooner, depending on when I get around to working on it). There are a couple of specific points in the database representation that are preventing parallelization of processes over different planes. Once those are fixed, it should be possible to use openmp to parallelize essentially all of the database search routines.

[dlmiles]

As this subject came up in this timeframe.

The following areas also require rework (an alternative approach to the one that currently exists in the location marked):
#436 (comment)

The f2c824c comment collected the other parts.

One of the major goals for me is to get a re-creatable output (given same input model, extraction input parameters, the output text is always identical, or at least you can ask for the output to be sorted), so there would be a collation thread/task that manages which order the pieces end up in the final output. This then makes the output usable with diff and not dependant on memory addresss / hash ordering. This helps with unit testing without needing to process this matter externally to magic.

FWIW not evaluated how openmp might be used, as not clear if it is heavily matrix computation bound or memory branch bound. Unclear what the working data set (as in cache-lines read, cache-lines written) is for per-device. Maybe openmp is only good on the scale of tight loop with clear computation and data available, but what I see is a lot of searching (branching) for the data to make it available so maybe regular pthreads of the existing mechanism. Anyhow easy enough to test minimum time to finish when they are implemented.
I liken this to openmp is most useful to go fast on the straight and put in the fastest time for that sector, but this task is more about going around the race circuit 70 times in the fastest possible time (and the corners are what makes the overall time slow, there is more to gain going fast there). But let the data show what works best.