PCA analysis is a common method of looking at embeddings spaces. It's a potentially relevant method here for our visualization.
For this task, we would like to take data from existing benchmarks (such as BACH), and embedd it, using a current checkpoint from the Midnight replication (this will be provided). These embeddings are also available after the evaluation process, natively.
We will try at least two different methods:
PCA analysis is a common method of looking at embeddings spaces. It's a potentially relevant method here for our visualization.
For this task, we would like to take data from existing benchmarks (such as BACH), and embedd it, using a current checkpoint from the Midnight replication (this will be provided). These embeddings are also available after the evaluation process, natively.
We will try at least two different methods: