vcf_zarr input

[hyanwong]

We are moving tsinfer to expect input files in VCF Zarr format (see https://www.biorxiv.org/content/10.1101/2024.06.11.598241v1). This is much more compact, well defined and faster to parse etc. than plain VCF files. We have a fast converter to do this at https://sgkit-dev.github.io/bio2zarr/intro.html. I wondered if this would also make a useful input format for Phlash?

[terhorst]

I like this idea! Seeing as it's apparently easy to go from VCF to VCF Zarr, I will probably make this the default/preferred input format.

[hyanwong]

Great! Part of the impetus for creating VCF Zarr was to avoid the need to depend on yet another VCF parser (i.e. we can then push off the horrible task of parsing VCFs to someone else's utility!)

We're mainly using it on large-scale phased data (e.g. UK biobank WGS), but it should be fine for unphased data too.

[hyanwong]

The default input to the tsinfer ARG inference tool is now VCF-Zarr, and there has been a new release of each (0.1.5 for https://github.com/sgkit-dev/bio2zarr, 0.4 for tsinfer). There is also now a way to efficiently convert tree sequences directly to VCF-Zarr, although it's not be formally released yet, and the python API is not yet documented. It should go something like this however:

# install bio2zarr from github
import bio2zarr 

bio2zarr.tskit.convert(
    ts_path
    zarr_path
    ...
)

I'll ping you here when this is fully released, but I guess this would mean that you could also get rid of all that tree sequence input code from this repo, but still maintain direct input from stdpopsim?