Skip to content

Adapting Wrench for use in an ML context, i.e. "frozen" training set parameters and "add-on" test set normalization #4

@hermidalc

Description

@hermidalc

Dear @hcorrada and @smuthiah - thanks for this incredible piece of work. I'm working on a project that is well suited for Wrench, i.e. very low depth, sparse, microbiome sequencing. @hcorrada I was in one of your graduate school classes at UMD!

I am trying to modify and adapt the Wrench code to work in an ML context. Since Wrench normalizes using information across samples I should not normalize a training and test set together, so I want to adapt it to fit and transform a training count matrix (with associated design matrix), save the "frozen" set of parameters, and then use the frozen training parameters to transform a test count matrix (with its design matrix) in an add-on fashion.

You already save many of the fitting parameters in W$others, I'm trying to figure out which ones are relevant for frozen training parameters that can then be applied directly to transform a test set?

When working backwards from how the compositional factors ccf is calculated using the default estimator type w.marg.mean, ccf is calculated using .getMargWeights and .getWeightedMean, and these two functions directly use s2thetag, radj, and pi0. radj is calculated using r, s2 and pi0. r is calculated using many other parameters.

pi0 appears should not be part of frozen set; calc only from test
s2 ? feature-wise vector could be part of frozen set
qref seems similar to TMM reference sample so yes should be part of frozen parameters
s2thetag ? group-wise could be part of frozen set
r appears should not be part of frozen set; but should be calc from test using frozen set
other params ?

Thanks in advance for any help you could give

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions