This is a walk through one relatively simple simulation written to check whether a
generalized linear model on a contingency table of counts (poisson distribution) would provide the same results as a
generalized linear model with one line per observation and the occurence of the variable of interest coded as Yes/No (binomial distribution).
I created this code while preparing my preregistration for a simple behavioral ecology experiment:
Methods for independently manipulating palatability and color in small insect prey (article, OSF preregistration)
The R screen screenshoted below can be found in the folder Ihle2020.
This walk through will use the steps as defined in the page 'general structure'
- define sample sizes (within a dataset, and number of replicates), experimental design (fixed dataset structure, e.g. treatment groups, factors) and parameters that will need to vary (here, the strength of the effect)
- generate data (here, using
sample()and the probabilities defined in step 1 and format it in two different ways to accomodate the two statistical tests to be compared.
- run the statistical test and save the parameter estimate of interest for that iteration. Here, this is done for both statistical tests to be compared.
- replicate step 2 (data simulation) and 3 (data analyses) to get the distribution of the parameter estimates by wrapping these steps in a function
definition of the function at the beginning:
output returned from the function at the end:
replicate the function nrep number of times. Here pbreplicate is used to provide a bar of progress for R to run this command.
- explore the parameter space. Here, vary the probabilities of sampling 0 or 1 depending on the treatment group category.
- analyze and interpret the combine results of many simulations. In this case, the results of the two models were qualitatively the same (comparison of results for a few simulations), and both models gave the same expected 5% false positive results when no effect were simulated. Varying the effect (the probability of sampling 0 or 1 depending on the experimental treatment) allowed to find the minimum effect size for which the number of positive results is over 80% of the tests.
