-
Jurjen Broeke authored
Requested addition: to trust conclusions from the mixed-effects models, need ICC and heterogeneity diagnostics alongside the fixed-effect estimates, not just "did the model fit without erroring". - model_diagnostics(): for lmer models, ICC (icc_adjusted/icc_conditional -- how much variance is between-animal, the entire justification for the random-effect structure), marginal/conditional R2, a likelihood-ratio test of whether the random intercept is statistically supported (lmerTest::ranova()), a singular-fit flag, and a heteroscedasticity check (Breusch-Pagan via performance::check_heteroscedasticity()). For the lm fallback case, only plain R2 and heteroscedasticity apply; the random-effect-specific columns are NA. - Refactored stats_funs.R so each model is fit once and both the tidy fixed-effect table and the diagnostics table are derived from that same fit (fit_models_for_responses() + tidy_models()/diagnose_models()), rather than fitting twice. - 04.Statistics.R now writes a diagnostics_*.csv alongside each stats_*.csv. Caught two real bugs while validating against actual data (both fixed, not routed around): - fit_models_for_responses() used purrr's `~ fn(df, .x, ...)` tilde shorthand to forward extra args. With nothing to forward, that bare `...` turned out to leak the response's own value into the next positional argument (covariates) -- every model silently became `response ~ response` instead of `response ~ <real covariates>`. No error, just an R2 of ~0 and a model.matrix warning easy to miss. Caught by cross- checking a diagnostic against an independently-verified value for the same model. Fixed by using an explicit closure instead of the tilde shorthand, which forwards ... correctly. - performance::icc() returns a plain NA (not the documented list/data.frame) when a model has a singular fit -- observed for real with the 12-animal synthetic repeated-measures dataset. `$ICC_adjusted` on that NA is a hard error ("$ operator is invalid for atomic vectors"), not a graceful missing value. Added a safe_field() accessor used for every performance- package field extraction (icc, r2) so a singular fit degrades to NA diagnostics instead of crashing the whole batch. Validated against both real data (9 animals, 1 session each -- lm fallback path, diagnostics correctly NA for the random-effect-specific columns) and the synthetic 12-animal repeated-measures dataset (lmer path: differentiated real results per response, including one genuine singular fit handled gracefully and two responses with significant heteroscedasticity flagged). Co-Authored-By:Claude Sonnet 5 <noreply@anthropic.com>
42c7e127Jurjen Broeke authoredRequested addition: to trust conclusions from the mixed-effects models, need ICC and heterogeneity diagnostics alongside the fixed-effect estimates, not just "did the model fit without erroring". - model_diagnostics(): for lmer models, ICC (icc_adjusted/icc_conditional -- how much variance is between-animal, the entire justification for the random-effect structure), marginal/conditional R2, a likelihood-ratio test of whether the random intercept is statistically supported (lmerTest::ranova()), a singular-fit flag, and a heteroscedasticity check (Breusch-Pagan via performance::check_heteroscedasticity()). For the lm fallback case, only plain R2 and heteroscedasticity apply; the random-effect-specific columns are NA. - Refactored stats_funs.R so each model is fit once and both the tidy fixed-effect table and the diagnostics table are derived from that same fit (fit_models_for_responses() + tidy_models()/diagnose_models()), rather than fitting twice. - 04.Statistics.R now writes a diagnostics_*.csv alongside each stats_*.csv. Caught two real bugs while validating against actual data (both fixed, not routed around): - fit_models_for_responses() used purrr's `~ fn(df, .x, ...)` tilde shorthand to forward extra args. With nothing to forward, that bare `...` turned out to leak the response's own value into the next positional argument (covariates) -- every model silently became `response ~ response` instead of `response ~ <real covariates>`. No error, just an R2 of ~0 and a model.matrix warning easy to miss. Caught by cross- checking a diagnostic against an independently-verified value for the same model. Fixed by using an explicit closure instead of the tilde shorthand, which forwards ... correctly. - performance::icc() returns a plain NA (not the documented list/data.frame) when a model has a singular fit -- observed for real with the 12-animal synthetic repeated-measures dataset. `$ICC_adjusted` on that NA is a hard error ("$ operator is invalid for atomic vectors"), not a graceful missing value. Added a safe_field() accessor used for every performance- package field extraction (icc, r2) so a singular fit degrades to NA diagnostics instead of crashing the whole batch. Validated against both real data (9 animals, 1 session each -- lm fallback path, diagnostics correctly NA for the random-effect-specific columns) and the synthetic 12-animal repeated-measures dataset (lmer path: differentiated real results per response, including one genuine singular fit handled gracefully and two responses with significant heteroscedasticity flagged). Co-Authored-By:Claude Sonnet 5 <noreply@anthropic.com>
Loading