• Article

Using a nonlinear mixed effects model to characterize cholinesterase activity in rats exposed to Aldicarb

A statistical mixed effects model was applied to data from a two-phased experiment in which cholinesterase activity (CA) in red blood cells (RBCs) of rats was monitored following single- and double-gavage administrations of the carbamate pesticide, Aldicarb. Our goals were to develop and estimate a nonlinear mixed effects model for describing the inhibition and recovery pattern of CA as a continuous function of time and Aldicarb dose, and to use the results to characterize both the typical-animal CA response and its variability across animals. The Phase I experiment involved adult male CD(R) rats randomly assigned to a control and two dose groups. Blood samples were taken via jugular cannula prior to exposure and at 11 post-dose time points (over six hours) for determination of RBC CA. The Phase II experiment was similar, but involved a second administration approximately 4.5 hours after the first. An open, one-compartment model with first-order kinetics appeared to provide an adequate structural basis for the statistical nonlinear mixed effects model, which was fit to the RBC data of the 26 treatment-group rats from both phases. For the 'typical' animal, the model predicts that about a 20% (40%) maximal reduction in activity will occur for the low (high) dose group and that recovery to 90% of the baseline level will occur in 105 minutes (179 minutes) for the low (high) dose group. Based on simulated distributions derived from the model's variance-covariance matrix of random effects, we estimated that about 99% of animals are expected to require less than 143 minutes (267 minutes) to recover to 90% of their baseline level for the low (high) dose group. Because virtually all animals had returned to near pre-exposure levels by the time of the second administration, the level of the first dosing had little impact on the CA patterns following the second dose


Clayton, C., Starr, T. B., Sielken, R. L., Williams, R., Pontal, P. G., & Tobia, A. J. (2003). Using a nonlinear mixed effects model to characterize cholinesterase activity in rats exposed to Aldicarb. Journal of Agricultural, Biological, and Environmental Statistics, 8(4), 420-437. DOI: 10.1198/1085711032651

DOI Links