Using a nonlinear mixed effects model to characterize cholinesterase activity in rats exposed to Aldicarb

Abstract

A statistical mixed effects model was applied to data from a two-phased experiment in which cholinesterase activity (CA) in red blood cells (RBCs) of rats was monitored following single- and double-gavage administrations of the carbamate pesticide, Aldicarb. Our goals were to develop and estimate a nonlinear mixed effects model for describing the inhibition and recovery pattern of CA as a continuous function of time and Aldicarb dose, and to use the results to characterize both the typical-animal CA response and its variability across animals. The Phase I experiment involved adult male CD(R) rats randomly assigned to a control and two dose groups. Blood samples were taken via jugular cannula prior to exposure and at 11 post-dose time points (over six hours) for determination of RBC CA. The Phase II experiment was similar, but involved a second administration approximately 4.5 hours after the first. An open, one-compartment model with first-order kinetics appeared to provide an adequate structural basis for the statistical nonlinear mixed effects model, which was fit to the RBC data of the 26 treatment-group rats from both phases. For the 'typical' animal, the model predicts that about a 20% (40%) maximal reduction in activity will occur for the low (high) dose group and that recovery to 90% of the baseline level will occur in 105 minutes (179 minutes) for the low (high) dose group. Based on simulated distributions derived from the model's variance-covariance matrix of random effects, we estimated that about 99% of animals are expected to require less than 143 minutes (267 minutes) to recover to 90% of their baseline level for the low (high) dose group. Because virtually all animals had returned to near pre-exposure levels by the time of the second administration, the level of the first dosing had little impact on the CA patterns following the second dose