Likelihood ratio tests for a dose‐response effect using multiple nonlinear regression models

Summary We consider the problem of testing for a dose‐related effect based on a candidate set of (typically nonlinear) dose‐response models using likelihood‐ratio tests. For the considered models this reduces to assessing whether the slope parameter in these nonlinear regression models is zero or not. A technical problem is that the null distribution (when the slope is zero) depends on non‐identifiable parameters, so that standard asymptotic results on the distribution of the likelihood‐ratio test no longer apply. Asymptotic solutions for this problem have been extensively discussed in the literature. The resulting approximations however are not of simple form and require simulation to calculate the asymptotic distribution. In addition, their appropriateness might be doubtful for the case of a small sample size. Direct simulation to approximate the null distribution is numerically unstable due to the non identifiability of some parameters. In this article, we derive a numerical algorithm to approximate the exact distribution of the likelihood‐ratio test under multiple models for normally distributed data. The algorithm uses methods from differential geometry and can be used to evaluate the distribution under the null hypothesis, but also allows for power and sample size calculations. We compare the proposed testing approach to the MCP‐Mod methodology and alternative methods for testing for a dose‐related trend in a dose‐finding example data set and simulations.