Prediction of the outcomes of natural enemy introductions remains the most fundamental challenge in biological control. Quantitative retrospective analyses of ongoing biocontrol projects provide a systematic strategy to evaluate and further develop ecological risk assessment. In this review, we highlight a crucial assumption underlying a continued reliance on the host specificity paradigm as a quantitative prediction of ecological risk, summarize the status of our retrospective analyses of nontarget effects of two weevils used against exotic thistles in North America, and discuss our prospective assessment of risk to a federally listed, threatened species (Cirsium pitcheri) based on those studies. Our analyses quantify the fact that host range and preference from host specificity tests are not sufficient to predict ecological impact if the introduced natural enemy is not strictly monophagous. The implicit assumption when such use is made of the host specificity data in risk assessment is that population impacts are proportional to relative preference and performance, the key components of host specificity. However, in concert with shifting awareness in the field, our studies demonstrate that the environment influences and can alter host use and population growth, leading to higher than expected direct impacts on the less preferred native host species at several spatial scales. Further, we have found that straightforward, easily anticipated indirect effects, on intraguild foragers as well as on the less preferred native host plant species, can be both widespread and significant. We conclude that intensive retrospective ecological studies provide some guidance for the quantitative prospective studies needed to assess candidate biological control agent dynamics and impacts and, so, contribute to improved rigor in the evaluation of total ecological risk to native species.