We investigate the possibility to develop methodologies for assessing effect specific structural changes of the breast tissue using a general statistical machine learning framework. We present an approach of obtaining objective mammographic pattern measures quantifying a specific biological effect, such as hormone replacement therapy (HRT). We compare results using this approach to using standard density measures. We show that the proposed method can quantify both age related effects and effects caused by HRT. Age effects are significantly detected by our method where standard methodologies fail. The separation of HRT subpopulations using our approach is comparable to the best methodology, which is interactive.