Computer Vision Detects Subtle Histological Effects of Dutasteride on Benign Prostate Tissue

Objective Dutasteride is commonly prescribed for benign prostatic hyperplasia, to shrink the prostate by blocking formation of the potent androgen, dihydrotestosterone (DHT). Despite the drug's dramatic effect on prostate size and growth, pathologists have struggled to identify its characteristic changes at the histological level. We studied H&E prostate biopsy images from participants in a randomized trial to determine if a computer vision approach could discriminate long‐term dutasteride vs. placebo subjects and provide insight into the discriminating features. Methods We used H&E slides obtained at a mandatory Year 4 biopsy, from 100 men assigned to either dutasteride or placebo (n=50 each) in the REDUCE trial. All of the men were compliant and none had prostate cancer detected during the trial. Year 2 biopsy H&E slides (n=25 per group) were set aside for comparison and validation. Whole slide images were taken at 40× and Definiens Developer XD® software was used to segment each image at two hierarchical but related levels: superpixels and nuclei. Superpixel features were used to train an L1‐regularized logistic regression model that distinguished epithelium from stroma. We then generated a library of 1,300 epithelial and stromal features from objects comprising superpixels and several types of nuclei, including spatial relations among objects between and within each hierarchical level. Model development involved feature normalization and exclusion of low variance features, followed by two rounds of variable selection based on 5‐fold cross‐validation (xCV), the first using an elastic net penalty and the second using an L1 penalty on 204 features selected during Round 1. Feature data from the Year 2 biopsies was fitted to a final model to assess semi‐independent external validation. Two pathologists, blinded to treatment, viewed each whole slide image and predicted randomized assignment by consensus. Results Pathologist classification had an accuracy equivalent to chance; treatment groups did not differ when scored for focal atrophy type or for drug‐related features postulated in the literature. A 22‐feature model gave an xCV AUC of 86.6% (95% CI: 0.80–0.94) for discrimination of drug from placebo. This model gave an AUC = 74.6% (95% CI: 0.64–0.92) in the set‐aside Year 2 biopsies. Histology scores did not change from Year 2 to Year 4 within subjects in either treatment group. Model scores were not correlated with change in PSA or serum DHT from baseline to Year 4. The three top‐ranked features involved stroma, and reflected greater shape uniformity of stromal nuclei, and greater color uniformity in drug‐treated tissue. Other top features associated with drug included: irregular clustering of epithelial nuclei and greater variation in lumen shape. Conclusions By mining a large set of hierarchical image features from whole slide images, we were able to develop a model that can discriminate dutasteride‐treated benign prostate tissue from placebo controls, a distinction not made by pathologists. Histomorphological changes linked to drug assignment were not associated with changes in PSA or serum DHT. The model scores and individual features provide insight into patient‐specific tissue response to dutasteride, and thus might predict response to this drug as a cancer chemopreventive agent. Support or Funding Information NIH/NCI RO1CA155301, Glaxo Smith Kline (Investigator Initiated Award)