Enhancing Structured Pathology Report Generation with Foundation Model and Modular Design

Pathology report generation is challenging as it requires analyzing giga-pixel Whole Slide Images (WSIs), making it a complex and labor-intensive task, yet crucial for clinical decision-making. An AI-driven report generation system can assist in producing accurate and concise reports. To address these needs, we designed a two-stage framework consisting image analysis and text generation, improving diagnostic performance and optimizing it for real-world clinical application. In the first stage, using 752 high resolution WSIs from bladder tumor tissue samples collected at Korea University Medical Center, we leveraged foundation models to extract robust features. Then, we built a multi-label classification based on key diagnostic elements to achieve high performance with an Attention-based Multi-Instance Learning (MIL) model and a Transformer-based MIL with Knowledge Distillation. For report generation stage, we designed a T5-based text-to-text model, simplifying input representations whereas integrating data augmentation to improve stability and generalization. The proposed model achieved a ROUGE score of 0.87, a BLEU-4 score of 0.94, a Jaccard score of 0.89 and a BioLLM score of 0.97. An additional evaluation conducted by the institution maintaining the exclusive K-MEDICON test datasets confirmed a consistent performance yielding an overall score of 0.88, calculated as a weighted sum of aforementioned metrics. Its consistency validated through a Mean Opinion Score (MOS) evaluation by pathologists. By systematically integrating foundation models and modular structure, the benchmark results demonstrate that the task can be effectively solved with limited data and computational resources, indicating promising potential for real-world clinical adaptation.