Fine-Tuned LLMs vs. Rule-Based NLP for UML Diagram Generation: An Educational Evaluation

The automatic generation of Unified Modeling Language (UML) diagrams from natural language requirements continues to be a difficult task in software engineering education and practice. Conventional rule-based Natural Language Processing (NLP) approaches often fail to handle semantic ambiguity and typically demand extensive manual adjustments. Recent developments in Large Language Models (LLMs) demonstrate stronger contextual reasoning abilities, but their capability to produce accurate and structured diagrams-particularly in instructional settings—has not been thoroughly examined. This work compares a traditional NLP pipeline with a fine-tuned Qwen2.5-Coder-7B-Instruct model using a dataset of 75 annotated specifications. The fine-tuned model attained a semantic fidelity of 94%, a G-Eval correctness score of 0.78, and reduced manual correction requirements by 78%. In contrast, the NLP pipeline achieved 67% fidelity and required a 52% correction effort. Statistical analysis using MANOVA and ANOVA confirmed that these differences across the student, NLP, and LLM groups were statistically significant (p < 0.001). Moreover, students working with LLM-generated diagrams reported a 30% decrease in effort and greater engagement during modeling tasks. The findings demonstrate that fine-tuned LLMs can substantially improve the precision and interpretability of automated UML diagram generation while supporting more interactive and effective learning in software engineering contexts.