Classificação de dificuldade de questões de programação com base na inteligibilidade do enunciado

In large programming classes mediated by online judges, one must randomize the chosen questions for evaluation tests, in order to avoid dishonest behavior. The problem of a naive randomization is the lack of equity in respect to the degree of difficulty of the questions. Thus, this paper proposes a method to classify the difficulty of programming questions based on textual intelligibility metrics extracted from their statements. In total, 450 questions were analyzed, and those were answered by 800 students in a introduction to programming discipline between the academic periods of 2017 to 2018. It was verified that an approach based only on the intelligibility metrics is suboptimal for the classification of difficulty of questions, but not irrelevant.