CORRELATION ANALYSIS OF AUTOMATIC QUALITY METRICS FOR TEXT GENERATION

Author(s): Sergeev D.S., Vasilyeva L.I.

Rubric: Information technology

DOI: 10.21777/2500-2112-2026-2-90-97

Release: 2026-2 (55)

Pages: 90-97

Keywords: large language models, text generation quality evaluation, automatic metrics, semantic similarity, expert judge- ment, correlation analysis, regression model, technical documentation

Annotation: The article describes the issue of evaluating the quality of texts generated by large language models when deployed locally in corporate information systems. The study focuses on analyzing the consistency between automatic evaluation metrics and expert assessment for short technical descriptions of data attributes. A medium-sized language model operating in a local computational environment with parameter optimization techniques was used as the research object. For the compiled dataset of generated texts, expert scores were obtained using a discrete rating scale, and automatic indicators reflecting semantic and lexical similarity to reference descriptions were calculated. Correlation analysis was performed, and a linear regression model was constructed to approximate expert evaluation based on automatic metric values. The results demonstrate that semantic metrics exhibit a high level of agreement with expert judgment and provide stable prediction of generation quality. The findings can be applied in automated validation and tuning of technical documentation generation systems.

Bibliography: Sergeev D.S., Vasilyeva L.I. CORRELATION ANALYSIS OF AUTOMATIC QUALITY METRICS FOR TEXT GENERATION // Education Resources and Technologies. – 2026. – № 2 (55). – С. 90-97. doi: 10.21777/2500-2112-2026-2-90-97