Towards Automated Lexicography: Generating and Evaluating Definitions for Learner's Dictionaries

Dictionary definitions are an essential resource for learning word senses, but manually creating them is costly. We thus study dictionary definition generation (DDG), i.e., the generation of non-contextualized definitions for given headwords. Specifically, we address learner's dictionary definition generation (LDDG), where definitions should be written using simple vocabulary. First, we introduce a reliable evaluation approach for DDG, based on newly proposed evaluation criteria and powered by an LLM-as-a-judge. To provide reference definitions for the evaluation, we construct a dataset of Japanese dictionary definitions in collaboration with a professional lexicographer. Validation results demonstrate that our evaluation approach agrees with human annotators at a level comparable to inter-annotator agreement. Second, we propose an LLM-based LDDG approach that employs iterative simplification. Experimental results show that our approach yields definitions that achieve high scores on the proposed criteria and exhibit high lexical simplicity.