Efficient Audit of Text Annotations Using Instance Hardness Assessments
Received: 24 Oct 2026 Revised: 26 Oct 2026 Accepted: 09 Nov 2026
Published: 2026, vol. 30, issue 4, pp. 417–420
Abstract
Detecting and correcting span boundary and label errors is essential in social sciences and humanities but costly to automate. To reduce costs, we propose a method using instance hardness assessments, efficient vectorization, and one-time document encoding. Comparison with LLMs on three correction sources shows comparable quality on several auditing tasks with substantially shorter runtimes.
Keywords: annotation auditing; text spans; instance hardness; annotation correction; LLM-as-a-judge.
BibTeX
@article{IS-Tabachenkov-Maysuradze2026,
author = {Tabachenkov, Andrey Mikhailovich and Maysuradze, Archil Iverievich},
title = {{Efficient Audit of Text Annotations Using Instance Hardness Assessments}},
journal = {Intelligent Systems. Theory and Applications},
year = {2026},
volume = {30},
number = {4},
pages = {417--420},
}
AMSBIB
\Bibitem{IS-Tabachenkov-Maysuradze2026}
\by A.\,M.~Tabachenkov, A.\,I.~Maysuradze
\paper Efficient Audit of Text Annotations Using Instance Hardness Assessments
\jour Intelligent Systems. Theory and Applications
\yr 2026
\vol 30
\issue 4
\pages 417--420
\lang In Russian
Published under
Creative Commons Attribution 4.0 International (CC BY 4.0)
RU