Intelligent Systems.
Theory and Applications

(Intellektual'nye Sistemy. Teoriya i Prilozheniya)

Efficient Audit of Text Annotations Using Instance Hardness Assessments

Abstract

Detecting and correcting span boundary and label errors is essential in social sciences and humanities but costly to automate. To reduce costs, we propose a method using instance hardness assessments, efficient vectorization, and one-time document encoding. Comparison with LLMs on three correction sources shows comparable quality on several auditing tasks with substantially shorter runtimes.

Keywords: annotation auditing; text spans; instance hardness; annotation correction; LLM-as-a-judge.

BibTeX
@article{IS-Tabachenkov-Maysuradze2026,
  author  = {Tabachenkov, Andrey Mikhailovich and Maysuradze, Archil Iverievich},
  title   = {{Efficient Audit of Text Annotations Using Instance Hardness Assessments}},
  journal = {Intelligent Systems. Theory and Applications},
  year    = {2026},
  volume  = {30},
  number  = {4},
  pages   = {417--420},
}
AMSBIB
\Bibitem{IS-Tabachenkov-Maysuradze2026}
\by A.\,M.~Tabachenkov, A.\,I.~Maysuradze
\paper Efficient Audit of Text Annotations Using Instance Hardness Assessments
\jour Intelligent Systems. Theory and Applications
\yr 2026
\vol 30
\issue 4
\pages 417--420
\lang In Russian
Published under Creative Commons Attribution 4.0 International (CC BY 4.0)

← Back to issue

× Issue cover