Matches in ScholarlyData for { <https://w3id.org/scholarlydata/inproceedings/lrec2008/papers/832> ?p ?o. }
Showing items 1 to 19 of
19
with 100 items per page.
- 832 creator deryle-lonsdale.
- 832 creator eric-ringger.
- 832 creator james-carroll.
- 832 creator kevin-seppi.
- 832 creator marc-carmen.
- 832 creator noel-ellison.
- 832 creator peter-mcclanahan.
- 832 creator robbie-haertel.
- 832 type InProceedings.
- 832 label "Assessing the Costs of Machine-Assisted Corpus Annotation through a User Study".
- 832 sameAs 832.
- 832 abstract "Fixed, limited budgets often constrain the amount of expert annotation that can go into the construction of annotated corpora. Estimating the cost of annotation is the first step toward using annotation resources wisely. We present here a study of the cost of annotation. This study includes the participation of annotators at various skill levels and with varying backgrounds. Conducted over the web, the study consists of tests that simulate machine-assisted pre-annotation, requiring correction by the annotator rather than annotation from scratch. The study also includes tests representative of an annotation scenario involving Active Learning as it progresses from a naïve model to a knowledgeable model; in particular, annotators encounter pre-annotation of varying degrees of accuracy. The annotation interface lists tags considered likely by the annotation model in preference to other tags. We present the experimental parameters of the study and report both descriptive and inferential statistics on the results of the study. We conclude with a model for estimating the hourly cost of annotation for annotators of various skill levels. We also present models for two granularities of annotation: sentence at a time and word at a time.".
- 832 hasAuthorList authorList.
- 832 hasTopic Linguistics.
- 832 isPartOf proceedings.
- 832 keyword "Acquisition, Machine Learning".
- 832 keyword "Corpus (creation, annotation, etc.)".
- 832 keyword "Tagging".
- 832 title "Assessing the Costs of Machine-Assisted Corpus Annotation through a User Study".