Detail Human Semantic MT Evaluation with HMEANT for IWSLT 2013   Oleh:  Lo, Chi-Kiu ; Wu, Dekai Jenis: Article from Proceeding Dalam koleksi: Proceedings of the 10th International Workshop on Spoken Language Translation (IWSLT 2013), Heidelberg, Germany: Dec. 5-6, 2013 Fulltext: Human Semantic MT Evaluation with HMEANT.pdf (11.77MB) Isi artikel We present the results of large-scale human semantic MT evaluation with HMEANT on the IWSLT 2013 German-English MT and SLT tracks and show that HMEANT evaluates the performance of the MT systems differently compared to BLEU and TER. Together with the references, all the translations are annotated by annotators who are native English speakers in both semantic role labeling stage and role filler alignment stage of HMEANT. We obtain high inter-annotator agreement and low annotation time costs which indicate that it is feasible to run a large-scale human semantic MT evaluation campaign using HMEANT. Our results also show that HMEANT is a robust and reliable semantic MT evaluation metric for running large-scale evaluation campaigns as it is inexpensive and simple while maintaining the semantic representational transparency to provide a perspective which is different from BLEU and TER in order to understand the performance of the state-of-the-art MT systems. Opini Anda Klik untuk menuliskan opini Anda tentang koleksi ini! Kembali

Copyright © 2006, 2007 Unika Atma Jaya, all rights reserved