News-on-Demand: Types of Errors

Of course, a fully automated system like News-on-Demand will inevitably make errors. We distinguish five types of errors, all of which are subjects of active research aimed at improving the system:

• False segmentation of stories. This happens when we incorrectly identify the beginning and end of a video paragraph associated with a single news story. Incorrect segmentation is usually due to inaccurate transcription, either from true closed-captioning errors, errors in processing the closed-captioning text into stories, or errors in the segmentation based on the speech transcription.
• False words in transcripts. Errors in the transcript are either the result of faulty speech recognition or errors in the closed-captioned text. The result is the appearance of incorrect words in stories and consequent errors in the index and in retrieval.
• False synchronization designates retrieved words that were actually spoken elsewhere in the video. This is generally due to closed-captioning that does not match the words returned by speech recognition.
• Incorrectly recognized query. This is the result of an incorrect speech recognition during the library exploration. The user can edit and correct query misrecognitions through typing, or simply repeat or rephrase the query.
• Incorrect set of stories returned for a query. This type of error is measured through information recall and precision. The user might get stories that are not relevant to the query or miss relevant stories. Some of these are caused by previously mentioned problems, and others are the result of shortcomings in the processing of the query into retrieval keywords.