Shared task proposal: Instruction giving in virtual worlds

This paper reports on the results of the working group “Virtual Environ-ments ” at the Workshop on Shared Tasks and Comparative Evaluation for NLG. This working group discussed the use of virtual environments as a platform for NLG evaluation, and more specifically of the generation of in

Erinnerungen an Niederhof im Riesengebirge: anläßlich des 400jährigen Bestehens der Gemeinde Niederhof: 1601 - 2001

unveränderter Nachdruck 2014 der Ausgabe 200

Generating Anaphoric Expressions : Contextual Reasoning in Sentence Planning

This thesis investigates the contextual reasoning involved in the production of anaphoric expressions in natural language generation systems. More specifically, I propose generation strategies for two types of discourse anaphora which have not been treated in generation before: bridging descriptions and additive particles. To this end the contextual conditions that govern the use of these expressions have to be formalized. The formalization that I propose is based on notions from linguistics and extends previous approaches to the generation of co-referential anaphora. I then specify the reasoning tasks that have to be carried out in order to check the contextual conditions. I describe how they can be implemented using a state-of-the-art reasoning system for description logics, and I compare my proposal to alternative approaches using other kinds of reasoning tools. Finally, I describe an experimental implementation of the proposed approach.Diese Arbeit untersucht die kontextuellen Schlussfolgerungen, die bei der Erzeugung von anaphorischen Ausdrücken in Textgenerierungssystemen anfallen. Ich entwickele Generierungsstratgien für zwei Arten von Diskursanaphern, die bisher nicht in der Generierung behandelt worden sind: indirekte Anaphern und additive Partikel. Dafür müssen die kontextuellen Bedingungen, unter denen diese Ausdrücke verwendet werden, bestimmt und formalisiert werden. Die Formalisierung, die ich vorschlage, basiert auf linguistischen Theorien und verallgemeinert ältere Ansätze zur Generierung von koreferentiellen Anaphern. Dann spezifiziere ich die Inferenzaufgaben, die gelöst werden müssen, um die kontextuellen Bedingungen zu überprüfen. Ich beschreibe, wie sie mit Hilfe von aktuellen Inferenzsystemen für terminologische Logiken implementiert werden können, und vergleiche meinen Ansatz mit Alternativen, die andere Arten von logischen Inferenzsystemen nutzen. Schließlich, beschreibe ich eine experimentelle Implementation meines Ansatzes

Generating Instructions in Virtual Environments (GIVE): A Challenge and an Evaluation Testbed for NLG

Would it be helpful or detrimental for the field of NLG to have a generally accepted competition? Competitions have definitely advanced the state of the art in some fields of NLP, but the benefits sometimes come at the price of over-competitiveness, and there is a danger of overfitting systems to the concrete evaluation metrics. Moreover, it has been argued that there are intrinsic difficulties in NLG that make it harder to evaluate than other NLP tasks (Scott and Moore, 2006)

The software architecture for the First Challenge on Generating Instructions in Virtual Environments.

The GIVE Challenge is a new Internet-based evaluation effort for natural language generation systems. In this paper, we motivate and describe the software infrastructure that we developed to support this challenge

Validating the Web-based Evaluation of NLG Systems

The GIVE Challenge is a recent shared task in which NLG systems are evaluated over the Internet. In this paper, we validate this novel NLG evaluation methodology by comparing the Internet-based results with results we collected in a lab experiment. We find that the results delivered by both methods are consistent, but the Internet-based approach offers the statistical power necessary for more fine-grained evaluations and is cheaper to carry out

Shared task proposal: Instruction giving in virtual worlds

The GIVE Challenge is a new Internet based evaluation effort for natural language generation systems. In this paper, we motivate and describe the software infrastructure that we developed to support this challenge

Report on the Second NLG Challenge on Generating Instructions in Virtual Environments (GIVE-2)

We describe the second installment of the Challenge on Generating Instructions in Virtual Environments (GIVE-2), a shared task for the NLG community which took place in 2009--10. We evaluated seven NLG systems by connecting them to 1825 users over the Internet, and report the results of this evaluation in terms of objective and subjective measures