Telecom Paris - JL Dessalles - Artificial Intelligence


	Telecom Paris Dep. Informatique & Réseaux J-L. Dessalles ← Home page July 2023

Cognitive Approach to Natural Language Processing (SD213)
➜ other AI courses

Objectives

Processing language is one of the most important and most challenging issues of Artificial Intelligence. NLP (Natural Language Processing) has many applications. It is commonly used in machine translation, in text mining, in speech recognition, in dialogue based applications, in text generation, in automatic summarization, in Web search, etc. Conversely, it is hard to imagine an "intelligent" machine that would be unable to understand language.
NLP remains a challenging task. Statistical techniques perform well in domains such as machine translation, but they are intrinsically limited to average meanings and cannot take contextual knowledge into account. This course explores some symbolic alternatives to mere statistics.
Some NLP techniques, like grammars, parsing and ontologies, are classic symbolic methods. Some others are inspired by cognitive modelling. They include procedural semantics, aspect processing, dialogue processing. The point is not only to adopt a "reverse engineering" approach to language, but also to adapt engineering techniques to human requirements to improve efficiency and acceptability.

Content

This course presents different NLP methods that are inspired by the study of natural language and of the underlying cognitive processes. However, the techniques and concepts that will be studying have a broader scope in artificial intelligence and are used to study reasoning, decision making and symbolic machine learning. They include:

Syntactic processing using context-free grammars. Basic parsing methods.
Knowledge representation – Meaning representation – Procedural semantics – Aspect.
Relevance: interest, newsworthiness, argumentative relevance and processing.

Prerequisites

SD206 (Logic and knowledge representation) is recommended, but not required. Labs are currently being translated into Python (so that both Python and Prolog versions will be available). Basic notions about Prolog will nevertheless be useful to know (unification, recursion, backtracking, declarativity).

Topics


Lecture 1	Introduction to symbolic NLP and to Linguistics ▶ Watch the lecture	Intro to the course Should AI imitate cognitive mechanisms? Linguistic resources, by M. Alam Natural Language Processing, by N. Holzenberger
Lecture 2	Procedural semantics ▶ Watch the lecture	Structures to represent meaning Procedural Semantics
Lecture 3	Contrast and aspect ▶ Watch the lecture on Aspect	Contrast Aspect
Lecture 4	Relevance and argumentation	Relevance Argumentation examples Argumentation theory CAN (conflict - abduction - negation) See also: Simplicity Theory website Wikipedia page on BDI (belief-desire-intention)
Lecture 5	Linguistic Linked Data (Mehwish Alam)	Linguistic Linked Data Demo (notebooks)
Lecture 6	Cognition-inspired Visual Question Answering (Nils Holzenberger)
-	Project, Quiz & evaluation	See 2022 quiz with answers

read →
	2023 Students’ micro-studies

Lab sessions


1. Syntax & parsing	19/04/2023 ➜ 25/04/2023
2. Procedural Semantics	10/05/2023 ➜ 16/05/2023
3. Processing aspect	17/05/2023 ➜ 23/05/2023
4. Relevance and argumentation	24/05/2023 ➜ 07/06/2023

Students are asked to complete the exercises of each session within 7 days.

Evaluation

Answers to questions during lab sessions will be recorded and evaluated.

Students will be asked to perform a small technical study (in pairs) by extending some issue addressed during lab sessions. They will also write a 3-4 page report (more if working in pairs, of course).
Students will answer a small quiz.

Project

Each student will choose a problem related to the above topics and perform a micro-research on that problem. Students will write a 3-page paper (typical structure: problem, relevant studies, claim, evidence, discussion, bibliography (with weblinks)).
Note: the project should include some programming (this is a computer science course). So pure bibliographic projects would not suffice.

The study should be related to symbolic NLP. The easiest way to do this study is to work on a topic closely related to one of the lab work sessions. You are free, however, to work on any other relevant topic. Be careful to keep it feasible: it’s supposed to be a mini-study.
Caveat: if your study involves statistical aspects, only the symbolic part will be considered in the evaluation. Implementation language should be Prolog or Python (ask in case of problem).

Examples: Extend a grammar to analyze more complex sentences (such as the fist sentence of this section); create a grammar for a different language; extend the lab work on procedural semantics to understand more sentences about chess; or to understand sentences about the genealogy of an actual family; extend the lab work on aspect to include more aspectual words (always, ancient, already, still, ...); create a mini-knowledge base on a specific domain (football, Roland-Garros...) and use CAN (last lab work) to propose interactive dialogues; etc.


Indicate the topic of your study ➜ HERE (you may change your mind at will). You may also See already chosen projects. Use this page to upload your report (see below report template).

Students may work in pairs. In this case, the respective contributions of each student should appear unambiguously. And the expectations are of course doubled.

You will present your work on the last day during no more than 3 minutes. Make sure to make it relevant to the audience.
A couple of days before presentations, you will post a few slides that will be displayed while you are talking.

Your small report will describe your project and what you found (typically: 2 or 3 pages of text). Don’t forget to use the template below.

Write you report ➜ Please use this template: MSWord or LibreOffice or LaTeX or PDF
Upload it ➜ Uploading page
Upload your program and any relevant material ➜ Uploading page

The project itself can be handed in until June 29th.
Please post:

your report
your source code
any suitable additional material

Bibliography

Introduction to AI

Konieczny, S. & Prade, H. (2021). Artificial Intelligence - What is it, exactly? College Publications.
Marquis, P., Papini, O. & Prade, H. (2020). A Guided Tour of Artificial Intelligence Research: Volume I: Knowledge Representation, Reasoning and Learning. Springer Nature.
Marquis, P., Papini, O. & Prade, H. (2020). A Guided Tour of Artificial Intelligence Research: Volume II: AI Algorithms. Springer Nature.
Marquis, P., Papini, O. & Prade, H. (2020). A Guided Tour of Artificial Intelligence Research: Volume III: Interfaces and Applications of Artificial Intelligence. Springer Nature.

Introduction to NLP

A short introduction to syntax:
Lasnik, H. (2002). The minimalist program in syntax. Trends in cognitive sciences, 6 (10), 432-437.
Another introduction to syntax:
Gerold Schneider. Introduction to Government and binding
About feature structures and unification: See these slides taken from:
Bender, E. M., Sag, I. A. & Wasow, T. (2003). Syntactic theory: A formal introduction. CSLI, Stanford, CA.
About using Prolog to process natural language:
Natural Language Processing Techniques in Prolog by Patrick Blackburn and Kristina Striegnitz
About using Python (NLTK) to process natural language:
Bird, S., Loper, E. & Klein, E. (2009). Natural Language Processing with Python. O’Reilly Media Inc.
Zhang, Y. & Teng, Z. (2021). Natural Language Processing: A Machine Learning Perspective. Cambridge University Press.
The Attempto Project: a rich subset of standard English designed to serve as knowledge representation language.

About the limits of statistical AI

Mitchell, M. (2021). Why AI is harder than we think. 16th Conf. of the European Chapter of the Association for Computational Linguistics (EACL 2021), (starts at 19:30). .
Difficulty of interpreting in context:
Scott, T. (2020). The Sentences Computers Can’t Understand -
Critique of statistical translation:
Hofstadter, D. R. (2018). The shallowness of Google Translate. The Atlantic, Jan, 30.
(voir aussi cette critique de l’apprentissage profond)
Dessalles, J.-L. (2019). Des intelligences très artificielles. Paris: Odile Jacob.
Marcus, G. (2018). Deep learning: A critical appraisal. ArXiv, 1801.006.
Marcus, G. (2020). The next decade in ai: Four steps towards robust artificial intelligence. ArXiv, 2002.062.
Marcus, G. (2020). GPT-2 and the Nature of Intelligence. The Gradient.
Marcus, G. & Davis, E. (2020). GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about. MIT Technology review.

Cognitive Linguistics

Gärdenfors, P. (2000). Conceptual spaces: The geometry of thought. Cambridge, MA: MIT Press.
Gärdenfors, P. (2001). Conceptuel spaces as a basis for cognitive semantics. In A. Clark, J. Ezquerro & J. M. Larrazabal (Eds.), Philosophy and Cognitive Science: Categories, Consciousness, and Reasoning: Proceeding of the Second International Colloquium on Cognitive Science, 159-180. New York, NY: Springer Science & Business Media.
Gärdenfors, P. & Williams, M.-A. (2001). Reasoning about categories in conceptual spaces. International Joint Conference on Artificial Intelligence (IJCAI), 385-392. .
Gärdenfors, P. (2014). The geometry of meaning - Semantics based on conceptual spaces. Cambridge, MA: MIT Press.
Moens, M. & Steedman, M. (1998). Temporal ontology and temporal reference. Computational Linguistics, 14 (2), 15-28.
Munch, D. & Dessalles, J.-L. (2014). Assessing Parsimony in Models of Aspect. In P. Bello, M. Guarini, M. McShane & B. Scassellati (Eds.), Proceedings of the 36th Annual Conference of the Cognitive Science Society, 2121-2126. Austin, TX: Cognitive Science Society.

Explainable AI

Doran, D., Schulz, S. & Besold, T. R. (2017). What does explainable AI really mean? A new conceptualization of perspectives. ArXiv, 1710.008.
Wang, D., Yang, Q., Abdul, A. & Lim, B. Y. (2019). Designing theory-driven user-centric explainable AI. Proceedings of the 37th international conference on Human factors in computing systems (CHI’19), paper 601. ACM.