Natural Language Interpretation

DIT410/TIN174, Artificial Intelligence

John J. Camilleri

7 April, 2017

https://img.memesuper.com/7ad355dacca363617cdfcff7defc07ed_-of-morpheus-offering-the-morpheus-pill-meme_520-412.jpeg

Last time…

“Mary saw the man with a telescope”

“Colourless green ideas sleep furiously”

Colourless green ideas sleep furiously

http://wmjasco.blogspot.se/2008/11/colorless-green-ideas-do-not-sleep.html

✋ Is this sentence valid? Yes or No

Why syntax?

Semantic representation

Introducing logical terms

  • Mary = Mary
  • the man = Man
  • Mary saw the man = Saw(Mary, Man)

Semantic interpretation (1)


With(Saw(Mary, Man), Telescope)

Semantic interpretation (2)


Saw(Mary, With(Man, Telescope))

Compositional semantics

  • Mary = Mary
  • the man = Man
  • Mary saw the man = Saw(Mary, Man)
  • saw = λy λx · Saw(x, y)
  • saw the man = λx · Saw(x, Man)

Interpretation

syntactic representation → semantic representation

parse tree → logical term

Utterance: “move the white ball into the red box”

✋ Is this ambiguous? Yes or No

Goal: inside(white_ball, red_box)

Utterance: “move the ball into the red box”

✋ Is this ambiguous? Yes or No

Shrdlite pipeline

  1. Parsing: text input → parse trees
  2. Interpretation: parse tree + world → goals
  3. Ambiguity resolution: many goals → one goal
  4. Planning: goal → robot movements

Parsing

text input → parse trees

```function parse(input:string) : string | ShrdliteResult[]

{: .code}

```interface ShrdliteResult {
    input : string
    parse : Command
    interpretation? : DNFFormula
    plan? : string[]
}

Grammar (simplified)

From file Grammar.ne

```command –> “put” entity location entity –> quantifier object object –> size:? color:? form object –> object location location –> relation entity

{: .code}

Notes:

- Recursion
- Draw a tree top-down on the board

---

_“put the white ball in a box on the floor”_  
![](img/nlp/shrdlite-small.png){:.noborder}  
 Is this ambiguous?
<span style="background:lime;color:white;padding:3px 6px;">Yes</span> or
<span style="background:magenta;color:white;padding:3px 6px;">No</span>  

---

_“put the white ball in a box on the floor”_  
![](img/nlp/shrdlite-small.png){:.noborder}  
 Is the ambiguity
<span style="background:lime;color:white;padding:3px 6px;">syntactic</span> or
<span style="background:magenta;color:white;padding:3px 6px;">semantic</span>?

Notes:

It is both syntactically **and** semantically ambiguous

---

## Parse 1

_"put the white ball **that is** in a box on the floor"_

![](img/nlp/put-the-white-ball-etc-parse-1.png){:.noborder}


Notes:

- Show optional "that is" in grammar.

---

## Parse 2

_"put the white ball in a box **that is** on the floor"_

![](img/nlp/put-the-white-ball-etc-parse-2.png){:.noborder}

---

# Interpretation

`parse tree + world  goals`

```function interpretCommand(
    cmd: Command,
    state: WorldState
  ): DNFFormula

Logical interpretations (“Goals”)

```type DNFFormula = Conjunction[] type Conjunction = Literal[]

{: .code}

DNF = Disjunctive Normal Form

Example: `(x  y)  (z)`

```DNFFormula([Conjunction([x, y]), Conjunction(z)])

Literals

```interface Literal { relation : string; args : string[]; polarity : boolean; }

{: .code}

Example: `ontop(a,b)`

```{ relation:"ontop", args:["a","b"], polarity:true }

Spatial relations

  • x is on top of y if it is directly on top
  • x is above y if it is somewhere above

Ambiguity

  • DNF inherently captures ambiguity
  • But impossible interperetations should be removed

“put the white ball that is in a box on the floor”

There is no spoon white ball in a box.

“put the white ball in a box on the floor”

inside(WhiteBall, YellowBox)
Yellow box is already on floor: 3 moves

inside(WhiteBall, RedBox) ∧ on(RedBox, floor)
Red box can be placed on floor first: 2 moves

Final interpretation

inside(WhiteBall, YellowBox) ∨ (inside(WhiteBall, RedBox) ∧ on(RedBox, floor))

Physical laws

  • Balls must be in boxes or on the floor, otherwise they roll away.
  • Small objects cannot support large objects.

Interpreter test cases

  • Each test case contains a list of interpretations
  • Each interpretation is already a list (a disunction of conjunctions)
  world: "small",
  utterance: "take a blue object",
  interpretations: [["holding(BlueTable)","holding(BlueBox)"]]
}
  world: "small",
  utterance: "put a black ball in a box on the floor",
  interpretations: [["inside(BlackBall,YellowBox)"],
                    ["ontop(BlackBall,floor)"]]
}

Conjunction

  world: "small",
  utterance: "put all balls on the floor",
  interpretations: [["ontop(WhiteBall,floor) & ontop(BlackBall,floor)"]]
}

No valid interpretations

  world: "small",
  utterance: "put a ball on a table",
  interpretations: []
}

Breaks the laws of nature!

Some interpretations are missing

  world: "small",
  utterance: "put a ball in a box on the floor",
  interpretations: [["COME UP WITH YOUR OWN INTERPRETATION"]]
}

Tips for interpreter in Shrdlite

  • Sub-functions based on grammar types
  • Use instanceof when traversing parse tree (Command)
  • Use recursion to handle nesting
    “put a box in a box on a table on the floor”

Ambiguity resolution

Handling multiple interpretations

  • Fail
  • Pick “first”
  • Use some rules of thumb
    e.g. prefer box already on floor
  • Ask the user for clarification (extension)

Planning

goal → robot movements

  • Movements: left, right, pick, drop
  • Use graph search
  • Given a disjunction of goals, should find the easiest to satisfy

Audience participation meta-question

✋ Do you prefer Socrative or post-it notes?

Thank you for returning your post-it notes! 🦄