Natural Language Interpretation

DIT410/TIN174, Artificial Intelligence

John J. Camilleri

7 April, 2017

Last time…

“Mary saw the man with a telescope”

“Colourless green ideas sleep furiously”

Colourless green ideas sleep furiously

✋ Is this sentence valid? Yes or No

Why syntax?

Semantic representation

Introducing logical terms

  • Mary = Mary
  • the man = Man
  • Mary saw the man = Saw(Mary, Man)

Semantic interpretation (1)

With(Saw(Mary, Man), Telescope)

Semantic interpretation (2)

Saw(Mary, With(Man, Telescope))

Compositional semantics

  • Mary = Mary
  • the man = Man
  • Mary saw the man = Saw(Mary, Man)
  • saw = λy λx · Saw(x, y)
  • saw the man = λx · Saw(x, Man)


syntactic representation → semantic representation

parse tree → logical term

Utterance: “move the white ball into the red box”

✋ Is this ambiguous? Yes or No

Goal: inside(white_ball, red_box)

Utterance: “move the ball into the red box”

✋ Is this ambiguous? Yes or No

Shrdlite pipeline

  1. Parsing: text input → parse trees
  2. Interpretation: parse tree + world → goals
  3. Ambiguity resolution: many goals → one goal
  4. Planning: goal → robot movements


text input → parse trees

```function parse(input:string) : string | ShrdliteResult[]

{: .code}

```interface ShrdliteResult {
    input : string
    parse : Command
    interpretation? : DNFFormula
    plan? : string[]

Grammar (simplified)

From file

```command –> “put” entity location entity –> quantifier object object –> size:? color:? form object –> object location location –> relation entity

{: .code}


- Recursion
- Draw a tree top-down on the board


_“put the white ball in a box on the floor”_  
 Is this ambiguous?
<span style="background:lime;color:white;padding:3px 6px;">Yes</span> or
<span style="background:magenta;color:white;padding:3px 6px;">No</span>  


_“put the white ball in a box on the floor”_  
 Is the ambiguity
<span style="background:lime;color:white;padding:3px 6px;">syntactic</span> or
<span style="background:magenta;color:white;padding:3px 6px;">semantic</span>?


It is both syntactically **and** semantically ambiguous


## Parse 1

_"put the white ball **that is** in a box on the floor"_



- Show optional "that is" in grammar.


## Parse 2

_"put the white ball in a box **that is** on the floor"_



# Interpretation

`parse tree + world  goals`

```function interpretCommand(
    cmd: Command,
    state: WorldState
  ): DNFFormula

Logical interpretations (“Goals”)

```type DNFFormula = Conjunction[] type Conjunction = Literal[]

{: .code}

DNF = Disjunctive Normal Form

Example: `(x  y)  (z)`

```DNFFormula([Conjunction([x, y]), Conjunction(z)])


```interface Literal { relation : string; args : string[]; polarity : boolean; }

{: .code}

Example: `ontop(a,b)`

```{ relation:"ontop", args:["a","b"], polarity:true }

Spatial relations

  • x is on top of y if it is directly on top
  • x is above y if it is somewhere above


  • DNF inherently captures ambiguity
  • But impossible interperetations should be removed

“put the white ball that is in a box on the floor”

There is no spoon white ball in a box.

“put the white ball in a box on the floor”

inside(WhiteBall, YellowBox)
Yellow box is already on floor: 3 moves

inside(WhiteBall, RedBox) ∧ on(RedBox, floor)
Red box can be placed on floor first: 2 moves

Final interpretation

inside(WhiteBall, YellowBox) ∨ (inside(WhiteBall, RedBox) ∧ on(RedBox, floor))

Physical laws

  • Balls must be in boxes or on the floor, otherwise they roll away.
  • Small objects cannot support large objects.

Interpreter test cases

  • Each test case contains a list of interpretations
  • Each interpretation is already a list (a disunction of conjunctions)
  world: "small",
  utterance: "take a blue object",
  interpretations: [["holding(BlueTable)","holding(BlueBox)"]]
  world: "small",
  utterance: "put a black ball in a box on the floor",
  interpretations: [["inside(BlackBall,YellowBox)"],


  world: "small",
  utterance: "put all balls on the floor",
  interpretations: [["ontop(WhiteBall,floor) & ontop(BlackBall,floor)"]]

No valid interpretations

  world: "small",
  utterance: "put a ball on a table",
  interpretations: []

Breaks the laws of nature!

Some interpretations are missing

  world: "small",
  utterance: "put a ball in a box on the floor",

Tips for interpreter in Shrdlite

  • Sub-functions based on grammar types
  • Use instanceof when traversing parse tree (Command)
  • Use recursion to handle nesting
    “put a box in a box on a table on the floor”

Ambiguity resolution

Handling multiple interpretations

  • Fail
  • Pick “first”
  • Use some rules of thumb
    e.g. prefer box already on floor
  • Ask the user for clarification (extension)


goal → robot movements

  • Movements: left, right, pick, drop
  • Use graph search
  • Given a disjunction of goals, should find the easiest to satisfy

Audience participation meta-question

✋ Do you prefer Socrative or post-it notes?

Thank you for returning your post-it notes! 🦄