texai.org

The English Comprehension Skill is a principle component of the Bootstrap Dialog System Design. In turn, this skill has the following subcomponents:

• ParsingRuleApplicationLibrary -looks up and applies Fluid Construction Grammar rules to the input utterance, word by word
• DiscourseElaborationLibrary - performs marker-passing spreading activation from ambiguous utterance concepts in order to find additional salient connections to previously known concepts in the discourse
• KintschSpreadingActivationLibrary -performs spreading activation over a graph of alternative meaning propositions to find the best interpretation of the input utterance

Two data structures support the above libraries:

• ParsingInterpretationNode - instances form a tree and each node contains all the significant state of the parser for a particular meaning interpretation at a particular point in the parse
• DiscourseContext - contains the text and corresponding meaning statements for each remembered utterance in the discourse

The following diagrams were produced during the processing of the use case utterance: “the book is on the table“. As part of the experiment, the discourse context is primed with knowledge of a room, which is an instance of cyc:RoomInAConstruction, and a book, which is an instance of cyc:BookCopy. Lexical rules matching word stems in the use case utterance yield these ambiguous meanings:

• book - a bound book copy
• book - a sheath of paper, e.g. match book
• is - has as an attribute
• is - situation described as
• on - an operational device
• “on the table” - subject to negotiation [ a multiword construction ]
• on - located on the surface of

The EnglishComprehensionSkill Java class contains Java libraries that adapt between Texai’s robotics-oriented hierarchical control system and plain old Java objects. The behavior of the ParsingRuleApplicationLibrary is drawn below as a tree of parsing interpretation nodes. Each node in the tree is either an input word, such as ‘the’, or the name of an applied fluid construction grammar rule. Branches in this tree occur when there are alternative interpretations (i.e. meanings) for a word such as “book“. The parsing interpretation tree is retained after the parsing process completes so that the user can ask questions about the parsing state (e.g. why a certain grammar rule did not apply as expected).

Figure 1. The tree of two alternative parsing interpretations for the partial utterance “the book” whose leaves are: node 16 which represents an instance of cyc:BookCopy, and node 17 which represents an instance of texai:SheetsBoundTogetherOnOneEdge.

The KintschSpreadingActivationLibrary is used by the EnglishComprehensionSkill to disambiguate alternative parsing interpretations. According to Walter Kintsch’s theory of reading comprehension, spreading activation flows over the nodes of a graph formed by the meaning propositions of the utterance. Links in this graph connect nodes mentioning the same term. The most relevant set of nodes receives the highest activation.

In figure 3 below are the ten propositions from the alternative parsing interpretations of the phrase “the book“. In the corresponding figure 4, magenta colored nodes indicate the interpretation: texai:SheetsBoundTogetherOnOneEdge, Cyan colored nodes indicated the alternative interpretation cycBookCopy. The yellow nodes indicates prior knowledge - N4 is the prior discourse context knowledge about a cyc:Table, and N5 is the prior discourse context knowledge about a cyc:BookCopy. N1 and N7 are positively connected, which is indicated by a black line, because they share the concept: texai:?SheetsBoundTogetherOnOneEdge-2. Node N1 and N10 are negatively connected, which is indicated by a red line, because they represent alternative, conflicting, interpretations.

• N1 (texai:?SheetsBoundTogetherOnOneEdge-2 texai:fcgStatus texai:SingleObject)
• N2 (texai:?BookCopy-1 rdf:type cyc:BookCopy)
• N3 (texai:?BookCopy-1 texai:fcgDiscourseRole texai:external)
• N4 (texai:table-0 rdf:type cyc:Table)
• N5 (texai:book-0 rdf:type cyc:BookCopy)
• N6 (texai:?BookCopy-1 rdf:type texai:PreviouslyIntroducedThingInThisDiscourse)
• N7 (texai:?SheetsBoundTogetherOnOneEdge-2 rdf:type texai:PreviouslyIntroducedThingInThisDiscourse)
• N8 (texai:?SheetsBoundTogetherOnOneEdge-2 rdf:type texai:SheetsBoundTogetherOnOneEdge)
• N9 (texai:?SheetsBoundTogetherOnOneEdge-2 texai:fcgDiscourseRole texai:external)
• N10 (texai:?BookCopy-1 texai:fcgStatus texai:SingleObject)

Figure 2. The propositions from the alternative parsing interpretations in “the book“.

According to the reading-comprehension algorithm of Walter Kintsch, it starts from an initial, equal-activation state, and iterative spreading activation is performed until either a maximum count is reached or until the node activation value change is smaller than a predetermined delta. The parsing interpretations are ranked according to their aggregate node activation. The KintschSpreadingActivationLibrary then prunes the lowest scoring alternatives from the parsing interpretation tree as specified by its beamwidth parameter. In this use case the beamwidth is one, allowing a single surviving interpreation.

Figure 3. The Kintsch spreading activation graph, which highly activates the cyc:BookCopy interpretation, colored cyan, due to the presence of an existing cyc:BookCopy (N5) in the discourse context.

Figure 4. The parsing interpretation tree for the partial utterance “the book” after pruning the less activated alternative interpretation.

After determining the most likely interpretation for the word “book“, and then proceding to parse the whole utterance: “the book is on the table“, the parsing interpretation tree again is branched due to the remaining ambiguous words and phrases.

 

 

Figure 5. The parsing interpretation tree for the utterance: “the book is on the table“, before pruning the less activated interpretations. Click on the image for a full sized version.

Discourse elaboration is performed before Kintsch spreading activation so that ambiguous concepts in the input utterance might be inferred to be conceptually related to previously known concepts in the discourse context. In this use case, the word “table” is ambiguous. It could either mean cyc:Table, or as part of the multiple word construction “on the table“, mean subject to negotiation. There is no known table in the discourse context, but there is a known instance of cyc:RoomInAConstruction. For this use case, there are commonsene rules in the Texai knowledge base:

• a room may typically contain furniture
• a room may typically have a window
• a room has a ceiling
• a room has a wall
• a room has a floor
• a room has a door
• a room has a means of illumination
• a room can contain a person
• a table is a type of furniture
• a family room is a type of room

These concepts form nodes in a graph, whose links designate a conceptual relationship between two concepts. Marker-passing spreading activation originates at the known discourse terms (e.g. cyc:RoomInAConstruction) and at each significant utterance term (e.g. cyc:Table) and terminates if paths meet (e.g. at cyc:FurniturePiece). When it can be inferred in this fashion that an utterance term and a known discourse term are conceptually related, then that proposition is added to the meaning propositions for subsequent Kintsch spreading activation to resolve ambiguities. The marker-passing spreading activation decays after only a few links to preclude weakly conceptually related results.

 

 

Figure 6. The discourse elaboration, marker-passing, spreading activation graph. The first path originating from RoomInAConstruction meets at FurniturePiece with the second path originating from Table. Thus Table is conceptually related with RoomInAConstruction. Click on the image for a full-sized version.

In figure 7 below are listed the 24 propositions from the alternative parsing interpretations. In the corresponding figure 8, magenta and cyan nodes represent sets of conflicting meanings interpretations and green nodes represent shared meanings (i.e. that ‘book’ is already decided to be a cyc:BookCopy). N15 was added by the discourse elaboration step.

• N1 (texai:?Table-8 rdf:type cyc:Table)
• N2 (texai:?OperationalDevice-3 rdf:type texai:OperationalDevice)
• N3 (texai:?Situation-Localized-6 cyc:situationConstituents texai:?BookCopy-4)
• N4 (texai:?BookCopy-4 texai:fcgDiscourseRole texai:external)
• N5 (texai:Room-0 rdf:type cyc:RoomInAConstruction)
• N6 (texai:?Situation-ObjHasAttribute-9 texai:situationHappeningOnDate cyc:Now)
• N7 (texai:?Table-8 texai:fcgDiscourseRole texai:external)
• N8 (texai:?Situation-Localized-6 texai:situationHappeningOnDate cyc:Now)
• N9 (texai:?BookCopy-4 texai:fcgStatus texai:SingleObject)
• N10 (texai:?BookCopy-4 rdf:type cyc:BookCopy)
• N11 (texai:Book-0 rdf:type cyc:BookCopy)
• N12 (texai:?Table-5 rdf:type cyc:Table)
• N13 (texai:?OperationalDevice-7 rdf:type texai:OperationalDevice)
• N14 (texai:?Situation-ObjHasAttribute-9 rdf:type texai:Situation-ObjHasAttribute)
• N15 (cyc:Table cyc:conceptuallyRelated cyc:RoomInAConstruction)
• N16 (texai:?Table-8 rdf:type texai:PreviouslyIntroducedThingInThisDiscourse)
• N17 (texai:?BookCopy-4 rdf:type texai:PreviouslyIntroducedThingInThisDiscourse)
• N18 (texai:?Table-8 texai:fcgStatus texai:SingleObject)
• N19 (texai:?Table-5 texai:fcgDiscourseRole texai:external)
• N20 (texai:?Situation-ObjHasAttribute-9 cyc:situationConstituents texai:?BookCopy-4)
• N21 (texai:?Table-5 rdf:type texai:PreviouslyIntroducedThingInThisDiscourse)
• N22 (texai:?Situation-Localized-6 rdf:type cyc:Situation-Localized)
• N23 (texai:?Table-5 texai:fcgStatus texai:SingleObject)
• N24 (texai:?BookCopy-4 rdf:type texai:FCGClauseSubject)

Figure 7. The propositions from the alternative parsing interpretations in “the book is on the table“.

 

Figure 8. The Kintsch spreading activation graph to resolve alternative object referring expressions. See Dr. Jerry Ball’s document for a description of the Double R Grammar. Click on the image for a full-sized version.

Here is the parsing interpretation tree after removing the less-activated interpretations of object referring expressions. It still has two alternative interpretations for the phrase “on the table”.

Figure 9. The parsing interpretation tree

In figure 10 below are the 25 propositions from the remaining two alternative parsing interpretations. Figure 11 is the corresponding graph.

• N1 (texai:?Negotiating-14 rdf:type cyc:Negotiating)
• N2 (texai:?BookCopy-11 rdf:type cyc:BookCopy)
• N3 (texai:Texai cyc:negotiators texai:?Negotiating-14)
• N4 (texai:?Situation-ObjHasAttribute-16 cyc:situationConstituents texai:?BookCopy-11)
• N5 (texai:?Table-10 texai:fcgStatus texai:SingleObject)
• N6 (texai:?Table-10 rdf:type texai:PreviouslyIntroducedThingInThisDiscourse)
• N7 (texai:?BookCopy-11 texai:fcgStatus texai:SingleObject)
• N8 (texai:?On-SituationLocalized-12 texai:aboveObject texai:?BookCopy-11)
• N9 (texai:?Situation-Localized-13 cyc:situationConstituents texai:?BookCopy-11)
• N10 (texai:?On-SituationLocalized-12 texai:belowObject texai:?Table-10)
• N11 (texai:?On-SituationLocalized-12 rdf:type texai:On-SituationLocalized)
• N12 (texai:?Situation-ObjHasAttribute-16 rdf:type texai:Situation-ObjHasAttribute)
• N13 (texai:?Table-10 rdf:type cyc:Table)
• N14 (texai:Room-0 rdf:type cyc:RoomInAConstruction)
• N15 (texai:?BookCopy-11 rdf:type texai:FCGClauseSubject)
• N16 (texai:?BookCopy-11 texai:subjectOfNegotiation texai:?Negotiating-14)
• N17 (texai:Book-0 rdf:type cyc:BookCopy)
• N18 (texai:?Table-10 texai:fcgDiscourseRole texai:external)
• N19 (texai:?Situation-Localized-13 rdf:type cyc:Situation-Localized)
• N20 (cyc:TheUser cyc:negotiators texai:?Negotiating-14)
• N21 (texai:?Situation-Localized-13 texai:situationHappeningOnDate cyc:Now)
• N22 (texai:?BookCopy-11 rdf:type texai:NegotiableObject)
• N23 (texai:?BookCopy-11 rdf:type texai:PreviouslyIntroducedThingInThisDiscourse)
• N24 (texai:?BookCopy-11 texai:fcgDiscourseRole texai:external)
• N25 (texai:?Situation-ObjHasAttribute-16 texai:situationHappeningOnDate cyc:Now)

Figure 10. The propositions from the two remaining alternative parsing interpretations in “the book is on the table“.

 

Figure 11. The Kintsch spreading activation graph. Click on the image for a full-sized image.

After the final application of Kintsch spreading activation, only a single interpretation remains in the tree:

Figure 12. The final parsing interpretation tree for the utterance: “the book is on the table“.

Here is final interpretation of the utterance “the book is on the table” depicted as a syntax tree according to the conventions of Double R Grammar:

Figure 13. The final syntax tree for “the book is on the table“. Click on the image for a full-sized version.