Roadmap
The roadmap for the Texai project is guided by these principles, which are viewed from a different perspective in this post about the challenge ahead.
- We must incrementally build up the capabilities of the system, having an operational system each step of the way to ensure that components and their interfaces are valid
- At each step of developement the intelligent system is operated with real sensing and action
- Components are developed in an order such that later components use the facilities provided by earlier components
- Bootstrap development is entirely in Java
- Existing Java libraries are preferred, however an existing application can be re-written from scratch as a less favored alternative
- Each component is based upon a technology which is known to work, so that the project risk for AI-hard tasks is reduced to the component combination and elaboration
- All facts and rules are stored in the knowledge base (KB). The Semantic Annotation for Persistence component: RDF Entiy Manager, facilitates storing any Java object as RDF statements in the Sesame RDF store.
- During the Bootstrap Stage, each component should be written or integrated by me
- Each component should be released in a manner that suits its incorporation into other projects
This project’s roadmap has as its goal the creation of artificial intelligence. Thus it is expected that many of the component combinations or component elaborations will be AI-hard problems (i.e. a problem which is beyond the ability of conventional software, but which a human can solve). Each such component combination/elaboration will be based upon my assessment of the best approach as evidenced by decades of research.
In brief, the approach will be to first construct an English dialog system whose initial goals are to acquire linguistic and common sense skills to improve its own performance. Next the system will acquire expertise in algorithms, and in Java programming for the purpose of explicitly representing its own behavior in the KB. Thus it will understand, revise, test and automatically compose its own source code. In parallel at this point, the system will acquire lexical and common sense knowledge from the glosses (word sense definitions) in the Texai lexicon, and begin to covert Wikipedia English text into KB statements, fleshing out the OpenCyc terms. In addition to scaling to many disparate users via Jabber chat from a single Texai instance, the system will be deployed as a virtual appliance to compute clusters and to a multitude of Internet users, where each instance hosts one or more nodes organized within an Albus Hierarchical Control System. These Albus Nodes (i.e. agents) will be organized into agencies, many mirroring current human organizations in which a node is a user’s proxy into the Albus HCS for some role. The artificial intelligence will then consist of a vast community of organizations whose members are Albus nodes, each quite intelligent with regard to its agency’s mission.
Bootstrap Stage
Bootstrap in the context of the Texai dialog system means that I, as sole contributor, Java programmer, and linguistics technician, create a tool whereby a great number of people can subsequently voluntarily contribute without being Java programmers or linguists. The Texai Bootstrap Stage is the process and set of milestones required to create such a tool. During this stage I will manually perform the development life-cycle for the essential dialog Java programs. Likewise I will manually compose the essential grammar constructions. By essential, I mean just those minimal features that permit the dialog tool to acquire new knowledge, new behaviors (i.e. scripts that compile to Java code), and new grammar constructions from a multitude of contributors. This post describes the dialog system architecture.
- Create an initial RDF knowledge base derived from OpenCyc (done September/2006)
- Incorporate a scalable RDF store (done September 2006, revised to using Sesame 2 August 2007)
- Create a facility to persist Java objects into an RDF store (released RDF Entity Manager, done December 2006)
- Create a Jabber chat interface (done November 2006 using the Smack API)
- Import the WordNet ontology into the Texai KB Lexicon and release as an RDF file (done September 2006, released September 2007)
- Import the CMU Pronouncing Dictionary into the Texai KB Lexicon and release as an RDF file (done November 2006, released September 2007)
- Import the Wiktionary into the Texai KB Lexicon and release as an RDF file (done November 2006, released September 2007)
- Integrate the Sphinx-4 automatic speech recognition and the FreeTTS speech synthesis engine (done January 2007, not yet released)
- Create a prototype language model plug-in for Sphinx-4 that intercepts calls to the N-gram model when evaluating the n-best list (done January 2007, not yet released)
- Create a Java version of the Fluid Construction Grammar engine (released November 2006, the original Lisp implemention provided via collaboration with Joachim De Beule at the Artificial Intelligence Lab, Free University of Bussels)
- Create an incremental Java-based implementation of the Fluid Construction Grammar engine (done, and released January, 2008)
- Adapt grammar construction rules from Double R Grammar (done, and released January, 2008)
- Incorporate the Kintsch Construction/Integration technique for pruning interpretations during parsing (done, April 2008)
- Implement Capability Description Language to support skill acquisition, and implement a task-to-capability matcher to support skill performance (in-progress as of May 2008)
- Create a controlled English vocabulary and grammar rules for the purpose of acquiring more vocabulary and grammar rules
- Create a knowledge-based intelligent dialog system that maintains a model of each user’s belief state, and which can guide users to communicate effectively using controlled English
- Create the ability to answer simple questions that do not involve deep deductive inference, nor induction, nor abduction
- Deploy a chat based system to acquire the lexical knowledge for a broad coverage of English with special emphasis on the ability to read (and to fully understand) the glosses in its own lexicon [AI-hard]
- Begin to acquire algorithmic knowledge, not yet including detailed Java programming knowledge, but sufficient that new skills can be acquired via teaching [AI-hard]
Mass Contribution Stage
At this point the system will perform mixed-initiative dialog, and entirely self-directed activities. The system is to be organized as an Albus Hierarchical Control System (AHCS). One can think of an AHCS as a model of a large human organization, in which the humans are the intelligent agents, and the corporate or military structure is the network.
- Engage a multitude of users to flesh out the commonsense OpenCyc-derived ontology. Collaboration with the Cyc Foundation
- Create a Knowledge Dashboard, which communicates the topic of discourse, meta knowledge, knowledge trails, and related info. Collaboration with Stu Rogers at AGS TechNet
- Enable Texai to acquire the domain knowledge and skills to understand, revise, test and to compose its own Java programs, or more efficient programs in machine language [AI-hard]
- Deploy the system on a virtual appliance that automatically connects itself into the Albus Hierarchical Control System
- Develop a robust AHCS architecture that can scale to millions of nodes
- Create an agency (i.e. set of Albus nodes) to oversee the creation of other useful agencies
- Enable the system to seek out and to motivate human mentors for various agencies
- Have Texai fully understand a substantial variety of the sentences contained in the English Wikipedia, and begin to read it [AI-hard]
- Give Texai the ability to compose article-length English compositions from its own knowledge [AI-hard]
- Have Texai provide an International Knowledge Infrastructure with human-usable and machine APIs
- Provide compelling benefits that motivate users to download a Texai instance for every capable computer and to use all spare cycles
- Among commonsense knowledge acquired, have an emphasis on ethics and friendship theory, be able to model adversaries, and prevent its knowledge corruption by malicious users [AI-hard]
- Configure Albus Nodes to control robots, using their physical sensors and actuators - single robots and coordinated teams
- Using mentors, incorporate AI algorithms for the following [AI-hard]
- bayesian decision making
- deductive planning
- symbol grounding from sensed percepts
- pattern recognition & machine learning
- induction of rules from observed instances
- abduction
- analogy
- prediction and estimation
- reinforcement learning
- markov decision processes
- salient associations (i.e. what is relevant in a context)
- problem solving heuristics
- robot control, physical sensors and actuators
- machine vision, object recognition
- swarm intelligence
- fuzzy logic
- genetic programming
- neural networks
Takeoff Stage
- Develop great expertise in recursive self-improvement, towards the goal of maximal Friendliness
- Achieve the Singularity, delivering benefits beyond what we can now project
Dennis Gorelik on 02 Jan 2008 at 12:27 pm #
How would you implement “Goal of maximum friendliness”?
In particular, how would you implement criteria that “judge” if goal was [partially] accomplished or not?
Steve Reed on 02 Jan 2008 at 9:57 pm #
Dennis, I will follow Eliezer Yudkowsky’s CFAI guidelines in the context of an Albus Hierarchical Control Network. Not only will the top level Albus Node have Maximum Friendliness as its sole top-level goal, each subordinate Node’s behavior policy will be constrained to preclude unfriendly behavior. In the Albus Reference Architecture, each Node has a component for Value Judgement, and one of this component’s responsibilities is to match current state against desired goal state(s).
For the goal of Maximum Friendliness, I see two components: (1) improving friendly behavior, and (2) preventing unfriendly behavior - with corresponding expert systems. I make a strong distinction between these because the latter should involve adversery modeling, devising ways to subvert the current protections, and so forth.
Judging whether a friendliness goal, or any goal for that matter, is (partially) accomplished or not, is addressed by Yudkowsky. Features of the both the goal and perceived states are probabilistic. The system matches states to goals according to its learned Bayesian experiences.
Steve Reed on 04 Jan 2008 at 5:18 am #
[Email comment from Stan Nilsen posted here with his permission.]
I admire your work. From reading your year in review 2007, I gather that you have great ability to get things done, and make wise choices. This is not meant as “My assessment of Steve Reed,” but as a preface to my comments. As additional “preface,” let me assure you that I have considerable “catch up” to do before I can appreciate what you’ve accomplished. That said…
Comment on Roadmap
The bootstrap section doesn’t seem to address the issue of bootstrap. Or perhaps I didn’t pick up the connection. I understand the natural language emphasis on acquiring a suitable KB, and, I see a KB as a
prerequisite to going into a bootstrap. What I don’t see in the section is a method to bootstrap.
Steps 16 and 17 hint at what I think of as bootstrap, but don’t go far
enough. What I find missing is emphasis on self direction. At what point does one move from having a big vocabulary to directing resources? especially one’s own resources.
To me bootstrap describes the minimal mechanism that will bring about a viable artificial intelligence. To put it in biological terms, there is a “feeding” aspect of life and then the assimilation, or production of proteins and energy from the food. As this KB grows, when will it begin to produce, that is make choices as to what it does next?
Until the unit can choose a way, it isn’t going to qualify as intelligent - although the system could still have great utility in many applications.
I offer an “example” of what I see as bootstrap - Imagine that the “chat” capability is implemented. To move from chat bot to intelligent chat goes something like this:
Chat has a goal - exchange text strings with humans for long periods of time.
Chat has a theory, or method of how it can optimize this -
1. use strings that include words humans find interesting (from a list of this category)
2. use the words in a sentence structure
3. create sentences that often take the form of questions
4. evaluate the session based on volume of exchanged material and length of the session.
5. evaluate the content of the exchanged material to see if new words could be added to the list of “words humans find interesting”
6. present the results to the email list of those interested in such results (another list)
7. accept suggestions from the “list of those interested” and modify such parameters as “list of words” and frequency of generation of questions.
And so forth …
In the process of “doing” these activities, the system expands it’s set of choices, and basis for making decisions.
If nothing else, this bootstrap could produce optimal rules for engagement of humans - to eventually lead to better understanding of general life e.g. what’s important to people and why…
Summary - My thought is that there is Architecture and there is Bootstrap - the two being different. My two cents..
Stan Nilsen
Steve Reed on 04 Jan 2008 at 5:19 am #
Stan,
Here are my remarks to your points, and I will enhance the Roadmap accordingly.
[The bootstrap section doesn’t seem to address the issue of bootstrap.]
Bootstrap in the context of the Texai dialog system means that I, as sole contributor, Java programmer and linguist, create a tool whereby a great number of people can subsequently contribute without being Java programmers or linguists. The Texai Bootstrap Stage is the process and milestones required to create such a tool. During this stage I will manually perform the development life-cycle for the essential dialog Java programs. Likewise I will manually compose the essential grammar constructions. By essential, I mean just those features that permit the dialog tool to acquire new behaviors (i.e. scripts that compile to Java code), and new grammar constructions from a multitude of contributors.
[What I find missing is emphasis on self direction. At what point does one move from having a big vocabulary to directing resources? especially ones own resources.]
Texai will perform self-direction in the Mass Contribution Stage (i.e. post bootstrap). The system at that point will be organized as an Albus Hierarchical Control Network (AHCN). One can think of an AHCN as a model of a large human organization, in which the humans are the intelligent agents, and the corporate or military structure is the network.
-Steve
David Clark on 16 Feb 2008 at 3:10 am #
Your list of todo projects is truly amazing.
I would first like to wish you all the best with your project but my approach to AGI differs fundamentally from yours.
I can probably best describe my views by talking about a set of points that when plotted describe a line. You could have thousands of data point pairs but if you were asked for an X given some random Y, it would be unlikely you could return a correct answer. I see this as the way you are trying to put a lot of common sense statements together and when you get enough, you somehow get intelligence. My alternative is to use a slope and the Y intercept instead of thousands of points. You only need 2 numbers and you can accurately answer any X given any Y or vice versa. The problem of course is that we don’t always know the algorithm and sometimes the actual formula for the function is hard to find, even if given a large set of data. I understand that but it seems to me, that if you strive for the algorithm then at least you know the relationship that creates the output where otherwise you wouldn’t.
I understand that not all forms of intelligence or methods of problem solving can be had with a single method. Therefore my solution is to use models where the algorithm that does the “thinking” can be any type of model.
I think starting your project by creating a minimally chatting bot is a good approach and I am doing this also. Even though I have the Wordnet ontology from CMU, I don’t think it will be of much use. It doesn’t seem very hard to get words or definitions of words but it does seem hard to get meaning out of these words. The meanings of the words or the “intelligence” in your system seems to come from the intelligence of the humans that encoded the statements, they say are true. I have created programs that some people have called “smart” (not in the sense of AGI smart but in the program sense of smart) but it wasn’t the program at all they were seeing but a little of me. It was my thinking ahead and providing them with smart defaults and information they didn’t know they even wanted. The intelligence was still 100% just me and it was a very small fraction of me at that. It seems that creating a huge database of these “facts” or true statements seems to be a way of trying to harvest this encoded human intelligence but I think in the end, the machine itself will only see letters and numbers and will not get to that elusive intelligent state.
I see words and their meanings coming from a huge number of models, big and small, that in the end execute programs that encode the meaning of the words and ideas. These models could incorporate pictures and simple action sequences that could be used by the AGI to test meanings of the words. In this sense meaning could be explored as needed by the AGI.
My idea is that language and it’s meaning (semantics) are intimately intertwined and having a database of text doesn’t do all that much toward creating the intelligence we are all looking for. I’m not saying that large amounts of data isn’t useful but I am saying that it is only useful in the sense that databases are useful to humans. They aren’t the source of our intelligence.
Please don’t take the above comments as a swipe against your project because I have great respect for all the people who are actually doing the AGI, rather than just talking about it.
Lots of luck
David Clark