The current state of play

One could argue that Human Machine Conversation (HMC) is now in a position like that of machine translation fifteen years ago: it is a real technology, coming into being in spite of scepticism, but with a huge gulf between busy practitioners getting on with the job, often within companies, and, on the other side, the researchers, in linguistics, artificial intelligence (AI) or whatever, whose papers fill the conferences. The rise of empirical linguistics has largely closed this gulf for machine translation and related arts, but not as yet, for HMC. First, let us set out historical trends in HMC, for if the HMC world is really as disparate as what follows it is hardly surprising there is so little consensus on how to progress.

1. Dialogue analysis based on models of individual agents' beliefs and knowledge structures, usually presented within an AI-Derived theory of plans, inference and possibly speech/dialogue acts and truth maintenance, many using a ``space" metaphor to represent individuals (e.g. Allen [8], Traum [9], Kobsa [10], Ballim and Wilks [11]).

2. AI-derived models of dialogue based on more linguistic notions and not primarily based on models of individuals; the representation is often in terms of partitioned semantic nets to represent domains but uses concepts of focus, failure, repair etc. (e.g. Grosz and Sidner [12], Webber [13]).

3. AI-derived models based largely on transitions in domain scripts, driving top-down inference but augmented by local inference rules representing quasi-plan etc., no models of individuals (e.g. Schank [14]).

4. Sociology/Ethnomethodology tradition of descriptive conversational analysis, usually based on local transition analysis types in actual dialogues, analysed non-statistically (e.g. Schegloff [15]).

5. Local AI theories of discourse, usually without models of individuals or domains, but with a taxonomy of speech/dialogue acts and inference rules applied bottom-up to utterances (Charniak [16], Bunt [17], Carletta [18]).

6. Transition network models of general discourse moves-apparently global but domain-independent scripts but largely dependent on local alternatives-a dialogue version of ``text grammar" and in some ways a normalization of (4) above (e.g. Whittaker and Stanton [19]).

7. Empirical analysis of dialogue corpora to produce statistical measure of dialogue turn transitions based on a taxonomy of dialogue actsùthe first empirical pragmatics? In some ways, the provision of evidence for forms of (6) and so (4). (e.g. Maier and Reithinger [20,21]).Pattern-matching approaches to bottom-up dialogue analysis, providing input to some higher representational form and rejecting the possibility of effective dialogue grammar (Colby [22], ATIS [23] and CONVERSE [24]).

8. Full treatment of empirical dialogue analysis, derived automatically from corpora, transducing from utterances to some form rich enough to support dialogue acts, whether in terms of some conventional grammatical representation or a fusion of taggers, lexical strings and pattern matcher outputs.

The themes and approaches in this list are probably not wholly independent and may not be exhaustive. Notice that, thirty years after PARRY, no real form of (9) exists, and the corpora from which it might be done (for English at least) have only very recently come into existence. An interesting question right now, is whether (9) can be done in a principled way, as an alternative to PARRY-like systems built up over long periods by hand, or many of the other types of systems above with trivial vocabularies and virtually no functionality. This was exactly the opposition in machine translation for many years: with SYSTRAN's large hand-crafted functionality contrasted with a host of theoretical, published, acclaimed but non-functional systems. In Machine Translation, that opposition began to collapse with the arrival of IBM's statistical MT system about 1990. The possibility of a meaningful empirical pragmatics could do the same for HMC.

One additional point should be made here: we have said nothing of computer recognition (and production) of speech in dialogue systems. Speech research has pursued its own agenda, separate from written text, and all the above systems communicated via screen typing. The chief speech problem was always decoding the signals into words, rather than the content of dialogue as we have described it above-researchers tended to assume that speech could be solved separately and then a dialogue model of one of the following types just bolted on, as it were. This agenda for research has had obvious defects, especially in that speech phenomena like pauses, stress, pitch etc. convey meaning as well-but basically there has been agreement on all sides until now to separate out the speech and language issues so as to progress.