uk.ac.sheffield.jast

Class BasicScanner

java.lang.Object

uk.ac.sheffield.jast.Logging

uk.ac.sheffield.jast.BasicScanner

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

Direct Known Subclasses:

DTDScanner, XMLScanner

public class BasicScanner
extends Logging
implements java.io.Closeable

BasicScanner is the ancestor of various scanners for XML or DTD tokens. BasicScanner implements basic scanning methods, which are inherited by its subclasses XMLScanner and DTDScanner. Subclasses must redefine scanAnyContent() according to the expected token grammar. BasicScanner provides the common methods for scanning name tokens, key words, quoted strings and XML entity references, which are inherited by all of its descendants.

BasicScanner is expected to buffer between 1-20 tokens on each call to getToken(), which allows parsers to work with the token stream during read-cycles from the underlying input stream. BasicScanner also implements Scanner's logging interface, so that the state of the underlying stream can be queried by pull-parsers using this BasicScanner. BasicScanner implements the Closeable interface, so may be used in try-with-resources blocks (since JDK 1.7).

Version:

2.5

Author:

Anthony J H Simons

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.String	encoding The name of the character set used when decoding characters from bytes.
protected java.io.LineNumberReader	input The input stream used by this BasicScanner.
private boolean	isStandardInput Secret flag to determine whether the wrapped stream is System.in, used to block closing the standard input stream.
protected int	lastChar The lookahead character.
protected java.util.Deque<java.lang.String>	symbolStack The stack of where we are in the parse of tokens.
protected java.util.Deque<java.lang.String>	textQueue The queue of tokenised text.
protected java.util.Deque<java.lang.Integer>	tokenQueue The queue of symbolic tokens.

Constructor Summary

Constructors

Constructor and Description
BasicScanner(java.io.File file, java.lang.String encoding) Creates a BasicScanner reading from a file, using a specified character encoding.
BasicScanner(java.io.InputStream stream, java.lang.String encoding) Creates a BasicScanner reading from a basic InputStream, using a specified character encoding.
BasicScanner(java.io.Reader reader, java.lang.String encoding) Creates a BasicScanner reading from a character Reader, using a specified character encoding.
BasicScanner(java.net.URL url, java.lang.String encoding) Creates a BasicScanner reading from a URL, using a specified character encoding.

Method Summary

Modifier and Type	Method and Description
void	close() Closes this BasicScanner.
boolean	endOfStream() Reports whether the end of the input stream has been reached.
java.lang.String	getContext() Returns the XML context in which a syntax error is detected.
java.lang.String	getEncoding() Reports the character encoding used by this BasicScanner.
int	getLineNumber() Returns the current line number reached by this BasicScanner.
java.lang.String	getText() Returns the first piece of segmented text at the head of the text queue.
int	getToken() Returns the first token at the head of the token queue.
protected void	scanAnyContent() Scans the input stream for arbitrary XML or DTD content, populating the token stream.
protected void	scanEntityReference() Parses an XML entity reference.
protected boolean	scanExactly(java.lang.String token) Checks that the input contains the exact token next on the stream.
protected void	scanKeyword() Scans a DTD keyword.
protected void	scanNameToken() Scans an XML name token.
protected void	scanQuotedValue() Scans a quoted value-string.
protected void	skipSpace() Skips whitespace characters.

Methods inherited from class uk.ac.sheffield.jast.Logging

encodingError, semanticError, syntaxError

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

isStandardInput

private boolean isStandardInput

Secret flag to determine whether the wrapped stream is System.in, used to block closing the standard input stream.

encoding

protected java.lang.String encoding

The name of the character set used when decoding characters from bytes. By default, we assume this is UTF-8, but clients can ask to use other character set encodings, such as ISO-8859-1 (Latin-1).

input

protected java.io.LineNumberReader input

The input stream used by this BasicScanner. The underlying stream is a LineNumberReader, which is created internally by wrapping a new LineNumberReader around some kind of Reader or InputStream.

tokenQueue

protected java.util.Deque<java.lang.Integer> tokenQueue

The queue of symbolic tokens. This uses an ArrayDeque implementation, which is most efficiently accessed through a Deque interface. The methods addLast(), removeFirst() and peekFirst() are the primary implementations, on which other methods are based.

textQueue

protected java.util.Deque<java.lang.String> textQueue

The queue of tokenised text. This uses an ArrayDeque implementation, which is most efficiently accessed through a Deque interface. The methods addLast(), removeFirst() and peekFirst() are the primary implementations, on which other methods are based.

symbolStack

protected java.util.Deque<java.lang.String> symbolStack

The stack of where we are in the parse of tokens. This is filled up with the names of elements, or other nodes, as we explore more nested XML structures.

lastChar

protected int lastChar

The lookahead character.

Constructor Detail

BasicScanner

public BasicScanner(java.io.File file,
                    java.lang.String encoding)
             throws java.io.FileNotFoundException,
                    java.io.UnsupportedEncodingException

Creates a BasicScanner reading from a file, using a specified character encoding. Recognised encodings include UTF-8, UTF-16, ISO-8859-1 (Latin-1) and others. Creates a LineNumberReader reading from an InputStreamReader with the given encoding, reading from a basic FileInputStream opened on the file.

Parameters:

file - the input file.

encoding - the name of the character encoding.

Throws:

java.io.FileNotFoundException - if the file cannot be found or opened.

java.io.UnsupportedEncodingException - if the character encoding is not supported.

BasicScanner

public BasicScanner(java.net.URL url,
                    java.lang.String encoding)
             throws java.io.IOException,
                    java.io.UnsupportedEncodingException

Creates a BasicScanner reading from a URL, using a specified character encoding. Recognised encodings include UTF-8, UTF-16, ISO-8859-1 (Latin-1) and others. Creates a LineNumberReader reading from an InputStreamReader with the given encoding, reading from a basic InputStream created by the URL's openStream() method.

Parameters:

url - the absolute URL giving the location of the XML data.

encoding - the name of the character encoding.

Throws:

java.io.IOException - if the attempt to connect to the URL and open an input stream to read from it fails.

java.io.UnsupportedEncodingException - if the character encoding is not supported.

BasicScanner

public BasicScanner(java.io.InputStream stream,
                    java.lang.String encoding)
             throws java.io.UnsupportedEncodingException

Creates a BasicScanner reading from a basic InputStream, using a specified character encoding. Recognised encodings include UTF-8, UTF-16, ISO-8859-1 (Latin-1) and others. Creates a LineNumberReader reading from an InputStreamReader with the given encoding, reading from the basic InputStream.

Parameters:

stream - the basic InputStream.

encoding - the name of the character encoding.

Throws:

java.io.UnsupportedEncodingException - if the character encoding is not supported.

BasicScanner

public BasicScanner(java.io.Reader reader,
                    java.lang.String encoding)

Creates a BasicScanner reading from a character Reader, using a specified character encoding. Recognised encodings include UTF-8, UTF-16, ISO-8859-1 (Latin-1) and others. The character encoding must match that used by the Reader, for consistency. Wraps the supplied Reader with a LineNumberReader, to track the location of possible XML syntax errors.

Parameters:

reader - the character Reader.

encoding - the name of the character encoding.

Method Detail

close

public void close()
           throws java.io.IOException

Closes this BasicScanner. Closes the underlying input stream, unless that stream is System.in, which is closed at program termination.

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Throws:

java.io.IOException - if an I/O error occurs during closing.

getEncoding

public java.lang.String getEncoding()

Reports the character encoding used by this BasicScanner. This is the character encoding that was supplied at construction.

Returns:

the character encoding used by this BasicScanner.

getLineNumber

public int getLineNumber()

Returns the current line number reached by this BasicScanner. Backs up to the previous line, if the last character read was a newline, to aid in locating errors. Line numbering starts at line 1. Errors reported by this scanner are reported at the current line, but errors reported by clients ... DTDReader, or ASTReader are reported after the current line has been consumed.

Specified by:

getLineNumber in class Logging

Returns:

the current line number.

getContext

public java.lang.String getContext()

Returns the XML context in which a syntax error is detected. This is in the form of a path leading to the current node in which the syntax error was detected, in the style of XPath.

Specified by:

getContext in class Logging

Returns:

the context, in the style of XPath.

endOfStream

public boolean endOfStream()

Reports whether the end of the input stream has been reached.

Specified by:

endOfStream in class Logging

Returns:

true, if the end of the stream has been reached.

getToken

public int getToken()
             throws java.io.UnsupportedEncodingException,
                    java.io.IOException,
                    SyntaxError

Returns the first token at the head of the token queue. This method is called repeatedly by pull-parsers to return the next symbolic token. Buffers up to twenty new tokens each time the queue is found to be empty. This strategy allows the parser to work during the latency of the underlying input stream, increasing efficiency.

Returns:

the next integer token.

Throws:

java.io.UnsupportedEncodingException - if a decoding fault occurs.

java.io.IOException - if an input stream error is detected.

SyntaxError - if an XML syntax error is detected.

getText

public java.lang.String getText()

Returns the first piece of segmented text at the head of the text queue. This could be a name token, or a block of textual content. The text queue is accessed far less frequently than the token queue, and only when the corresponding token is pulled by a parser.

Returns:

the next piece of segmented text.

scanAnyContent

protected void scanAnyContent()
                       throws java.io.UnsupportedEncodingException,
                              java.io.IOException,
                              SyntaxError

Scans the input stream for arbitrary XML or DTD content, populating the token stream. Scans one complete XML or DTD artifact, before returning control to the caller. This default method does nothing and should be redefined in each Scanner subclass to look for the expected kinds of structure in the scanned input stream.

Throws:

java.io.UnsupportedEncodingException - if a decoding fault occurs.

java.io.IOException - if an input stream error is detected.

SyntaxError - if a syntax error is detected.

scanNameToken

protected void scanNameToken()
                      throws java.io.IOException,
                             SyntaxError

Scans an XML name token. A name token is a sequence of characters starting with a letter or underscore, and continuing with any mix of letter, digit, hyphen, underscore or period characters. A name token may also contain a namespace prefix, separated from the main name by a colon. A name token is parsed by accumulating legal constituent characters until a non-legal character is encountered. If this is a legitimate terminator, the name token is returned; otherwise a syntax error is reported. Queues a NAME_TOKEN.

Throws:

java.io.IOException - if an input stream error is detected.

SyntaxError - if the name token is malformed.

scanKeyword

protected void scanKeyword()
                    throws java.io.IOException,
                           SyntaxError

Scans a DTD keyword. A keyword is more restricted than a name token, consisting of optional '#' and then only upper case alpha-characters. Some examples include the PUBLIC or SYSTEM identifiers used in a Doctype declaration, or the #PCDATA indicator of parsed character data. A plain keyword is parsed by accumulating legal characters until a non-legal character is encountered. Queues a KEY_WORD token.

Throws:

java.io.IOException - if an input stream error is detected.

SyntaxError - if the plain token is malformed.

scanEntityReference

protected void scanEntityReference()
                            throws java.io.IOException

Parses an XML entity reference. This is a special escape-sequence starting with '&' and ending with ';'. Scans forward until either a proper terminator ';', or a space or EOF is encountered. Queues an ENTITY_REF.

Throws:

java.io.IOException - if an input stream error is detected.

java.io.EOFException - if premature end of input is detected.

SyntaxError - if an XML syntax error is detected.

scanQuotedValue

protected void scanQuotedValue()
                        throws java.io.IOException,
                               SyntaxError

Scans a quoted value-string. A value-string must be enclosed in a matching pair of either single-quotes or double-quotes, and may not contain markup characters, unless these are escaped as entities. The character '<' may not appear in the value-string. Quotes of the non-delimiter kind may be included in the value-string. If the value-string contains escaped character data, this is scanned as a separate token. Queues an OPEN_QUOTE, followed by one or more occurrences of PRINT_TEXT and ENTITY_REF, followed by CLOSE_QUOTE.

Throws:

java.io.IOException - if an input stream error is detected.

SyntaxError - if an XML syntax error is detected.

scanExactly

protected boolean scanExactly(java.lang.String token)
                       throws java.io.IOException

Checks that the input contains the exact token next on the stream. After checking the token, sets the next lookahead character. This method assumes that all tokens to be checked are low-end characters in Unicode (like 'DOCTYPE' or 'CDATA').

Parameters:

token - the token to be matched.

Returns:

true if the input contains the exact next token.

Throws:

java.io.IOException - if an input stream error is detected.

skipSpace

protected void skipSpace()
                  throws java.io.IOException

Skips whitespace characters. Advances the read-point to the next non-whitespace character.

Throws:

java.io.IOException - if an input stream error is detected.