org.nineml.coffeegrinder.tokens

Class Token

java.lang.Object

org.nineml.coffeegrinder.util.Decoratable

org.nineml.coffeegrinder.tokens.Token

Direct Known Subclasses:

TokenCharacter, TokenCharacterSet, TokenEOF, TokenEPSILON, TokenRegex, TokenString

public abstract class Token
extends Decoratable

An input token.

This is an abstraction for input tokens. It allows the parser to be used, for example, for both sequences of characters and sequences of strings. (Sequences of anything you like, provided you define the tokens.)

The only thing that's important about tokens is that we can tell when they match each other. This is not the same as equality becuase, for example, the same regular expression token might match many different input strings.

Constructor Summary

Constructors

Constructor and Description
Token(java.util.Collection<ParserAttribute> attributes) A token with attributes.

Method Summary

Modifier and Type	Method and Description
abstract java.lang.String	getValue() What is this token?
abstract boolean	matches(Token input) Does this token match the input?

Methods inherited from class org.nineml.coffeegrinder.util.Decoratable

addAttribute, addAttributes, getAttribute, getAttributes, getAttributesMap, getAttributeValue, hasAttribute

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Token

public Token(java.util.Collection<ParserAttribute> attributes)

A token with attributes.

Parameters:

attributes - the attributes

Throws:

GrammarException - if the attribute names are not unique

AttributeException - if an attribute has an invalid value

Method Detail

matches

public abstract boolean matches(Token input)

Does this token match the input?

Parameters:

input - The input.

Returns:

true if this token matches that input.

getValue

public abstract java.lang.String getValue()

What is this token?

Returns:

the token string