org.eclipse.xtext.parser.antlr
Class XtextTokenStream
java.lang.Object
org.antlr.runtime.CommonTokenStream
org.eclipse.xtext.parser.antlr.XtextTokenStream
- All Implemented Interfaces:
- org.antlr.runtime.IntStream, org.antlr.runtime.TokenStream
- Direct Known Subclasses:
- ObservableXtextTokenStream
public class XtextTokenStream
- extends org.antlr.runtime.CommonTokenStream
A token stream that is aware of the current lookahead.
- Author:
- Jan Köhnlein - Initial contribution and API, Sebastian Zarnekow - Support for dynamic hidden tokens,
reworked lookahead algorithm
Fields inherited from class org.antlr.runtime.CommonTokenStream |
channel, channelOverrideMap, discardOffChannelTokens, discardSet, lastMarker, p, tokens, tokenSource |
Methods inherited from class org.antlr.runtime.CommonTokenStream |
discardOffChannelTokens, discardTokenType, fillBuffer, get, getSourceName, getTokens, getTokens, getTokens, getTokens, getTokens, getTokenSource, index, LB, release, reset, rewind, rewind, setTokenSource, setTokenTypeChannel, size, skipOffTokenChannels, skipOffTokenChannelsReverse, toString, toString |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
XtextTokenStream
public XtextTokenStream()
XtextTokenStream
public XtextTokenStream(org.antlr.runtime.TokenSource tokenSource,
int channel)
XtextTokenStream
public XtextTokenStream(org.antlr.runtime.TokenSource tokenSource,
ITokenDefProvider tokenDefProvider)
toString
public java.lang.String toString(int start,
int stop)
- Specified by:
toString
in interface org.antlr.runtime.TokenStream
- Overrides:
toString
in class org.antlr.runtime.CommonTokenStream
setHiddenTokens
public XtextTokenStream.HiddenTokens setHiddenTokens(java.lang.String... lexerRules)
setInitialHiddenTokens
public void setInitialHiddenTokens(java.lang.String... lexerRules)
doSetHiddenTokens
protected void doSetHiddenTokens(java.lang.String... lexerRules)
skipHiddenTokens
protected void skipHiddenTokens()
LA
public int LA(int i)
- Specified by:
LA
in interface org.antlr.runtime.IntStream
- Overrides:
LA
in class org.antlr.runtime.CommonTokenStream
consume
public void consume()
- Specified by:
consume
in interface org.antlr.runtime.IntStream
- Overrides:
consume
in class org.antlr.runtime.CommonTokenStream
getFirstMarker
protected int getFirstMarker()
- Since:
- 2.1
mark
public int mark()
- Specified by:
mark
in interface org.antlr.runtime.IntStream
- Overrides:
mark
in class org.antlr.runtime.CommonTokenStream
seek
public void seek(int index)
- Specified by:
seek
in interface org.antlr.runtime.IntStream
- Overrides:
seek
in class org.antlr.runtime.CommonTokenStream
LT
public org.antlr.runtime.Token LT(int k)
- Same as
CommonTokenStream.LT(int)
except that we skip
hidden tokens even for k == 1.
- Specified by:
LT
in interface org.antlr.runtime.TokenStream
- Overrides:
LT
in class org.antlr.runtime.CommonTokenStream
getCurrentLookAhead
public int getCurrentLookAhead()
initCurrentLookAhead
public void initCurrentLookAhead(int currentLookAhead)
getLexerErrorMessage
public java.lang.String getLexerErrorMessage(org.antlr.runtime.Token invalidToken)