TokenizerByChar

java.lang.Object

com.topologi.diffx.load.text.TokenizerByChar

All Implemented Interfaces:: TextTokenizer

public final class TokenizerByChar extends Object implements TextTokenizer

The tokeniser for characters events.

This class is not synchronized.

Version:: 10 May 2010

Field Summary

Fields

Modifier and Type

Field

Description

private final Map<Character,TextEvent>

recycling

Map characters to events in order to recycle events as they are created.
Constructor Summary

Constructors

Constructor

Description

TokenizerByChar()

Creates a new tokenizer.
Method Summary

Modifier and Type

Method

Description

TextGranularity

granurality()

Always TextGranularity.CHARACTER.

List<TextEvent>

tokenize(CharSequence seq)

Returns the list of TextEvent corresponding to the specified character sequence.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details
- recycling
  
  private final Map<Character,TextEvent> recycling
  
  Map characters to events in order to recycle events as they are created.
Constructor Details
- TokenizerByChar
  
  public TokenizerByChar()
  
  Creates a new tokenizer.
Method Details
- tokenize
  
  public List<TextEvent> tokenize(CharSequence seq)
  
  Returns the list of TextEvent corresponding to the specified character sequence.
  
  Specified by:
  
  tokenize in interface TextTokenizer
  
  Parameters:
  
  seq - the character sequence to tokenize.
  
  Returns:
  
  the corresponding list.
- granurality
  
  public TextGranularity granurality()
  
  Always TextGranularity.CHARACTER. Returns the text granularity of this tokenizer.
  
  Specified by:
  
  granurality in interface TextTokenizer
  
  Returns:
  
  the text granularity of this tokenizer.