StreamTokenizer Class Reference

Can tokenize a stream. More...

#include <StreamTokenizer.h>

List of all members.

Public Member Functions

 StreamTokenizer (istream &is, const string delim=" \t")
 The constructor.
string next_token ()
 Get the next token.
bool has_token ()
 Ask if the stream has other tokens.

Private Types

typedef istream_iterator<
char > 
sIt
 A stream iterator typedef.

Private Member Functions

bool is_delimiter (char c)
 Checks if the given char is a delimiter.

Private Attributes

sIt p
 The current position in the stream.
sIt end_of_stream
 The end of the stream.
const string delimiters
 The delimiters.


Detailed Description

Can tokenize a stream.

It takes a stream, and defined the delimiters, it returns the tokens between these delimiters.

Date:
16/11/2005

Definition at line 44 of file StreamTokenizer.h.


Constructor & Destructor Documentation

StreamTokenizer istream &  is,
const string  delim = " \t"
[inline]
 

The constructor.

It defines the delimiters, and use the normal white-spaces on the stream.

Parameters:
is The input stream.
delim The delimiters.
Date:
16/11/2005

Definition at line 57 of file StreamTokenizer.h.


Member Function Documentation

bool has_token  )  [inline]
 

Ask if the stream has other tokens.

Returns:
True if it is at the end of the stream, false otherwise.
Date:
16/11/2005
Bug:
In fact this method does not know if there are other tokens awaiting. If there are characters after the last delimiter at the end of the stream, they will be considered a token and therefore, this method will return true. This happens also if there is just a newline (in the case it is not a delimiter). It is possible to avoid this problem, but will affect efficency. Because a "spurious" token will just be ignored by the parser I decided to just leave it as it is.

Definition at line 107 of file StreamTokenizer.h.

References StreamTokenizer::end_of_stream, and StreamTokenizer::p.

Referenced by UnSerialization::seekAndParseEnclosedValue(), UnSerialization::seekAndParseVectorTag(), and UnSerialization::seekSimpleTag().

bool is_delimiter char  c  )  [inline, private]
 

Checks if the given char is a delimiter.

Parameters:
c The char to be checked
Returns:
True if c is a delimiter, false otherwise.
Date:
16/11/2005

Definition at line 123 of file StreamTokenizer.h.

References StreamTokenizer::delimiters.

Referenced by StreamTokenizer::next_token().

string next_token  )  [inline]
 

Get the next token.

Returns:
The string of the next token in the list.
Date:
16/11/2005

Definition at line 66 of file StreamTokenizer.h.

References StreamTokenizer::end_of_stream, StreamTokenizer::is_delimiter(), and StreamTokenizer::p.

Referenced by UnSerialization::seekAndParseEnclosedValue(), UnSerialization::seekAndParseParamTag(), UnSerialization::seekAndParseVectorTag(), and UnSerialization::seekSimpleTag().


The documentation for this class was generated from the following file:
Generated on Mon Nov 28 21:43:48 2005 for MultiBoost by  doxygen 1.4.5