StreamTokenizer Class Reference

Can tokenize a stream. More...

#include <StreamTokenizer.h>

List of all members.

Public Member Functions
	StreamTokenizer (istream &is, const string delim=" \t")
	The constructor.
string	next_token ()
	Get the next token.
bool	has_token ()
	Ask if the stream has other tokens.
Private Types
typedef istream_iterator< char >	sIt
	A stream iterator typedef.
Private Member Functions
bool	is_delimiter (char c)
	Checks if the given char is a delimiter.
Private Attributes
sIt	p
	The current position in the stream.
sIt	end_of_stream
	The end of the stream.
const string	delimiters
	The delimiters.

---

Detailed Description

Can tokenize a stream.

It takes a stream, and defined the delimiters, it returns the tokens between these delimiters.

Date:

16/11/2005

Definition at line 44 of file StreamTokenizer.h.

---

Constructor & Destructor Documentation

StreamTokenizer (  istream &  is, const string  delim = " \t" )  [inline]

The constructor. It defines the delimiters, and use the normal white-spaces on the stream. Parameters: is  The input stream. delim  The delimiters. Date: 16/11/2005 Definition at line 57 of file StreamTokenizer.h.

---

Member Function Documentation

bool has_token (   )  [inline]

Ask if the stream has other tokens. Returns: True if it is at the end of the stream, false otherwise. Date: 16/11/2005 Bug: In fact this method does not know if there are other tokens awaiting. If there are characters after the last delimiter at the end of the stream, they will be considered a token and therefore, this method will return true. This happens also if there is just a newline (in the case it is not a delimiter). It is possible to avoid this problem, but will affect efficency. Because a "spurious" token will just be ignored by the parser I decided to just leave it as it is. Definition at line 107 of file StreamTokenizer.h. References StreamTokenizer::end_of_stream, and StreamTokenizer::p. Referenced by UnSerialization::seekAndParseEnclosedValue(), UnSerialization::seekAndParseVectorTag(), and UnSerialization::seekSimpleTag().

bool is_delimiter (  char  c  )  [inline, private]

Checks if the given char is a delimiter. Parameters: c  The char to be checked Returns: True if c is a delimiter, false otherwise. Date: 16/11/2005 Definition at line 123 of file StreamTokenizer.h. References StreamTokenizer::delimiters. Referenced by StreamTokenizer::next_token().

string next_token (   )  [inline]

Get the next token. Returns: The string of the next token in the list. Date: 16/11/2005 Definition at line 66 of file StreamTokenizer.h. References StreamTokenizer::end_of_stream, StreamTokenizer::is_delimiter(), and StreamTokenizer::p. Referenced by UnSerialization::seekAndParseEnclosedValue(), UnSerialization::seekAndParseParamTag(), UnSerialization::seekAndParseVectorTag(), and UnSerialization::seekSimpleTag().

---

The documentation for this class was generated from the following file:

• src/Utils/StreamTokenizer.h

---