text parser

Re: text parser

In the constructor of the tokeniser, you can speify if the delimiters become
tokens or not.
If all you want to do is split the string at each token, then use this
feature. If you are wanting the delimiter ragged onto the end of each token,
simply append it..

//Chop the string then put it back together!!!
StringBuffer buf = new StringBuffer();
StringTokenizer tok = new StringTokenizer ("My|Token|Stri ng","|",false );
while(tok.hasMo reTokens()){
buf.append(tok. nextToken()).ap pend("|");
}
System.out.prin tln(tok.toStrin g());

HTH

Steve


"Rene" <rene.vallant@g mx.at> wrote in message
news:c27f7ab7.0 310270401.5f6fb [email protected] gle.com...[color=blue]
> Hi,
>
> i'm a newbie at java and have a question for beginners.
> So i'm try to create a type of parser for a input-string. i take a
> StringTokenizer to cut the Sting into token. But i want, that the
> seperator belongs to the token and not to the next token. is there a
> chance to do this, or is there a better way to spilt a Strong?
>
> Thanks,
> Rene[/color]

Re: text parser

> If all you want to do is split the string at each token, then use this[color=blue]
> feature. If you are wanting the delimiter ragged onto the end of each token,
> simply append it..[/color]
[color=blue]
> StringTokenizer tok = new StringTokenizer ("My|Token|Stri ng","|",false );[/color]
^ (you
assume set of deleimeters to be just one character)
if you use StringTokenizer ("String goes","delims") , the flag is false
by default (i think so) check docs

if you have set of delimeters with multiple characters, this would not
work.
i suggest write your own logic for parsing (if you have multiple
characters as delimeters)

regards
amey

Re: text parser

[email protected] om (Amey Samant) wrote in message news:<669e50b8. 0310280437.3713 [email protected] ogle.com>...[color=blue][color=green]
> > If all you want to do is split the string at each token, then use this
> > feature. If you are wanting the delimiter ragged onto the end of each token,
> > simply append it..[/color]
>[color=green]
> > StringTokenizer tok = new StringTokenizer ("My|Token|Stri ng","|",false );[/color]
> ^ (you
> assume set of deleimeters to be just one character)
> if you use StringTokenizer ("String goes","delims") , the flag is false
> by default (i think so) check docs
>
> if you have set of delimeters with multiple characters, this would not
> work.
> i suggest write your own logic for parsing (if you have multiple
> characters as delimeters)
>
> regards
> amey[/color]

Hi,
thanks for your response!

I wanted to write my own logic of parsing, but i just abortive to
write my own logic! So i started to work with the Tokenizer. Maybe
some of you can give me a direction how i can do it. I wanted to make
a state machine, but as i already said i just faild.

Thanks,
Rene