StringTokenizer Class in Java

Java StringTokenizer class is useful when a string needs to be broken into smaller tokens based on delimiters. In this chapter, we will learn what the StringTokenizer class is, along with its constructors, methods, and usage through examples.

What is StringTokenizer class in Java?

The StringTokenizer class is used to break a string into tokens based on specified delimiters. It belongs to the java.util package and provides a simple way to split a string into smaller parts. The delimiters can be defined at the time of object creation or applied while retrieving tokens.

StringTokenizer is considered a legacy class, and it does not provide advanced features such as distinguishing between numbers, identifiers, or quoted strings. For more flexible and modern string processing, classes like String.split() or StreamTokenizer are generally preferred.

For example, consider the string "hello welcome to TpointTech". As shown in the image, the StringTokenizer breaks this string into separate tokens based on the specified delimiter (such as a space), resulting in individual words like hello, welcome, to, and TpointTech.

Constructors of StringTokenizer Class

There are three constructors defined in the StringTokenizer class, which are:

1. StringTokenizer(String str)

It creates a StringTokenizer for the specified string using the default delimiter (space, tab, newline, etc.).

Syntax:

It has the following syntax:

2. StringTokenizer(String str, String delim)

It creates a StringTokenizer for the specified string using the provided delimiter(s).

Syntax:

It has the following syntax:

3. StringTokenizer(String str, String delim, boolean returnDelims)

It creates a StringTokenizer for the specified string and delimiter(s). If returnDelims is true, the delimiters are also returned as tokens. If false, delimiters are only used to separate tokens.

Syntax:

It has the following syntax:

You can choose any of the constructors, how to split your string and whether to consider delimiters as tokens, depending on your parsing needs.

Example of StringTokenizer Constructors

The following example demonstrates all three constructors of the StringTokenizer class:

Output:

Tokens using default delimiter:
Hello
Welcome
To
TpointTech

Tokens using comma as delimiter:
Hello
Welcome
To
TpointTech

Tokens including delimiters:
Hello
,
Welcome
,
To
,
TpointTech

Explanation:

• The first constructor uses the default space delimiter.
• The second constructor uses a custom delimiter (comma) to split the string.
• The third constructor also uses a comma, but with returnDelims = true, so the delimiters themselves are returned as tokens.

Methods of the StringTokenizer Class

The following table shows the commonly used method of StringTokenizer class:

Example of StringTokenizer Class Methods

These examples demonstrate how to use the various methods of the StringTokenizer class.

Example 1: StringTokenizer.nextToken(String delim) Method

The following example demonstrates how to get the next token from a string using a specified delimiter.

Output:

Next token is : my

Note: The StringTokenizer class is deprecated now. It is recommended to use the split() method of the String class or the Pattern class from java.util.regex.

Example 2: StringTokenizer.hasMoreTokens() Method

The following example demonstrates how to check if more tokens are available and iterate through them.

Output:

Demonstrating
methods
from
StringTokenizer
class

Example 3: StringTokenizer.hasMoreElements() Method

This method works similarly to hasMoreTokens() but is useful when using the Enumeration interface. Consider the following code:

Output:

Hello
everyone
I
am
a
Java
developer

Example 4: StringTokenizer.nextElement() Method

The nextElement() method returns the next token as an Object and can be used with Enumeration. Here is the example code:

Output:

Hello
Everyone
Have
a
nice
day

Example 5: StringTokenizer.countTokens() Method

This method counts the total number of tokens in the string. The following example demonstrates the same:

Output:

Total number of Tokens: 6