esuslogo
 [To advertise Java(tm) Events here, contact us!]
banner

Java™
by example!






New @ Esus.com


  gb  In-house search engine for better results!

  gb  Get updates with the esus.com
newsletter!









  Home 
 Browse Categories 
 Ask a Java Question 
 Help 
  For Java Tips & Tricks, subscribe to the esus.com newsletter!
Search Java Q&A, Links, API's:   adv 

How do I tokenize a text with a StringTokenizer?

First determine what characters delimit your text tokens, for example if you're parsing a comma-separated values file (csv), your delimiter character would probably be the comma. Then create an instance of StringTokenizer with the String you want to split up into tokens and pass it your set of delimiter characters. You can also use the default set of delimiter characters which is " \t\n\r\f", the space character, the tab character, the newline character, the carriage-return character and the form-feed character.

Then continually call the method hasMoreTokens and nextToken until there are no more tokens. The StringTokenizer will keep track of the current position and nextToken will return the set of characters that appear before the next delimiter character. You can change the delimiter character(s) anytime by calling nextToken(String delim).

Here's an example that reads in a csv and prints out its tokens.

csv.txt:

This code sample is only viewable to esus.com members
Login or become a member!


Main.java:

This code sample is only viewable to esus.com members
Login or become a member!


outputs:

 
This code sample is only viewable to esus.com members
Login or become a member!




Further Information
Author of answer: Joris Van den Bogaert

Comments to this answer are only viewable by members. Login or become a member!





Terms of Service | Privacy Policy | Contact

Copyright © 2000-2003 Esus.com - All Rights Reserved 
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. Esus.com is independent of Sun Microsystems, Inc. All other trademarks are the sole property of their respective owners.