string - When trying to add a space back into a tokenized String, why are two spaces added into my output?

Question

Beginner here. Sorry for the vague title but the code should put my question into perspective.

public static void main(String [] args)
{
    String sentence = "hi. how are you! i'm just dandy.";
    String tokenSent;

    tokenSent = sentenceCapitalizer(sentence);

    System.out.println(tokenSent);
}

public static String sentenceCapitalizer(String theSentence)
{
    StringTokenizer strTokenizer = new StringTokenizer(theSentence, ".!", true);

    String token = null;
    String totalToken = "";
    String ch = "";

    while(strTokenizer.hasMoreTokens())
    {
        token = strTokenizer.nextToken().trim();

        token = token.replace(token.charAt(0), Character.toUpperCase(token.charAt(0)));

        StringBuilder str = new StringBuilder(token);
        str.append(" ");

        totalToken += str;

    }

    return totalToken; 
}

OUTPUT AS IS: Hi . How are you ! I'm just dandy .

I was able to capitalize the first letter of each sentence but I'm wanting the output to keep the same format as the original String. The problem is that it puts a space before and after the ending punctuation. Is there any way to fix this problem using only a StringBuilder and/or StringTokenizer? Thank you for your time.

Answer 1

When you declare new StringTokenizer(theSentence, ".!", true) ,true enables to return delimiter characters also as next token. Since you appending a space to each token, it is normal to get that output.

You can check if the token is a delimiter character, if so, you add the space.

if(token.length() == 1 &&
   delims.indexOf(token.charAt(0)) != -1)
 str.append(" "); //we just hit the delim char, add space

Answer 2

You can do this.

String delim = ".!";
StringTokenizer strTokenizer = new StringTokenizer(theSentence, delim, true);

Then,

if(delim.contains(token))
    str.append(" ");