I have extracted tokens from the document and inserted them in an array. Now what i want is determining how many times a token appears in the document.
The present algorithm that i have in my mind gets each token and scans the whole array of tokens to find similar tokens. This is as you have guessed not very scalable.
I was looking for some other good algorithm to help me solve this problem.
I have some similar questions on SO but they all assume that the token is provided at compile time.