javascript - 计算javascript中的唯一词干数组？

Question

在 javascript 中，我有一个字符串数组。

[
   "test",
   "tests",
   "abc",
   "abcdef"
]

我想创建一个仅包含字符串的唯一词干的新数组。例如，上面的数组将减少为...

[
   "test",
   "abc"
]

...因为“test”是“tests”的词干，而“abc”是“abcdef”的词干。

最简单的方法是什么？

Answer 1

最简单的方法是循环。首先，我建议按字母数排序单词，所以你可以这样做：

var myArray = ["test", "tests", "abc", "abcdef"];

//this sorts from smallest to largest
myArray.sort(function(a,b){return a.length - b.length}); 

所以，现在 myArray 从小到大排序。现在您将对每个元素进行循环以检查它是否是下一个元素的茎。

//this is the array where we will store the stems
var stemArray = [];

//the temporary stem goes here
var stem;

//this variable is used to capture a substring from each string
// to check against the stem variable
var check;

//loop over all the variables except the last
//since they are ordered from smallest to last, we are guaranteed that  
//the last object wont be much of a stem 
//and thus we can avoid that check
for (var i = 0; i < myArray.length - 1; i++){

    //set the current stem
    stem = myArray[i];

    //then loop over the remainding objects
    for (var j = i+1; j < myArray.length; j++){

        //capture a substring
        //so for example, stem = "abc" and the element we're testing against
        //is "test", check will be equal to "tes" so the size matches "abc"
        check = myArray[j].substring(0,stem.length);

        //if stem and check are the same
        //and since you wanted unique we check if it is unique
        //alternatively, we could just break inside the next 
        //conditional statement and it would be more efficient 
        //and thus remove the indexOf test
        //but i include it to explain the logic of how the algorithm works
        if (stem === check){

            if (stemArray.indexOf(stem) === -1){
                //add the verified stem to the array of stems
                stemArray.push(stem);
            }

            //In the case of ["t", "te", "test"], the above code
            //will output ["t", "te"]
            //if you want it to output just ["t"], uncomment the following

            //myArray.splice(j,1);
            //j--;
            //these lines will remove the duplicate from myArray
            //and correct the iteration due to the deletion

        }
    }
}

Answer 2

“简单”当然是相对的。

以简单的观点认为词干将始终匹配较长单词的第一个字符，对数组进行排序以使词干出现在较长单词之前（例如，“test”在“tests”之前），然后遍历数组并针对每个成员进行测试跟随成员，删除那些是词干延伸的成员，例如

function getStems(arr) {
  var a = arr.slice().sort();
  var stem, len;

  for (var i=0, iLen=a.length - 1; i<iLen; i++) {
    stem = a[i];
    len = stem.length;

    while (i<iLen && stem == a[i+1].substring(0,len)) {
      a.splice(i+1, 1);
      --iLen;
    }
  }
  return a;
}

var a = ["test","tests","abcdef","abcf","abcdefqw","abc"];

alert(getStems(a)); // abc, test

但是，给定“a”，“any”，“anyone”，上面将返回“a”，这可能不是您想要的。

Answer 3

您还需要使用下划线字符串。[https://github.com/epeli/underscore.string]

// make a new array that is the old array without...
var myArrayWithNoStems =_.without(myArray,function(word) {
  // anything that matches the filter
   return _.find(myArray, function(word2) {
      // if this word wholly includes another word, return true
      if (word != word2 && _.str.include(word, word2)) return true;
    })
});