5

我有flagVectorOutlier如下代码所示的功能。我正在使用 BreezeDenseVector和DenseMatrix对象来计算distance. 我希望按照函数签名上的编码，得到一个 Spark RDD[(Double, Boolean)]。mi和invCovMatrix分别是 Breeze 的DenseVector[Double]和DenseMatrix[Double]：

  def flagVectorOutlier(testVectors: RDD[(String, SparkVector)], distanceThreshold: Double): RDD[(Double, Boolean)] = {
    val testVectorsDenseRDD = testVectors.map { vector => DenseVector(vector._2.toArray)}
    val mahalanobisDistancesRDD = testVectorsDenseRDD.map { vector =>
      val distance = DenseVector[Double](DenseVector(Transpose(vector - mi) * invCovMatrix) * DenseVector(vector - mi)).toArray
      (distance(0), if(distance(0) >= distanceThreshold) true else false)
    }

    mahalanobisDistancesRDD
  }

编译器最终向我显示以下 2 个错误：

Error:(75, 93) could not find implicit value for parameter op: breeze.linalg.operators.OpMulMatrix.Impl2[breeze.linalg.DenseVector[breeze.linalg.Transpose[breeze.linalg.DenseVector[Double]]],breeze.linalg.DenseVector[breeze.linalg.DenseVector[Double]],That]
      val distance = DenseVector[Double](DenseVector(Transpose(vector - mi) * invCovMatrix) * DenseVector(vector - mi)).toArray
                                                                                            ^

和

Error:(75, 93) not enough arguments for method *: (implicit op: breeze.linalg.operators.OpMulMatrix.Impl2[breeze.linalg.DenseVector[breeze.linalg.Transpose[breeze.linalg.DenseVector[Double]]],breeze.linalg.DenseVector[breeze.linalg.DenseVector[Double]],That])That.
Unspecified value parameter op.
      val distance = DenseVector[Double](DenseVector(Transpose(vector - mi) * invCovMatrix) * DenseVector(vector - mi)).toArray
                                                                                            ^

我错过了什么？我正在考虑以这种方式在 Breeze's 之间进行乘法运算是可能的DenseVector。

4

4 回答 4

3

If you look at the error message carefully, it tells you what went wrong.

(implicit op: OpMulMatrix.Impl2[
       DenseVector[Transpose[DenseVector[Double]]],
       DenseVector[DenseVector[Double]],
       That])

The operands for your multiply are almost certainly not what you intended. The LHS is a DenseVector whose elements are Transpose'd DenseVectors. The RHS is a DenseVector[DenseVector[Double]]. Your outer DenseVector(..) calls are wrapping their arguments in new DenseVectors, rather than turning the argument into a DenseVector.

I believe this is what you wanted:

val diff = (vector - mi).toDenseVector
(diff.t * invCovMatrix * diff)
于 2015-03-12T16:00:28.410 回答
2

Breeze relies heavily on implicits (read on those if you are not familiar with them) in order to determine compatible types for different operations. Clearly, the operator you are calling requires such an implicit parameter and the compiler is not being able to find one in scope. So there are two options:

  1. You are missing an import which will produce the implicit parameters the method is missing.

  2. You are trying to multiply incompatible types, and the proper implicit parameter does not exist. For instance, this tutorial mentions the following:

Assignments with incompatible cardinality or a larger numeric type won't compile.

scala> m := x :13: error: could not find implicit value for parameter op: breeze.linalg.operators.BinaryUpdateOp[breeze.linalg.DenseMatrix[Int],breeze.linalg.DenseVector[Double],breeze.linalg.operators.OpSet] m := x ^

On one hand is nice to have the compiler be able to pick this kind of errors of compatibility, but sadly the error you get is hardly very telling.

于 2015-03-12T15:08:19.420 回答
1

我也遇到了这个问题，Breeze 给了我同样的错误信息。在我的例子中，我的 X 类型是 DenseVector[Double] 而 Y 类型是 DenseVector[Int]，在我将 Y 类型更改为 DenseVector[Double] 后，这个错误就消失了。希望这可以帮到你。

于 2017-02-28T08:52:58.937 回答
0

you should declare the type explicitly as following,focus on variables a and b:

package com.tencent.ieg.dm.demo
import breeze.linalg.{Vector, DenseVector, SparseVector}

object BreezeDemo extends App {

    val a:Vector[Long] = DenseVector(2, 10, 3)
    // val a = DenseVector(2, 10, 3) // cause compiling error
    val b:Vector[Long] = new SparseVector(Array(0),Array(2),3)
    val rst = a dot b
    println(rst)

}
于 2016-09-05T07:59:25.913 回答