2

我编写了一个简单的 python 代码来计算集合的熵，并且我正在尝试在 Theano 中编写相同的东西。

import math

# this computes the probabilities of each element in the set
def prob(values):
    return [float(values.count(v))/len(values) for v in values]

# this computes the entropy
def entropy(values):
    p = prob(values)
    return -sum([v*math.log(v) for v in p])

我正在尝试在 Theno 中编写等效代码，但我不知道该怎么做：

import theano
import theano.tensor as T

v = T.vector('v') # I create a symbolic vector to represent my initial values
p = T.vector('p') # The same for the probabilities 

# this is my attempt to compute the probabilities which would feed vector p
theano.scan(fn=prob,outputs_info=p,non_sequences=v,n_steps=len(values))

# considering the previous step would work, the entropy is just
e = -T.sum(p*T.log(p))
entropy = theano.function([values],e)

但是，扫描线不正确，我收到大量错误。我不确定是否有一种简单的方法可以做到这一点（计算向量的熵），或者我是否必须在扫描功能上投入更多精力。有任何想法吗？

4

1 回答 1

0

除了 nouiz 提出的观点之外，不应将 P 声明为 T.vector，因为它将是对值向量的计算结果。

此外，要计算熵之类的东西，您不需要使用 Scan （Scan 引入了计算开销，因此只能使用它，因为没有其他方法可以计算您想要的内容或减少内存使用量）；你可以采取这样的方法：

values = T.vector('values')
nb_values = values.shape[0]

# For every element in 'values', obtain the total number of times
# its value occurs in 'values'.
# NOTE : I've done the broadcasting a bit more explicitly than
# needed, for clarity.
freqs = T.eq(values[:,None], values[None, :]).sum(0).astype("float32")

# Compute a vector containing, for every value in 'values', the
# probability of that value in the vector 'values'.
# NOTE : these probabilities do *not* sum to 1 because they do not
# correspond to the probability of every element in the vector 'values
# but to the probability of every value in 'values'. For instance, if
# 'values' is [1, 1, 0] then 'probs' will be [2/3, 2/3, 1/3] because the
# value 1 has probability 2/3 and the value 0 has probability 1/3 in
# values'.
probs = freqs / nb_values

entropy = -T.sum(T.log2(probs) / nb_values)
fct = theano.function([values], entropy)

# Will output 0.918296...
print fct([0, 1, 1])
于 2015-07-15T15:46:09.383 回答