1

I need to count perplexity and I try to do it with

def get_perplexity(test_set, model):
    perplexity = 1
    n = 0
    for word in test_set:
        n += 1
        perplexity = perplexity * 1 / get_prob(model, word)
    perplexity = pow(perplexity, 1/float(n))
    return perplexity

And after some steps my perplexity is equal to infinity. I need to get number and as last step to do pow(perplexity, 1/float(n))

Is any to multiply numbers like and get result?

3.887311155784627e+243
8.311806360146177e+250
1.7707049372801292e+263
1.690802669602979e+271
3.843294667766984e+278
5.954424789834101e+290
8.859529887856071e+295
7.649470766862909e+306
4

2 回答 2

1

重复乘法会导致一些棘手的数值不稳定性，因为乘法的结果需要越来越多的位来表示。我建议您将其转换为对数空间并使用求和而不是乘法：

import math

def get_perplexity(test_set, model):
    log_perplexity = 0
    n = 0
    for word in test_set:
        n += 1
        log_perplexity -= math.log(get_prob(model, word))
    log_perplexity /= float(n)
    return math.exp(log_perplexity)

这样，您的所有对数都可以用标准位数表示，并且不会出现数值爆炸和精度损失。decimal此外，您可以使用该模块引入任意程度的精度：

import decimal

def get_perplexity(test_set, model):
    with decimal.localcontext() as ctx:
        ctx.prec = 100  # set as appropriate
        log_perplexity = decimal.Decimal(0)
        n = 0
        for word in test_set:
            n += 1
            log_perplexity -= decimal.Decimal(get_prob(model, word))).ln()
        log_perplexity /= float(n)
        return log_perplexity.exp()
于 2018-12-16T18:15:02.860 回答
0

因为 e+306 只是 10^306 你可以做两部分

class BigPowerFloat:
    POWER_STEP = 10**100
    def __init__(self, input_value):
        self.value = float(input_value)
        self.power = 0

    def _move_to_power(self):
        while self.value > self.POWER_STEP:
            self.value = self.value / self.POWER_STEP
            self.power += self.POWER_STEP
        # you can add similar for negative values           


    def __mul__(self, other):
        self.value *= other
        self._move_to_power()

    # TODO other __calls for /, +, - ...

    def __str__(self):
        pass
        # make your cust to str
于 2018-12-16T18:23:30.470 回答