10

我正在尝试为将记住结果的类的实例方法构建一个装饰器。(这已经做过一百万次了)但是,我希望能够在任何时候重置记忆缓存的选项(例如,如果实例状态中的某些内容发生了变化,这可能会改变方法的结果没有任何内容与它的参数有关)。因此,我尝试将装饰器构建为类而不是函数,以便我可以作为类成员访问缓存。这使我走上了学习描述符的道路,特别是__get__方法,这是我真正陷入困境的地方。我的代码如下所示:

import time

class memoized(object):

    def __init__(self, func):
        self.func = func
        self.cache = {}

    def __call__(self, *args, **kwargs):

        key = (self.func, args, frozenset(kwargs.iteritems()))

        try:
            return self.cache[key]
        except KeyError:
            self.cache[key] = self.func(*args, **kwargs)
            return self.cache[key]
        except TypeError:
            # uncacheable, so just return calculated value without caching
            return self.func(*args, **kwargs)

    # self == instance of memoized
    # obj == instance of my_class
    # objtype == class object of __main__.my_class
    def __get__(self, obj, objtype=None):
        """Support instance methods"""
        if obj is None:
            return self

        # new_func is the bound method my_func of my_class instance
        new_func = self.func.__get__(obj, objtype)

        # instantiates a brand new class...this is not helping us, because it's a 
        # new class each time, which starts with a fresh cache
        return self.__class__(new_func)

    # new method that will allow me to reset the memoized cache
    def reset(self):
        print "IN RESET"
        self.cache = {}

class my_class:
    @memoized
    def my_func(self, val):
        print "in my_func"
        time.sleep(2)
        return val


c = my_class()

print "should take time"
print c.my_func(55)
print

print "should be instant"
print c.my_func(55)
print

c.my_func.reset()

print "should take time"
print c.my_func(55)

这清楚和/或可能吗?每次__get__调用时,我都会得到一个全新的记忆类实例,它会丢失包含实际数据的缓存。我一直在努力工作__get__,但没有取得太大进展。

是否有一个完全独立的方法来解决我完全遗漏的这个问题?欢迎和赞赏所有建议/建议。谢谢。

4

3 回答 3

6

我没有尝试解决您的实现机制,而是从PythonDecoratorLibrarymemoized中获取了装饰器类,并将其修改为 add 。下面是结果;我使用的技巧是向装饰函数本身添加一个可调用属性。resetreset

    class memoized2(object):
       """Decorator that caches a function's return value each time it is called.
       If called later with the same arguments, the cached value is returned, and
       not re-evaluated.
       """
       def __init__(self, func):
          self.func = func
          self.cache = {}
       def __call__(self, *args):
          try:
             return self.cache[args]
          except KeyError:
             value = self.func(*args)
             self.cache[args] = value
             return value
          except TypeError:
             # uncachable -- for instance, passing a list as an argument.
             # Better to not cache than to blow up entirely.
             return self.func(*args)
       def __repr__(self):
          """Return the function's docstring."""
          return self.func.__doc__
       def __get__(self, obj, objtype):
          """Support instance methods."""
          fn = functools.partial(self.__call__, obj)
          fn.reset = self._reset
          return fn
       def _reset(self):
          self.cache = {}


    class my_class:
        @memoized2
        def my_func(self, val):
            print "in my_func"
            time.sleep(2)
            return val


    c = my_class()

    print "should take time"
    print c.my_func(55)
    print

    print "should be instant"
    print c.my_func(55)
    print

    c.my_func.reset()

    print "should take time"
    print c.my_func(55)
于 2010-12-13T17:59:00.483 回答
4

在@aix 给出的原始问题的答案的基础上,我创建了一个我认为可以改进它的类。主要特点是缓存的值被存储为方法被修饰的实例的属性,因此很容易重置它们。

class memoize(object):
  def __init__(self, func):
    #print "Init"
    self.func = func

  def __call__(self, *args):
    #print "Call"
    if not self.func in self.cache:
        self.cache[self.func] = {}
    try:
        return self.cache[self.func][args]
    except KeyError:
        value = self.func(*args)
        self.cache[self.func][args] = value
        return value
    except TypeError:
        # uncachable -- for instance, passing a list as an argument.
        # Better to not cache than to blow up entirely.
        return self.func(*args)

  def __repr__(self):
    """Return the function's docstring."""
    return self.func.__doc__

  def __get__(self, obj, objtype):
    """Support instance methods."""
    #print "Get", obj, objtype
    fn = functools.partial(self.__call__, obj)
    try:
        self.cache = obj.cache
    except:
        obj.cache = {}
        self.cache = obj.cache
    #print self.cache
    return fn

作为使用示例:

class MyClass(object):
    def __init__(self,data):
        self.data = data

    def update(self,data):
        self.data = data
        self.cache = {}

    @memoize
    def func1(self,x):
        print "Computing func1"
        return "I am func1 of %s. Data is %s. x is %s\n" % (self, self.data, x)

    @memoize
    def func2(self,x):
        print "Computing func2"
        return "I am func2 of %s. Data is %s. x is %s\n" % (self, self.data, x)

    def func3(self,x):
        print "Computing func3"
        return "I am func3 of %s. Data is %s. x is %s\n" % (self, self.data, x)

mc1 = MyClass("data1")
mc2 = MyClass("data2")
mc3 = MyClass("data3")

print mc1.func1(1) 
print mc1.func1(1) 
print mc1.func2(1) 
print mc1.func2(1) 
print mc1.func3(1) 
print mc1.func3(1) 

print mc2.func1(1) 
print mc2.func1(1) 
print mc2.func2(1) 
print mc2.func2(1) 
print mc2.func3(1) 
print mc2.func3(1) 

print "Update mc1\n"
mc1.update("data1new")

print mc1.func1(1) 
print mc1.func2(1) 
print mc1.func3(1) 
print mc2.func1(1) 
print mc2.func2(1) 
print mc2.func3(1) 

得到输出:

Computing func1
I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1. x is 1

Computing func1
I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Update mc1

Computing func1
I am func1 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

Computing func2
I am func2 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100470fd0>. Data is data1new. x is 1

I am func1 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

I am func2 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1

Computing func3
I am func3 of <__main__.MyClass object at 0x100476050>. Data is data2. x is 1
于 2011-12-25T11:44:47.613 回答
0

好吧,我想指出您的代码中的两个性能问题。这不是您问题的答案,但我不能发表评论。感谢@delnan 指出has_key已弃用。代替:

    try:
        return self.cache[key]
    except KeyError:
        self.cache[key] = self.func(*args, **kwargs)
        return self.cache[key]
    except TypeError:
        # uncacheable, so just return calculated value without caching
        return self.func(*args, **kwargs)

我会这样:

resultDone = False
result = None
try:
  if key in self.cache: return self.cache[key]
  else:
    result = self.func(*args, **kwargs)
    resultDone = True
    self.cache[key] = result
except TypeError: # unhashable key
  pass
if resultDone: return result
else: return self.func(*args, **kwargs)

这避免了: a) try/except KeyError;b) 调用cache[key]返回;c) 在不可散列的键上再次调用该函数。

于 2010-12-13T18:11:30.357 回答