2

I got an unexpected closure when creating a nested class. I suspect that this is something related to metaclasses, super, or both. It is definitely related to how closures get created. I am using python2.7.

Here are five simplified examples that demonstrate the same problem that I am seeing (they all build off the first):

EXAMPLE 1:

class Metaclass(type): 
    def __init__(self, name, bases, dict): 
        self.CONST = 5 

class Base(object): 
    __metaclass__=Metaclass 
    def __init__(self): 
        "Set things up." 

class Subclass(Base):
    def __init__(self, name):
        super(Subclass, self).__init__(self)
        self.name = name
    def other(self, something): pass

class Test(object):
    def setup(self):
        class Subclass(Base):
            def __init__(self, name):
                super(Subclass, self).__init__(self)
                self.name = name
            def other(self, something): pass
        self.subclass = Subclass
        class Subclass2(Base):
            def __init__(self, name):
                super(Subclass, self).__init__(self)
        self.subclass2 = Subclass2

"0x%x" % id(Metaclass)
# '0x8257f74'
"0x%x" % id(Base)
# '0x825814c'
t=Test()
t.setup()
"0x%x" % id(t.subclass)
# '0x8258e8c'
"0x%x" % id(t.subclass2)
# '0x825907c'
t.subclass.__init__.__func__.__closure__
# (<cell at 0xb7d33d4c: Metaclass object at 0x8258e8c>,)
t.subclass.other.__func__.__closure__
# None
t.subclass2.__init__.__func__.__closure__
# (<cell at 0xb7d33d4c: Metaclass object at 0x8258e8c>,)
Subclass.__init__.__func__.__closure__
# None

EXAMPLE 2:

class Test2(object):
    def setup(self):
        class Subclass(Base):
            def __init__(self, name):
                self.name = name
            def other(self, something): pass
        self.subclass = Subclass

t2=Test2()
t2.setup()
t2.subclass.__init__.__func__.__closure__
# None

EXAMPLE 3:

class Test3(object):
    def setup(self):
        class Other(object):
            def __init__(self): 
                super(Other, self).__init__()
        self.other = Other
        class Other2(object):
            def __init__(self): pass
        self.other2 = Other2

t3=Test3()
t3.setup()
"0x%x" % id(t3.other)
# '0x8259734'
t3.other.__init__.__func__.__closure__
# (<cell at 0xb7d33e54: type object at 0x8259734>,)
t3.other2.__init__.__func__.__closure__
# None

EXAMPLE 4:

class Metaclass2(type): pass

class Base2(object): 
    __metaclass__=Metaclass2 
    def __init__(self): 
        "Set things up." 

class Base3(object): 
    __metaclass__=Metaclass2 

class Test4(object):
    def setup(self):
        class Subclass2(Base2):
            def __init__(self, name):
                super(Subclass2, self).__init__(self)
        self.subclass2 = Subclass2
        class Subclass3(Base3):
            def __init__(self, name):
                super(Subclass3, self).__init__(self)
        self.subclass3 = Subclass3
        class Subclass4(Base3):
            def __init__(self, name):
                super(Subclass4, self).__init__(self)
        self.subclass4 = Subclass4

"0x%x" % id(Metaclass2)
# '0x8259d9c'
"0x%x" % id(Base2)
# '0x825ac9c'
"0x%x" % id(Base3)
# '0x825affc'
t4=Test4()
t4.setup()
"0x%x" % id(t4.subclass2)
# '0x825b964'
"0x%x" % id(t4.subclass3)
# '0x825bcac'
"0x%x" % id(t4.subclass4)
# '0x825bff4'
t4.subclass2.__init__.__func__.__closure__
# (<cell at 0xb7d33d04: Metaclass2 object at 0x825b964>,)
t4.subclass3.__init__.__func__.__closure__
# (<cell at 0xb7d33e9c: Metaclass2 object at 0x825bcac>,)
t4.subclass4.__init__.__func__.__closure__
# (<cell at 0xb7d33ddc: Metaclass2 object at 0x825bff4>,)

EXAMPLE 5:

class Test5(object):
    def setup(self):
        class Subclass(Base):
            def __init__(self, name):
                Base.__init__(self)
        self.subclass = Subclass

t5=Test5()
t5.setup()
"0x%x" % id(t5.subclass)
# '0x8260374'
t5.subclass.__init__.__func__.__closure__
# None

Here is what I understand (referencing examples):

  • Metaclasses are inherited, so Subclass gets Base’s metaclass.
  • Only __init__ is affected, Subclass.other method is not (#1).
  • Removing Subclass.other does not make a difference (#1).
  • Removing self.name=name from Subclass.__init__ does not make a difference (#1).
  • The object in the closure cell is not a function.
  • The object is not Metaclass or Base, but some object of type Metaclass, just like Base is (#1).
  • The object is actually an object of the type of the nested Subclass (#1).
  • The closure cells for t1.subclass.__init__ and t1.subclass2.__init__ are the same, even though they are from two different classes (#1).
  • When I do not nest the creation of Subclass (#1) then there is no closure created.
  • When I do not call super(...).__init__ in Subclass.init__ no closure is created (#2).
  • If I assign no __metaclass__ and inherit from object then the same behavior shows up (#3).
  • The object in the closure cell for t3.other.__init__ is t3.other (#3).
  • The same behavior happens if the metaclass has no __init__ (#4).
  • The same behavior happens if the Base has no __init__ (#4).
  • The closure cells for the three subclasses in example 4 are all different and each matches the corresponding class (#4).
  • When super(...).__init__ is replaced with Base.__init__(self), the closure disappears (#5).

Here is what I do not understand:

  • Why does a closure get set for __init__?
  • Why doesn't the closure get set for other?
  • Why is the object in the closure cell set to the class to which __init__ belongs?
  • Why does this only happen when super(...).__init__ is called?
  • Why doesn't this happen when Base.__init__(self) is called?
  • Does this actually have anything at all to do with using metaclasses (probably, since the default metaclass is type)?

Thanks for the help!

-eric

(Update) Here is something that I found then (based on Jason's insight):

def something1():
    print "0x%x" % id(something1)
    def something2():
        def something3():
            print "0x%x" % id(something1)
            print "0x%x" % id(something2)
            print "0x%x" % id(something3)
        return something3
    return something2

something1.__closure__
# None
something1().__closure__
# 0xb7d4056c
# (<cell at 0xb7d33eb4: function object at 0xb7d40df4>,)
something1()().__closure__
# 0xb7d4056c
# (<cell at 0xb7d33fec: function object at 0xb7d40e64>, <cell at 0xb7d33efc: function object at 0xb7d40e2c>)
something1()()()
# 0xb7d4056c
# 0xb7d4056c
# 0xb7d40e9c
# 0xb7d40ed4

First, a function's name is in scope within its own body. Second, functions get closures for the functions in which they are defined if they reference those functions.

I hadn't realized that the function name was in scope like that. The same goes for classes. When a class is defined within a function's scope, any references to that class name inside the class's methods cause the class to bound in a closure on that method's function, like so:

def test():
    class Test(object):
        def something(self):
            print Test
    return Test

test()
# <class '__main__.Test'>
test().something.__func__.__closure__
# (<cell at 0xb7d33c2c: type object at 0x825e304>,)

However, since closures cannot be created on non-functions the following fails:

def test():
    class Test(object):
        SELF=Test
        def something(self):
            print Test
    return Test

# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
#   File "<stdin>", line 2, in test
#   File "<stdin>", line 3, in Test
# NameError: free variable 'Test' referenced before assignment in enclosing scope

Good stuff!

4

1 回答 1

4

为什么要设置闭包__init__？

它指的Subclass是封闭函数（即 ）中的一个局部变量（即setup）。

为什么不关闭关闭other？

因为它不引用任何封闭函数中的任何局部变量（或参数）。

为什么闭包单元格中的对象设置为__init__所属的类？

那是被引用的封闭变量的值。

为什么这只会在super(...).__init__被调用时发生？

Base.__init__(self)为什么调用时不会发生这种情况？

因为Base不是任何封闭函数中的局部变量。

这实际上与使用元类有什么关系吗？

不。

于 2010-07-30T18:42:01.780 回答