0

我有多个脚本正在导出相同的接口,它们是在绝缘范围内使用 execfile() 执行的。

问题是,我希望他们共享一些资源,这样每个新脚本就不必从一开始就重新加载它们,从而降低启动速度并使用不必要的 RAM。

与下面的示例相比,这些脚本实际上被更好地封装和防范恶意插件,这就是我的问题开始的地方。

问题是,我希望创建资源的脚本能够用数据填充它、删除数据或删除资源,当然还可以访问它的数据。

但是其他脚本应该无法更改其他脚本资源,只需阅读即可。我想确保新安装的插件不会通过滥用共享资源来干扰已经加载和运行的插件。

例子:

class SharedResources:
    # Here should be a shared resource manager that I tried to write
    # but got stuck. That's why I ask this long and convoluted question!
    # Some beginning:
    def __init__ (self, owner):
        self.owner = owner

    def __call__ (self):
        # Here we should return some object that will do
        # required stuff. Read more for details.
        pass

class plugin (dict):
    def __init__ (self, filename):
        dict.__init__(self)
        # Here some checks and filling with secure versions of __builtins__ etc.
        # ...
        self["__name__"] = "__main__"
        self["__file__"] = filename
        # Add a shared resources manager to this plugin
        self["SharedResources"] = SharedResources(filename)
        # And then:
        execfile(filename, self, self)

    # Expose the plug-in interface to outside world:
    def __getattr__ (self, a):
        return self[a]
    def __setattr__ (self, a, v):
        self[a] = v
    def __delattr__ (self, a):
        del self[a]
    # Note: I didn't use self.__dict__ because this makes encapsulation easier.
    # In future I won't use object itself at all but separate dict to do it. For now let it be

----------------------------------------
# An example of two scripts that would use shared resource and be run with plugins["name"] = plugin("<filename>"):
# Presented code is same in both scripts, what comes after will be different.

def loadSomeResource ():
    # Do it here...
    return loadedresource

# Then Load this resource if it's not already loaded in shared resources, if it isn't then add loaded resource to shared resources:
shr = SharedResources() # This would be an instance allowing access to shared resources
if not shr.has_key("Default Resources"):
    shr.create("Default Resources")
if not shr["Default Resources"].has_key("SomeResource"):
    shr["Default Resources"].add("SomeResource", loadSomeResource())
resource = shr["Default Resources"]["SomeResource"]
# And then we use normally resource variable that can be any object.
# Here I Used category "Default Resources" to add and/or retrieve a resource named "SomeResource".
# I want more categories so that plugins that deal with audio aren't mixed with plug-ins that deal with video for instance. But this is not strictly needed.
# Here comes code specific for each plug-in that will use shared resource named "SomeResource" from category "Default Resources".
...
# And end of plugin script!
----------------------------------------

# And then, in main program we load plug-ins:
import os
plugins = {} # Here we store all loaded plugins
for x in os.listdir("plugins"):
    plugins[x] = plugin(x)

假设我们的两个脚本存储在插件目录中,并且都使用了一些加载到内存中的 WAVE 文件。首先加载的插件将加载 WAVE 并将其放入 RAM。其他插件将能够访问已经加载的 WAVE 但不能替换或删除它,从而与其他插件混淆。

现在,我希望每个资源都有一个所有者、插件脚本的某个 ID 或文件名,并且该资源只能由其所有者写入。

任何调整或变通方法都不应该使另一个插件能够访问第一个插件。

我几乎做到了,然后被卡住了,我的脑海里充满了一些概念,这些概念在实施时会做,但只是部分做。这会吃掉我,所以我不能再集中精力了。任何建议都非常受欢迎!

添加:

这就是我现在使用的,不包括任何安全性:

# Dict that will hold a category of resources (should implement some security):
class ResourceCategory (dict):
    def __getattr__ (self, i): return self[i]
    def __setattr__ (self, i, v): self[i] = v
    def __delattr__ (self, i): del self[i]

SharedResources = {} # Resource pool

class ResourceManager:
    def __init__ (self, owner):
        self.owner = owner

    def add (self, category, name, value):
        if not SharedResources.has_key(category):
            SharedResources[category] = ResourceCategory()
        SharedResources[category][name] = value

    def get (self, category, name):
        return SharedResources[category][name]

    def rem (self, category, name=None):
        if name==None: del SharedResources[category]
        else: del SharedResources[category][name]

    def __call__ (self, category):
        if not SharedResources.has_key(category):
            SharedResources[category] = ResourceCategory()
        return SharedResources[category]

    __getattr__ = __getitem__ = __call__

    # When securing, this must not be left as this, it is unsecure, can provide a way back to SharedResources pool:
    has_category = has_key = SharedResources.has_key

现在是一个插件胶囊:

class plugin(dict):
    def __init__ (self, path, owner):
        dict.__init__()
        self["__name__"] = "__main__"
        # etc. etc.
        # And when adding resource manager to the plugin, register it with this plugin as an owner
        self["SharedResources"] = ResourceManager(owner)
        # ...
        execfile(path, self, self)
        # ...

插件脚本示例:

#-----------------------------------
# Get a category we want. (Using __call__() ) Note: If a category doesn't exist, it is created automatically.
AudioResource = SharedResources("Audio")
# Use an MP3 resource (let say a bytestring):
if not AudioResource.has_key("Beep"):
    f = open("./sounds/beep.mp3", "rb")
    Audio.Beep = f.read()
    f.close()
# Take a reference out for fast access and nicer look:
beep = Audio.Beep # BTW, immutables doesn't propagate as references by themselves, doesn't they? A copy will be returned, so the RAM space usage will increase instead. Immutables shall be wrapped in a composed data type.

这很有效,但正如我所说,在这里搞乱资源太容易了。

我想要一个 ResourceManager() 的实例来负责返回什么版本的存储数据。

4

1 回答 1

1

所以,我的一般方法是这样的。

  1. 有一个中央共享资源池。通过这个池的访问对每个人都是只读的。将所有数据包装在共享池中,这样“按规则行事”的人就无法编辑其中的任何内容。

  2. 每个代理(插件)在加载它时都保存着它“拥有”的知识。它为自己保留一个读/写引用,并将对该资源的引用注册到集中式只读池。

  3. 加载插件时,它会获得对中央只读池的引用,它可以向其注册新资源。

因此,仅解决 python 本机数据结构(而不​​是自定义类的实例)的问题,一个相当锁定的只读实现系统如下。请注意,用于锁定它们的技巧与有人可以用来绕过锁定的技巧相同,因此如果具有一点 Python 知识的人积极尝试破解沙盒,则沙盒非常弱。

import collections as _col
import sys

if sys.version_info >= (3, 0):
    immutable_scalar_types = (bytes, complex, float, int, str)
else:
    immutable_scalar_types = (basestring, complex, float, int, long)

# calling this will circumvent any control an object has on its own attribute lookup
getattribute = object.__getattribute__

# types that will be safe to return without wrapping them in a proxy
immutable_safe = immutable_scalar_types

def add_immutable_safe(cls):
    # decorator for adding a new class to the immutable_safe collection
    # Note: only ImmutableProxyContainer uses it in this initial
    # implementation
    global immutable_safe
    immutable_safe += (cls,)
    return cls

def get_proxied(proxy):
    # circumvent normal object attribute lookup
    return getattribute(proxy, "_proxied")

def set_proxied(proxy, proxied):
    # circumvent normal object attribute setting
    object.__setattr__(proxy, "_proxied", proxied)

def immutable_proxy_for(value):
    # Proxy for known container types, reject all others
    if isinstance(value, _col.Sequence):
        return ImmutableProxySequence(value)
    elif isinstance(value, _col.Mapping):
        return ImmutableProxyMapping(value)
    elif isinstance(value, _col.Set):
        return ImmutableProxySet(value)
    else:
        raise NotImplementedError(
            "Return type {} from an ImmutableProxyContainer not supported".format(
                type(value)))

@add_immutable_safe
class ImmutableProxyContainer(object):

    # the only names that are allowed to be looked up on an instance through
    # normal attribute lookup
    _allowed_getattr_fields = ()

    def __init__(self, proxied):
        set_proxied(self, proxied)

    def __setattr__(self, name, value):
        # never allow attribute setting through normal mechanism
        raise AttributeError(
            "Cannot set attributes on an ImmutableProxyContainer")

    def __getattribute__(self, name):
        # enforce attribute lookup policy
        allowed_fields = getattribute(self, "_allowed_getattr_fields")
        if name in allowed_fields:
            return getattribute(self, name)
        raise AttributeError(
            "Cannot get attribute {} on an ImmutableProxyContainer".format(name))

    def __repr__(self):
        proxied = get_proxied(self)
        return "{}({})".format(type(self).__name__, repr(proxied))

    def __len__(self):
        # works for all currently supported subclasses
        return len(get_proxied(self))

    def __hash__(self):
        # will error out if proxied object is unhashable
        proxied = getattribute(self, "_proxied")
        return hash(proxied)

    def __eq__(self, other):
        proxied = get_proxied(self)
        if isinstance(other, ImmutableProxyContainer):
            other = get_proxied(other)
        return proxied == other


class ImmutableProxySequence(ImmutableProxyContainer, _col.Sequence):

    _allowed_getattr_fields = ("count", "index")

    def __getitem__(self, index):
        proxied = get_proxied(self)
        value = proxied[index]
        if isinstance(value, immutable_safe):
            return value
        return immutable_proxy_for(value)


class ImmutableProxyMapping(ImmutableProxyContainer, _col.Mapping):

    _allowed_getattr_fields = ("get", "keys", "values", "items")

    def __getitem__(self, key):
        proxied = get_proxied(self)
        value = proxied[key]
        if isinstance(value, immutable_safe):
            return value
        return immutable_proxy_for(value)

    def __iter__(self):
        proxied = get_proxied(self)
        for key in proxied:
            if not isinstance(key, immutable_scalar_types):
                # If mutable keys are used, returning them could be dangerous.
                # If owner never puts a mutable key in, then integrity should
                # be okay. tuples and frozensets should be okay as keys, but
                # are not supported in this implementation for simplicity.
                raise NotImplementedError(
                    "keys of type {} not supported in "
                    "ImmutableProxyMapping".format(type(key)))
            yield key


class ImmutableProxySet(ImmutableProxyContainer, _col.Set):

    _allowed_getattr_fields = ("isdisjoint", "_from_iterable")

    def __contains__(self, value):
        return value in get_proxied(self)

    def __iter__(self):
        proxied = get_proxied(self)
        for value in proxied:
            if isinstance(value, immutable_safe):
                yield value
            yield immutable_proxy_for(value)

    @classmethod
    def _from_iterable(cls, it):
        return set(it)

注意:这仅在 Python 3.4 上进行了测试,但我尝试将其编写为与 Python 2 和 3 兼容。

使共享资源的根成为字典。给ImmutableProxyMapping插件一个字典。

private_shared_root = {}
public_shared_root = ImmutableProxyMapping(private_shared_root)

创建一个 API,插件可以在其中向 注册新资源public_shared_root,可能以先到先得的方式进行(如果已经存在,则无法注册)。private_shared_root使用您知道将需要的任何容器或您想要与所有插件共享但您知道自己想要只读的任何数据进行预填充。

如果共享根映射中键的约定都是字符串,例如文件系统/home/dalen/local/python路径os.path.expanduser(这样,如果插件尝试将相同的资源添加到池中,碰撞检测会立即且微不足道/显而易见。

于 2015-09-06T04:38:42.013 回答