67

我尝试实现一个协议,该协议可以使用 TLS 运行 TLS twisted.protocols.tls,这是一个使用内存 BIO 的 OpenSSL 接口。

我将它实现为一个协议包装器,它看起来像一个常规的 TCP 传输,但它具有分别添加startTLSstopTLS删除 TLS 层的方法。这适用于 TLS 的第一层。如果我通过“本机”Twisted TLS 传输运行它,它也可以正常工作。但是,如果我尝试使用startTLS此包装器提供的方法添加第二个 TLS 层,则会立即出现握手错误,并且连接最终会处于某种未知的不可用状态。

让它工作的包装器和两个助手看起来像这样:

from twisted.python.components import proxyForInterface
from twisted.internet.error import ConnectionDone
from twisted.internet.interfaces import ITCPTransport, IProtocol
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory

class TransportWithoutDisconnection(proxyForInterface(ITCPTransport)):
    """
    A proxy for a normal transport that disables actually closing the connection.
    This is necessary so that when TLSMemoryBIOProtocol notices the SSL EOF it
    doesn't actually close the underlying connection.

    All methods except loseConnection are proxied directly to the real transport.
    """
    def loseConnection(self):
        pass


class ProtocolWithoutConnectionLost(proxyForInterface(IProtocol)):
    """
    A proxy for a normal protocol which captures clean connection shutdown
    notification and sends it to the TLS stacking code instead of the protocol.
    When TLS is shutdown cleanly, this notification will arrive.  Instead of telling
    the protocol that the entire connection is gone, the notification is used to
    unstack the TLS code in OnionProtocol and hidden from the wrapped protocol.  Any
    other kind of connection shutdown (SSL handshake error, network hiccups, etc) are
    treated as real problems and propagated to the wrapped protocol.
    """
    def connectionLost(self, reason):
        if reason.check(ConnectionDone):
            self.onion._stopped()
        else:
            super(ProtocolWithoutConnectionLost, self).connectionLost(reason)


class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol, it can run over
    any other ITransport.  As a transport, it implements stackable TLS.  That is,
    whatever application traffic is generated by the protocol running on top of
    OnionProtocol can be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS conversation can
    be encapsulated in yet *another* TLS conversation.

    Each layer of TLS can use different connection parameters, such as keys, ciphers,
    certificate requirements, etc.  At the remote end of this connection, each has to
    be decrypted separately, starting at the outermost and working in.  OnionProtocol
    can do this itself, of course, just as it can encrypt each layer starting with the
    innermost.
    """
    def makeConnection(self, transport):
        self._tlsStack = []
        ProtocolWrapper.makeConnection(self, transport)


    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given contextFactory.

        If *client* is True, this side of the connection will be an SSL client.
        Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of the SSL handshake
        were received by the protocol running on top of OnionProtocol, they must be
        passed here as the **bytes** parameter.
        """
        # First, create a wrapper around the application-level protocol
        # (wrappedProtocol) which can catch connectionLost and tell this OnionProtocol 
        # about it.  This is necessary to pop from _tlsStack when the outermost TLS
        # layer stops.
        connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
        connLost.onion = self
        # Construct a new TLS layer, delivering events and application data to the
        # wrapper just created.
        tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        # Push the previous transport and protocol onto the stack so they can be
        # retrieved when this new TLS layer stops.
        self._tlsStack.append((self.transport, self.wrappedProtocol))

        # Create a transport for the new TLS layer to talk to.  This is a passthrough
        # to the OnionProtocol's current transport, except for capturing loseConnection
        # to avoid really closing the underlying connection.
        transport = TransportWithoutDisconnection(self.transport)

        # Make the new TLS layer the current protocol and transport.
        self.wrappedProtocol = self.transport = tlsProtocol

        # And connect the new TLS layer to the previous outermost transport.
        self.transport.makeConnection(transport)

        # If the application accidentally got some bytes from the TLS handshake, deliver
        # them to the new TLS layer.
        if bytes is not None:
            self.wrappedProtocol.dataReceived(bytes)


    def stopTLS(self):
        """
        Remove a layer of TLS.
        """
        # Just tell the current TLS layer to shut down.  When it has done so, we'll get
        # notification in *_stopped*.
        self.transport.loseConnection()


    def _stopped(self):
        # A TLS layer has completely shut down.  Throw it away and move back to the
        # TLS layer it was wrapping (or possibly back to the original non-TLS
        # transport).
        self.transport, self.wrappedProtocol = self._tlsStack.pop()

我有用于执行此操作的简单客户端和服务器程序,可从启动板 ( bzr branch lp:~exarkun/+junk/onion) 获得。当我使用它startTLS两次调用上述方法时,没有干预调用stopTLS,出现此 OpenSSL 错误:

OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]

为什么事情会出错?

4

3 回答 3

19

至少有两个问题OnionProtocol

  1. 当它应该是最外层时,最里面 TLSMemoryBIOProtocol的变成了;wrappedProtocol
  2. ProtocolWithoutConnectionLost不会弹出任何TLSMemoryBIOProtocols offOnionProtocol的堆栈,因为connectionLost仅在FileDescriptorsdoReaddoWrite方法返回断开连接的原因后调用。

如果不改变OnionProtocol管理堆栈的方式,我们就无法解决第一个问题,而在我们弄清楚新的堆栈实现之前,我们无法解决第二个问题。毫不奇怪,正确的设计是数据在 Twisted 中如何流动的直接结果,因此我们将从一些数据流分析开始。

twisted.internet.tcp.ServerTwisted 表示与或的实例建立的连接twisted.internet.tcp.Client。由于我们程序中唯一的交互发生在 中stoptls_client,我们将只考虑进出Client实例的数据流。

让我们用一个最小的客户端来热身,LineReceiver它回显从端口 9999 上的本地服务器接收到的行:

from twisted.protocols import basic
from twisted.internet import defer, endpoints, protocol, task

class LineReceiver(basic.LineReceiver):
    def lineReceived(self, line):
        self.sendLine(line)

def main(reactor):
    clientEndpoint = endpoints.clientFromString(
        reactor, "tcp:localhost:9999")
    connected = clientEndpoint.connect(
        protocol.ClientFactory.forProtocol(LineReceiver))
    def waitForever(_):
        return defer.Deferred()
    return connected.addCallback(waitForever)

task.react(main)

一旦建立连接,aClient就成为我们LineReceiver协议的传输并调解输入和输出:

客户端和线路接收器

来自服务器的新数据导致反应器调用Client'doRead方法,然后将接收到的内容传递给LineReceiver'dataReceived方法。最后,在至少有一条线路可用时LineReceiver.dataReceived调用。LineReceiver.lineReceived

我们的应用程序通过调用将一行数据发送回服务器LineReceiver.sendLine。这调用write绑定到协议实例的传输,该Client实例与处理传入数据的实例相同。 Client.write安排reactor发送数据,而Client.doWrite实际上是通过socket发送数据。

我们已经准备好看看OnionClient从不调用的行为startTLS

没有 startTLS 的 OnionClient

OnionClients 被包裹在OnionProtocols中,这是我们尝试嵌套 TLS 的关键。作为 的子类twisted.internet.policies.ProtocolWrapper, 的实例OnionProtocol是一种协议传输三明治;它将自己呈现为较低级别传输的协议,并作为协议的传输它通过在连接时由WrappingFactory.

现在,Client.doRead调用OnionProtocol.dataReceived,它将数据代理到OnionClient。作为OnionClient的传输,OnionProtocol.write接受要发送的线路并将OnionClient.sendLine它们代理到Client自己的传输。这是 a 、它的封装协议和它自己的传输之间的正常交互ProtocolWrapper,因此数据自然地流入和流出每个都没有任何麻烦。

OnionProtocol.startTLS做一些不同的事情。它试图在已建立的协议-传输对之间插入一个新的ProtocolWrapper——恰好是一个——。这似乎很简单:a将上层协议存储为其属性,并将代理和其他属性存储到它自己的传输中。 应该能够通过修补它自己的实例和来注入一个包装到连接中的新实例:TLSMemoryBIOProtocolProtocolWrapperwrappedProtocolwritestartTLSTLSMemoryBIOProtocolOnionClientwrappedProtocoltransport

def startTLS(self):
    ...
    connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
    connLost.onion = self
    # Construct a new TLS layer, delivering events and application data to the
    # wrapper just created.
    tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)

    # Push the previous transport and protocol onto the stack so they can be
    # retrieved when this new TLS layer stops.
    self._tlsStack.append((self.transport, self.wrappedProtocol))
    ...
    # Make the new TLS layer the current protocol and transport.
    self.wrappedProtocol = self.transport = tlsProtocol

这是第一次调用后的数据流startTLS

startTLS 一个 TLSMemoryBIOProtocol,工作中

正如预期的那样,传递到的新数据OnionProtocol.dataReceived被路由到TLSMemoryBIOProtocol存储在 上_tlsStack,它将解密的明文传递给OnionClient.dataReceivedOnionClient.sendLine还将其数据传递给TLSMemoryBIOProtocol.write,后者对其进行加密并将生成的密文发送给OnionProtocol.write然后Client.write

不幸的是,该方案在第二次调用startTLS. 根本原因是这一行:

    self.wrappedProtocol = self.transport = tlsProtocol

每次调用都将startTLS替换为wrappedProtocol内层 TLSMemoryBIOProtocol,即使接收到的数据Client.doRead已被最外层加密:

startTLS 两个 TLSMemoryBIOProtocols,坏了

但是,transports 嵌套正确。 OnionClient.sendLine只能调用它的传输层write——也就是说OnionProtocol.write——所以OnionProtocol应该用transport最里面的替换它,TLSMemoryBIOProtocol以确保写入连续嵌套在额外的加密层中。

因此,解决方案是确保数据依次从第一个流到下一个以便每一层加密都按照其应用的相反顺序剥离:TLSMemoryBIOProtocol_tlsStack

startTLS 与两个 TLSMemoryBIOProtocols,工作

_tlsStack鉴于这一新要求,以列表的形式表示似乎不太自然。幸运的是,线性表示传入的数据流建议了一种新的数据结构:

传入数据作为链表遍历

传入数据的错误和正确流都类似于单链表,wrappedProtocol用作ProtocolWrappers 下一个链接并protocol用作Client's。该列表应从 向下增长OnionProtocol并始终以OnionClient. 发生该错误是因为违反了该排序不变量。

单链表可以很好地将协议推送到堆栈上,但很难将它们弹出,因为它需要从其头部向下遍历到节点才能删除。当然,每次接收到数据时都会发生这种遍历,因此关注的是额外遍历所隐含的复杂性,而不是最坏情况下的时间复杂度。幸运的是,该列表实际上是双向链接的:

带有协议和传输的双向链表

transport属性将每个嵌套协议与其前身链接起来,以便transport.write在最终通过网络发送数据之前,可以依次进行较低级别的加密。我们有两个哨兵来帮助管理列表:Client必须始终位于顶部,并且OnionClient必须始终位于底部。

将两者放在一起,我们最终得到:

from twisted.python.components import proxyForInterface
from twisted.internet.interfaces import ITCPTransport
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory


class PopOnDisconnectTransport(proxyForInterface(ITCPTransport)):
    """
    L{TLSMemoryBIOProtocol.loseConnection} shuts down the TLS session
    and calls its own transport's C{loseConnection}.  A zero-length
    read also calls the transport's C{loseConnection}.  This proxy
    uses that behavior to invoke a C{pop} callback when a session has
    ended.  The callback is invoked exactly once because
    C{loseConnection} must be idempotent.
    """
    def __init__(self, pop, **kwargs):
        super(PopOnDisconnectTransport, self).__init__(**kwargs)
        self._pop = pop

    def loseConnection(self):
        self._pop()
        self._pop = lambda: None


class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol,
    it can run over any other ITransport.  As a transport, it
    implements stackable TLS.  That is, whatever application traffic
    is generated by the protocol running on top of OnionProtocol can
    be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS
    conversation can be encapsulated in yet *another* TLS
    conversation.

    Each layer of TLS can use different connection parameters, such as
    keys, ciphers, certificate requirements, etc.  At the remote end
    of this connection, each has to be decrypted separately, starting
    at the outermost and working in.  OnionProtocol can do this
    itself, of course, just as it can encrypt each layer starting with
    the innermost.
    """

    def __init__(self, *args, **kwargs):
        ProtocolWrapper.__init__(self, *args, **kwargs)
        # The application level protocol is the sentinel at the tail
        # of the linked list stack of protocol wrappers.  The stack
        # begins at this sentinel.
        self._tailProtocol = self._currentProtocol = self.wrappedProtocol


    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given
        contextFactory.

        If *client* is True, this side of the connection will be an
        SSL client.  Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of
        the SSL handshake were received by the protocol running on top
        of OnionProtocol, they must be passed here as the **bytes**
        parameter.
        """
        # The newest TLS session is spliced in between the previous
        # and the application protocol at the tail end of the list.
        tlsProtocol = TLSMemoryBIOProtocol(None, self._tailProtocol, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        if self._currentProtocol is self._tailProtocol:
            # This is the first and thus outermost TLS session.  The
            # transport is the immutable sentinel that no startTLS or
            # stopTLS call will move within the linked list stack.
            # The wrappedProtocol will remain this outermost session
            # until it's terminated.
            self.wrappedProtocol = tlsProtocol
            nextTransport = PopOnDisconnectTransport(
                original=self.transport,
                pop=self._pop
            )
            # Store the proxied transport as the list's head sentinel
            # to enable an easy identity check in _pop.
            self._headTransport = nextTransport
        else:
            # This a later TLS session within the stack.  The previous
            # TLS session becomes its transport.
            nextTransport = PopOnDisconnectTransport(
                original=self._currentProtocol,
                pop=self._pop
            )

        # Splice the new TLS session into the linked list stack.
        # wrappedProtocol serves as the link, so the protocol at the
        # current position takes our new TLS session as its
        # wrappedProtocol.
        self._currentProtocol.wrappedProtocol = tlsProtocol
        # Move down one position in the linked list.
        self._currentProtocol = tlsProtocol
        # Expose the new, innermost TLS session as the transport to
        # the application protocol.
        self.transport = self._currentProtocol
        # Connect the new TLS session to the previous transport.  The
        # transport attribute also serves as the previous link.
        tlsProtocol.makeConnection(nextTransport)

        # Left over bytes are part of the latest handshake.  Pass them
        # on to the innermost TLS session.
        if bytes is not None:
            tlsProtocol.dataReceived(bytes)


    def stopTLS(self):
        self.transport.loseConnection()


    def _pop(self):
        pop = self._currentProtocol
        previous = pop.transport
        # If the previous link is the head sentinel, we've run out of
        # linked list.  Ensure that the application protocol, stored
        # as the tail sentinel, becomes the wrappedProtocol, and the
        # head sentinel, which is the underlying transport, becomes
        # the transport.
        if previous is self._headTransport:
            self._currentProtocol = self.wrappedProtocol = self._tailProtocol
            self.transport = previous
        else:
            # Splice out a protocol from the linked list stack.  The
            # previous transport is a PopOnDisconnectTransport proxy,
            # so first retrieve proxied object off its original
            # attribute.
            previousProtocol = previous.original
            # The previous protocol's next link becomes the popped
            # protocol's next link
            previousProtocol.wrappedProtocol = pop.wrappedProtocol
            # Move up one position in the linked list.
            self._currentProtocol = previousProtocol
            # Expose the new, innermost TLS session as the transport
            # to the application protocol.
            self.transport = self._currentProtocol



class OnionFactory(WrappingFactory):
    """
    A L{WrappingFactory} that overrides
    L{WrappingFactory.registerProtocol} and
    L{WrappingFactory.unregisterProtocol}.  These methods store in and
    remove from a dictionary L{ProtocolWrapper} instances.  The
    C{transport} patching done as part of the linked-list management
    above causes the instances' hash to change, because the
    C{__hash__} is proxied through to the wrapped transport.  They're
    not essential to this program, so the easiest solution is to make
    them do nothing.
    """
    protocol = OnionProtocol

    def registerProtocol(self, protocol):
        pass


    def unregisterProtocol(self, protocol):
        pass

(这也可以在GitHub上找到。)

第二个问题的解决方案在于PopOnDisconnectTransport。原始代码试图通过 来从堆栈中弹出一个 TLS 会话connectionLost,但是因为只有一个关闭的文件描述符会导致connectionLost被调用,所以它无法删除没有关闭底层套接字的已停止的 TLS 会话。

在撰写本文时,恰好在两个地方TLSMemoryBIOProtocol调用它的传输:和. 在主动关闭时调用(、、和之后并且所有挂起的写入都已刷新),而在被动关闭时调用(握手失败空读取读取错误写入错误)。这一切都意味着关闭连接的双方都可以. 这样做是幂等的,因为通常是幂等的,并且当然期望它是幂等的。loseConnection_shutdownTLS_tlsShutdownFinished_shutdownTLSloseConnectionabortConnectionunregisterProducerloseConnection_tlsShutdownFinishedloseConnectionPopOnDisconnectTransportloseConnectionTLSMemoryBIOProtocol

将堆栈管理逻辑放入其中的缺点loseConnection是它取决于TLSMemoryBIOProtocol's 实现的细节。一个通用的解决方案需要跨多个 Twisted 级别的新 API。

在那之前,我们会遇到另一个海伦定律的例子。

于 2017-12-09T11:42:39.423 回答
1

您可能需要在启动之前通知远程设备您希望启动一个环境并为第二层分配资源(如果该设备具有该功能)。

于 2011-05-20T13:04:46.803 回答
0

如果您对两个层使用相同的 TLS 参数并且连接到同一主机,那么您可能对两个加密层使用相同的密钥对。尝试为嵌套层使用不同的密钥对,例如隧道到第三个主机/端口。即:(localhost:30000客户端)-> localhost:8080(使用密钥对 A 的 TLS 层 1)-> localhost:8081(使用密钥对 B 的 TLS 层 2)。

于 2011-07-20T06:41:14.803 回答