问题标签 [conduit]

在pipes包中，教程Control.Pipes.Tutorial[1] 展示了如何使用 monad>>运算符“垂直堆叠”管道组件。

我将如何使用 Control.Frame 来做到这一点？

例如，使用 Control.Frame 教程中的定义：

在此处使用>>for???不会进行类型检查。

[1] http://hackage.haskell.org/packages/archive/pipes/latest/doc/html/Control-Pipe-Tutorial.html#g:4

更新：这是我一直在尝试的粘贴：http: //hpaste.org/77986

看起来close是问题所在 - 请参阅bar8上面粘贴中的函数。>>如果我没有明确说明，这些框架是可组合的close。当然，我最终需要关闭它们。嗯……

在我看来，这两个想法之间有很强的联系。我的猜测是，如果有一种方法可以用 Iteratees 表达任意图，那么 FRP 可以用 Iteratees 来实现。但是 afaik 他们只支持链状结构。

有人可以对此有所了解吗？

我正在试验导管包。还找到了 network-conduit 包，我尝试制作一个简单的 TCP 客户端，它将文件的内容发送到套接字：

但是，这不起作用，因为应用程序接收器不存在 ResourceT：

但是，如果没有 runResourceT，它显然也无法工作：

我的猜测是我应该在不实际管理资源的情况下将 appSink（其中 m=IO）包装到 ResourceT 中。但我就是不知道该怎么做。

...？

我在考虑 Haskell 中的列表，我认为在其他语言中，不会对所有内容都使用列表。当然，如果您稍后需要这些值，您可能想要存储一个列表，但如果它只是一个关闭，比如说从 迭代[1..n]，为什么要使用一个真正需要的只是一个递增变量的列表？

我还阅读了有关“列表融合”的内容，并注意到虽然 Haskell 编译器尝试实现这种优化以消除中间列表，但它们通常不成功，导致垃圾收集器不得不清理只使用一次的列表。

此外，如果您不小心，很容易共享一个列表，这意味着垃圾收集器不会清理它，这可能导致内存不足，而以前设计为在恒定空间中运行的算法。

所以我认为最好完全避免列表，至少当一个人实际上不想“存储”列表时。

然后我遇到了conduit，上面写着：

流数据问题的解决方案，允许在恒定内存中生产、转换和消费数据流。

这听起来很完美。我知道conduit它是为IO资源获取和释放问题而设计的，但是可以将它用作列表的替代品吗？

例如，我可以执行以下操作：

并且通过一些适当放置的类型注释，在整个过程中使用管道而不是列表？

我希望这可能classy-prelude会允许这样的代码，但我不确定。如果可能的话，有人可以举个例子，像上面说的那样吗？

是否可以将多个管道合并为一个，反之亦然？

我正在努力实现以下功能。

为了学习conduit库的基础知识，我曾经network-conduit制作了一个简单的回显服务器：

它做了我想要的，但是当客户端关闭它的连接部分时，服务器仍在等待输入，而不是写出任何剩余的数据并关闭它的连接发送部分：

我试着删除echo并做

但问题仍然存在。我究竟做错了什么？

我有这个有效的管道表达式：

我无法将它分成两部分，即：

这样我就可以使用：

或者其他什么......我从类型检查器得到的错误信息有些难以理解。

I've written a daemon in Haskell that scrapes information from a webpage every 5 minutes.

The daemon originally ran fine for about 50 minutes, but then it unexpectedly died with out of memory (requested 1048576 bytes). Every time I ran it it died after the same amount of time. Setting it to sleep only 30 seconds, it instead died after 8 minutes.

I realized the code to scrape the website was incredibly memory inefficient (going from about 30M while sleeping to 250M while parsing 9M of html), so I rewrote it so that now it only uses about 15M extra while parsing. Thinking the problem was fixed, I ran the daemon overnight and when I woke up it was actually using less memory than it was that night. I thought I was done, but roughly 20 hours after it had started, it had crashed with the same error.

I started looking into ghc profiling but I wasn't able to get that to work. Next I started messing with rts options, and I tried setting -H64m to set the default heap size to be larger than my program was using, and also using -Ksize to shrink the maximum size of the stack to see if that would make it crash sooner.

Despite every change I've made, the daemon still seems to crash after a constant number of iterations. Making the parsing more memory efficient made this value higher, but it still crashes. This doesn't make sense to me because none of these have runs have even come close to using all of my memory, much less swap space. The heap size is supposed to be unlimited by default, shrinking the stack size didn't make a difference, and all my ulimits are either unlimited or significantly higher than what the daemon is using.

In the original code I pinpointed the crash to somewhere in the html parsing, but I haven't done the same for the more memory efficient version because 20 hours takes so long to run. I don't know if this would even be useful to know because it doesn't seem like any specific part of the program is broken because it run successfully for dozens of iterations before crashing.

Out of ideas, I even looked through the ghc source code for this error, and it appears to be a failed call to mmap, which wasn't very helpful to me because I assume that isn't the root of the problem.

(Edit: code rewritten and moved to end of post)

I'm pretty new at Haskell, so I'm hoping this is some quirk of lazy evaluation or something else that has a quick fix. Otherwise, I'm fresh out of ideas.

I'm using GHC version 7.4.2 on FreeBsd 9.1

Edit:

Replacing the downloading with static html got rid of the problem, so I've narrowed it down to how I'm using http-conduit. I've edited the code above to include my networking code. The hackage docs mention to share a manager so I've done that. And it also says that for http you have to explicitly close connections, but I don't think I need to do that for httpLbs.

Here's my code.

and it's output:

Getting rid of the regex computations fixed the problem, but it seems that the error happens after the networking and during the regex, presumably because of something I'm doing wrong with http-conduit. Any ideas?

Also, when I try to compile with profiling enabled I get this error:

Indeed, I have not installed profiling libraries for http-conduit and I don't know how.

我试图了解管道概念的不同实现之间的差异。导管和管道之间的区别之一是它们如何将管道融合在一起。导管有

而管道有

如果我理解正确，则使用管道，当两个管道中的任何管道停止时，将返回其结果并停止另一个。使用管道，如果左管道完成，其结果将向下游发送到右管道。

我想知道，管道方法的好处是什么？我希望看到一些示例（最好是真实世界），使用管道和 很容易实现，但>+>使用管道和>->.

我试图了解导管和管道之间的区别。与管道不同，管道有剩菜的概念。剩菜有什么用？我想看看一些剩菜必不可少的例子。

而且由于管道没有剩菜的概念，有没有办法用它们实现类似的行为？