3

我们得到sourceFile一个 ByteString 流。

参考我的另一个问题“将多个源/生产者组合成一个”，我能够使用 获取 (StdGen, ByteString)的源ZipSink，sourceFile以及生成无限 StdGen 流的自定义源。

我想要实现的是将每个 StdGen 与一个字节的 ByteString 配对，但是在我当前的实现中，我得到一个 StdGen 与来自sourceFile.

我已经研究了Conduit.Binary's 的isolate功能，但是当我使用如下时它似乎对我不起作用：

{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE OverloadedStrings #-}

import System.Random (StdGen(..), split, newStdGen, randomR)
import ClassyPrelude.Conduit as Prelude
import Control.Monad.Trans.Resource (runResourceT, ResourceT(..))
import qualified Data.ByteString as BS
import Data.Conduit.Binary (isolate)

-- generate a infinite source of random number seeds
sourceStdGen :: MonadIO m => Source m StdGen
sourceStdGen = do
    g <- liftIO newStdGen
    loop g
    where loop gin = do
            let g' = fst (split gin)
            yield gin
            loop g'

-- combine the sources into one
sourceInput :: (MonadResource m, MonadIO m) => FilePath -> Source m (StdGen, ByteString)
sourceInput fp = getZipSource $ (,)
    <$> ZipSource sourceStdGen
    <*> ZipSource (sourceFile fp $= isolate 1)

-- a simple conduit, which generates a random number from provide StdGen
-- and append the byte value to the provided ByteString
simpleConduit :: Conduit (StdGen, ByteString) (ResourceT IO) ByteString
simpleConduit = mapC process 

process :: (StdGen, ByteString) -> ByteString
process (g, bs) =
    let rnd = fst $ randomR (40,50) g
    in bs ++ pack [rnd]

main :: IO ()
main = do
    runResourceT $ sourceInput "test.txt" $$ simpleConduit =$ sinkFile "output.txt"

在 Conduit 术语中，我认为isolate会做一个await，产生head传入的 ByteString 流，leftOver其余的（将它放回传入流的队列）。基本上，我要做的是将传入的 ByteString 流切成字节块。

我是否正确使用它？如果isolate不是我应该使用的功能，那么任何人都可以提供另一个将其拆分为任意字节块的功能吗？

4

2 回答 2

2

如果我理解正确，你想要这样的东西：

import System.Random (StdGen, split, newStdGen, randomR)
import qualified Data.ByteString as BS
import Data.Conduit 
import Data.ByteString (ByteString, pack, unpack, singleton)
import Control.Monad.Trans (MonadIO (..))
import Data.List (unfoldr)
import qualified Data.Conduit.List as L
import Data.Monoid ((<>))

input :: MonadIO m => FilePath -> Source m (StdGen, ByteString)
input path = do 
  gs <- unfoldr (Just . split) `fmap` liftIO newStdGen 
  bs <- (map singleton . unpack) `fmap` liftIO (BS.readFile path)
  mapM_ yield (zip gs bs)

output :: Monad m => Sink (StdGen, ByteString) m ByteString
output = L.foldMap (\(g, bs) -> let rnd = fst $ randomR (97,122) g in bs <> pack [rnd])

main :: IO ()
main = (input "in.txt" $$ output) >>=  BS.writeFile "out.txt"

省略 s 可能更有效map singleton，你也可以直接使用Word8s 并在最后转换回ByteString。

于 2014-04-29T17:27:42.947 回答
0

我设法自己编写了一个管道 ( condWord)，它将传入的 ByteString 拆分为 Word8 块。我不确定我是否在这里重新发明轮子。

为了获得我想要的行为，我只是简单地condWord使用sourceFile.

{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE OverloadedStrings #-}

import System.Random (StdGen(..), split, newStdGen, randomR)
import ClassyPrelude.Conduit as Prelude
import Control.Monad.Trans.Resource (runResourceT, ResourceT(..))
import qualified Data.ByteString as BS
import Data.Conduit.Binary (isolate)
import Data.Maybe (fromJust)

-- generate a infinite source of random number seeds
sourceStdGen :: MonadIO m => Source m StdGen
sourceStdGen = do
    g <- liftIO newStdGen
    loop g
    where loop gin = do
            let g' = fst (split gin)
            yield gin
            loop g'

-- combine the sources into one
sourceInput :: (MonadResource m, MonadIO m) => FilePath -> Source m (StdGen, Word8)
sourceInput fp = getZipSource $ (,)
    <$> ZipSource sourceStdGen
    <*> ZipSource (sourceFile fp $= condWord)

-- a simple conduit, which generates a random number from provide StdGen
-- and append the byte value to the provided ByteString
simpleConduit :: Conduit (StdGen, Word8) (ResourceT IO) ByteString
simpleConduit = mapC process 

process :: (StdGen, Word8) -> ByteString
process (g, ch) =
    let rnd = fst $ randomR (97,122) g
    in pack [fromIntegral ch, rnd]

condWord :: (Monad m) => Conduit ByteString m Word8
condWord = do
    bs <- await
    case bs of
        Just bs' -> do
            if (null bs')
                then return ()
                else do
                    let (h, t) = fromJust $ BS.uncons bs'
                    yield h
                    leftover t 
                    condWord
        _ -> return ()

main :: IO ()
main = do
    runResourceT $ sourceInput "test.txt" $$ simpleConduit =$ sinkFile "output.txt"
于 2014-04-30T10:02:02.570 回答