Add `fromBuilder` once `bytestring >= 0.10` is included in Debian stable

[jdnavarro]

I'm opening this issue in order to not forget about the changes depending on the bytestring upgrade in Debian stable: https://github.com/jdnavarro/Haskell-Pipes-ByteString-Library

Meanwhile, I'll try to keep rebasing that branch with further changes introduced in HEAD.

[Gabriella439]

Thanks! I appreciate the time and effort you have put into this.

[michaelt]

I notice that the streaming-commons library has devices for dealing with streams of builders -- with a little class that covers both blaze and bytestring builders, here.

I made a little module to apply it. It is much simpler than the corresponding conduit module

I think, by the way, that streaming-commons doesn't presuppose a recent bytestring -- it seems it is supposed to pull in the bytestring-builder package if you have a pre-builder bytestring. On the other hand streaming-commons presupposes things like zlib and blaze, so it is probably not sensible to try to include something like a Pipes.ByteString.Builder, written this way, in pipes-bytestring

Here's a little test module that uses it:

{-#LANGUAGE OverloadedStrings #-}

import Pipes 
import qualified Pipes.ByteString as PB
import Pipes.ByteString.Builder (fromBuilders) 
import Data.ByteString (ByteString)
import Data.ByteString.Builder
import Data.Streaming.ByteString.Builder.Class
import Data.Monoid
import System.Environment

main :: IO ()
main = do
  xs <- getArgs
  case xs of 
    []        -> runEffect $ builders    1000000 >-> PB.stdout
    ('x':_):_ -> runEffect $ bytes       1000000 >-> PB.stdout
    _         -> runEffect $ viaBuilders 1000000 >-> PB.stdout

builders :: MonadIO m => Int -> Producer ByteString m ()
builders n = fromBuilders defaultStrategy (loop n)
  where
  yn = char8 'y' <> char8 '\n'
  loop 0 = return ()
  loop n = yield yn >> loop (n-1)

bytes :: Monad m => Int -> Producer ByteString m ()
bytes = loop
  where
  yn = "y\n"
  loop 0 = return ()
  loop n = yield  yn >> loop (n-1)

viaBuilders n  = fromBuilders defaultStrategy builders_
  where builders_ = for (bytes n) (yield . byteString) 

-- $ time ./pipebuilder  >> /dev/null
-- real 0m2.005s
-- $ time ./pipebuilder x >> /dev/null
-- real 0m0.565s
-- $ time ./pipebuilder y >> /dev/null
-- real 0m2.135s

-- $ time ./pipebuilder 
-- real 0m6.689s
-- $ time ./pipebuilder x 
-- real 0m14.472s
-- time ./pipebuilder y
-- real 0m7.081s

The programs that go by way of builders are somewhat slower (0m0.234s vs 0m0.191s) if I pipe the output to /dev/null. But they are both much faster (0m2.153s vs 0m4.736s) if I just let the output spill into my (os x) terminal, where some opaque buffering nonsense is at work. They also use much less cpu as they go along than plain >-> PB.stdout does -- to judge just looking at 'top' ; this doesn't change if the number of lines is increased, whereas sending all the little bytestrings directly to stdout tends to become pathological at a certain point, which is what first got me looking into this.

[michaelt]

Sorry, I forgot to link the draft helper module I was talking about https://github.com/michaelt/pipes-builder/blob/master/Pipes/ByteString/Builder.hs

[Gabriella439]

Yeah, so we can merge in all the Builder stuff now if you want

[jdnavarro]

The current version of bytestring in Debian Jessie is 0.10.0.2 which includes the Builder module. I guess we can safely increase the lower version bounds for bytestring.

[Gabriella439]

Yeah, you can definitely increase the lower bounds now

[jdnavarro]

Does it make sense to merge my previous fromBuilder function? I think having just @michaelt implementation is better for a clean Builder API.

[Gabriella439]

Alright, we can merge @michaelt 's code

[michaelt]

Sorry, I saw this when I was abroad. My thought was that the streaming-commons dependency would be too much. If not, I will prepare a proper patch; streaming-commons is in any case a meritorious library. I could also try to reason the matter out and inline something more specialized; but on the other hand, one would prefer that bug-control be in the hands of streaming-commons.