This project is read-only.

Microsoft.Tpl.Dataflow

Nov 27, 2014 at 5:35 PM
Edited Nov 28, 2014 at 8:10 AM
Hey guys

Check out this NuGet package by Microsoft: https://www.nuget.org/packages/Microsoft.Tpl.Dataflow/4.5.23

I can't say much about it yet. I'm still doing the tutorials, but I'm confident that this is something that we want to use. Especially for the deepest layers of our code, which does nothing other than allow data to flow back and forth.

http://msdn.microsoft.com/en-us/library/hh228603(v=vs.110).aspx
http://www.michaelfcollins3.me/blog/2013/07/18/introduction-to-the-tpl-dataflow-framework.html
http://taskmatics.com/blog/simplifying-producer-consumer-processing-with-tpl-dataflow-structures/
Nov 28, 2014 at 8:38 AM
I can immediately think of some use cases for the TransformBlock type in the TPL Dataflow library
  1. An optional TransformBlock that injects "Accept-Encoding: gzip" into a request (could be configuration-based?)
  2. A companion block that decompresses a response stream if the response header contains "Content-Encoding: gzip" (but otherwise does nothing)
  3. One TransformBlock for each JSON schema, where each block wraps JsonConvert to convert only a specific type (as opposed to JsonConvert itself, which can serialize any type. I would argue that that's a violation of the single responsibility principle).
Nov 28, 2014 at 1:22 PM
Turns out that client code can benefit as well.

https://gist.github.com/StevenLiekens/a706cb143e0a0b064e15

My first working experiment uses the BufferBlock, BatchBlock and TransformBlock types to download all items from /v2/items by identifier.

The code first gets a list of all ids, then inserts each identifier into the Dataflow pipeline for downloading. This is pretty standard, but the Dataflow library introduces a twist. The next block in the pipeline doesn't just send one HTTP request for each identifier. It uses the BatchBlock to create batches of up to 200 ids. Each batch is then grouped into a single HTTP request. Finally, the results are copied to the output BufferBlock and execution is handed over to client code.
Dec 1, 2014 at 12:53 PM
uhhh shiny. I like it, I really like it.

if you want you can implement it. This would be the perfect time to start with our new branching model. Create a new branch and you can start. I'll look after the release today evening.
Dec 1, 2014 at 1:05 PM
Edited Dec 6, 2014 at 2:20 PM
I'm still finding errors in the binaries for 0.9.12, so don't move them out of "planning" just yet. I'll upload fixed binaries later today.

edit

New binaries are up (finally). Version 0.9.12 it still marked as "planning", but it should be good to go now. It's pretty stable and feature-complete (i.e. supports all /v1 and /v2 services and also MumbleLink).