[jira] [Created] (FLINK-2837) FlinkTopologyBuilder cannot handle multiple input streams

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-2837) FlinkTopologyBuilder cannot handle multiple input streams

Shang Yuanchun (Jira)
Matthias J. Sax created FLINK-2837:
--------------------------------------

             Summary: FlinkTopologyBuilder cannot handle multiple input streams
                 Key: FLINK-2837
                 URL: https://issues.apache.org/jira/browse/FLINK-2837
             Project: Flink
          Issue Type: Bug
          Components: Storm Compatibility
            Reporter: Matthias J. Sax
            Assignee: Matthias J. Sax


FlinkTopologyBuilder cannot handle multiple input streams correctly. Instead of union the incoming streams, it replicates the consuming bolt and each (logical) instance processes one of the input streams.

For example:
{noformat}
final FlinkTopologyBuilder builder = new FlinkTopologyBuilder();

builder.setSpout(spoutId1, new FiniteRandomSpout(0, 10));
builder.setSpout(spoutId2, new FiniteRandomSpout(1, 8));
builder.setSpout(spoutId3, new FiniteRandomSpout(2, 13));

builder.setBolt(boltId, new MergerBolt())
        .shuffleGrouping(spoutId1)
        .shuffleGrouping(spoutId2)
        .shuffleGrouping(spoutId3);

builder.setBolt("sink", new BoltPrintSink(new SimpleOutputFormatter()))
        .shuffleGrouping(boltId);
{noformat}

will only print the data from a single source instead of all sources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)