[jira] [Created] (FLINK-21507) Given a sequence of multiple custom stream partitioning operations only last transformation in sequence is applied while others are dropped

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-21507) Given a sequence of multiple custom stream partitioning operations only last transformation in sequence is applied while others are dropped

Shang Yuanchun (Jira)
Iaroslav Zeigerman created FLINK-21507:
------------------------------------------

             Summary: Given a sequence of multiple custom stream partitioning operations only last transformation in sequence is applied while others are dropped
                 Key: FLINK-21507
                 URL: https://issues.apache.org/jira/browse/FLINK-21507
             Project: Flink
          Issue Type: Bug
          Components: API / DataStream, API / Scala
    Affects Versions: 1.11.0
            Reporter: Iaroslav Zeigerman


When I use multiple custom partitioning operations in a row like this:
{code:java}
stream
  .partitionCustom(<custom_partitioner1>, _.key)
  .process(...)
  .partitionCustom(<custom_partitioner2>, _.key)
  .process(...)
  ....{code}
I see that only last partitioning operation (custom_partitioner2) is reflected in the DAG while the 1st one is ignored entirely.

I've also confirmed that the 1st partitioning wasn't applied at runtime from application logs.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)