[jira] [Created] (FLINK-18558) Introduce collect iterator with at least once semantics and exactly once semantics without fault tolerance

Posted by Shang Yuanchun (Jira) on Jul 10, 2020; 9:11am
URL: http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/jira-Created-FLINK-18558-Introduce-collect-iterator-with-at-least-once-semantics-and-exactly-once-see-tp43116.html

Caizhi Weng created FLINK-18558:
-----------------------------------

             Summary: Introduce collect iterator with at least once semantics and exactly once semantics without fault tolerance
                 Key: FLINK-18558
                 URL: https://issues.apache.org/jira/browse/FLINK-18558
             Project: Flink
          Issue Type: Improvement
          Components: API / DataStream, Table SQL / API
            Reporter: Caizhi Weng
             Fix For: 1.12.0


Currently {{TableResult#collect}} and {{DataStreamUtils#collect}} can only produce results if users explicitly enable checkpoint for infinite streaming jobs. It would be strange to require the users to do so if they just want to take a look at their data.

We should introduce collect iterator with at least once semantics and exactly once semantics without fault tolerance. When calling the {{collect}} method, we should automatically pick an iterator for the user:
* If the user does not explicitly enable a checkpoint, we use exactly once iterator without fault tolerance. That is to say, the iterator will throw exception once the job restarts.
* If the user explicitly enables an exactly once checkpoint, we use the current implementation of collect iterator.
* If the user explicitly enables an at least once checkpoint, we use the at least once iterator. That is to say, the iterator ignores both checkpoint information and job restarts.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)