[jira] [Created] (FLINK-9396) Allow streaming sources (especially Files, Kafka) to set max queued input elements

[Shang Yuanchun (Jira)]

Josh Lemer created FLINK-9396:
---------------------------------

             Summary: Allow streaming sources (especially Files, Kafka) to set max queued input elements
                 Key: FLINK-9396
                 URL: https://issues.apache.org/jira/browse/FLINK-9396
             Project: Flink
          Issue Type: Task
          Components: DataStream API
            Reporter: Josh Lemer


In some applications, there can be situations where a file source queues up so many files, that when it is time for a checkpoint to start, the checkpoint takes way too long. It would be nice if we could limit the total queued files to read to some number, to place an upper bound on how long checkpoints take.

See http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Fwd-Decrease-initial-source-read-speed-td20207.html for previous questions about this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)