[jira] [Created] (FLINK-16428) Fine-grained network buffer management for backpressure

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-16428) Fine-grained network buffer management for backpressure

Shang Yuanchun (Jira)
Zhijiang created FLINK-16428:
--------------------------------

             Summary: Fine-grained network buffer management for backpressure
                 Key: FLINK-16428
                 URL: https://issues.apache.org/jira/browse/FLINK-16428
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Network
            Reporter: Zhijiang
             Fix For: 1.11.0


It is an umbrella ticket for tracing the progress of this improvement.

This is the second ingredient to solve the “checkpoints under backpressure” problem (together with unaligned checkpoints). It consists of two steps:
 * See if we can use less network memory in general for streaming jobs (with potentially different distribution of floating buffers in the input side)
 * Under backpressure, reduce network memory to have less in-flight data (needs specification of algorithm and experiments)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)