[jira] [Created] (FLINK-2084) Create a dedicated streaming mode

[Shang Yuanchun (Jira)]

Stephan Ewen created FLINK-2084:
-----------------------------------

             Summary: Create a dedicated streaming mode
                 Key: FLINK-2084
                 URL: https://issues.apache.org/jira/browse/FLINK-2084
             Project: Flink
          Issue Type: Improvement
          Components: Distributed Runtime
    Affects Versions: 0.9
            Reporter: Stephan Ewen
            Assignee: Stephan Ewen
             Fix For: 0.9


As per discussion on the mailing list

http://mail-archives.apache.org/mod_mbox/flink-dev/201505.mbox/browser

 - We add a dedicated streaming mode for now. The streaming mode supersedes the batch mode, so it can run both type of programs.

 - The streaming mode sets the memory manager to "lazy allocation".
    -> So long as it runs pure streaming jobs, the full heap will be
available to window buffers and UDFs.
    -> Batch programs can still run, so mixed workloads are not prevented.
Batch programs are a bit less robust there, because the memory manager does not pre-allocate memory. UDFs can eat into Flink's memory portion.

 - The streaming mode starts the necessary configured components/services for state backups


Over the next versions, we want to bring these things together:
  - use the managed memory for window buffers
  - on-demand starting of the state backend

Then, we deprecate the streaming mode, let both modes start the cluster in the same way.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)