taskmanager.heap.mb recommendation

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

taskmanager.heap.mb recommendation

Greg Hogan
Hi all,

The Flink docs recommend "if the cluster is exclusively running Flink, the
total amount of available memory per machine minus some memory for the
operating system (maybe 1-2 GB) is a good value" [1].

Would it be worthwhile to recommend a higher memory reserve and note that
the file write and read-ahead caches must also fit in memory?

[1] https://ci.apache.org/projects/flink/flink-docs-master/setup/config.html

Greg
Reply | Threaded
Open this post in threaded view
|

Re: taskmanager.heap.mb recommendation

Gyula Fóra
I agree that this might actually be pretty bad recommendation especially
for streaming programs depending on the state backend.

Gyula
On Thu, Apr 28, 2016 at 11:18 PM Greg Hogan <[hidden email]> wrote:

> Hi all,
>
> The Flink docs recommend "if the cluster is exclusively running Flink, the
> total amount of available memory per machine minus some memory for the
> operating system (maybe 1-2 GB) is a good value" [1].
>
> Would it be worthwhile to recommend a higher memory reserve and note that
> the file write and read-ahead caches must also fit in memory?
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/setup/config.html
>
> Greg
>