Flink managed memory in cluster mode

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink managed memory in cluster mode

Gyula Fóra-2
Hey guys,

Is it normal that when I start the cluster with start-cluster-streaming.sh
out of the 16gb tm memory 10.6 gb becomes flink managed? (I get pretty much
the same number when I use start-cluster.sh)

I thought that Flink would only use a very small fraction in streaming mode.

Cheers,
Gyula
Reply | Threaded
Open this post in threaded view
|

Re: Flink managed memory in cluster mode

Fabian Hueske-2
Hi Gyula,

I just checked with jconsole that the memory allocation is correct.
However, the log message is a bit misleading. In case of the streaming
mode, the managed memory is lazily allocated and the logged amount is an
upper bound.

Cheers, Fabian

2015-11-12 13:37 GMT+01:00 Gyula Fóra <[hidden email]>:

> Hey guys,
>
> Is it normal that when I start the cluster with start-cluster-streaming.sh
> out of the 16gb tm memory 10.6 gb becomes flink managed? (I get pretty much
> the same number when I use start-cluster.sh)
>
> I thought that Flink would only use a very small fraction in streaming
> mode.
>
> Cheers,
> Gyula
>
Reply | Threaded
Open this post in threaded view
|

Re: Flink managed memory in cluster mode

Gyula Fóra
Thanks Fabian  :)

Fabian Hueske <[hidden email]> ezt írta (időpont: 2015. nov. 12., Cs,
14:03):

> Hi Gyula,
>
> I just checked with jconsole that the memory allocation is correct.
> However, the log message is a bit misleading. In case of the streaming
> mode, the managed memory is lazily allocated and the logged amount is an
> upper bound.
>
> Cheers, Fabian
>
> 2015-11-12 13:37 GMT+01:00 Gyula Fóra <[hidden email]>:
>
> > Hey guys,
> >
> > Is it normal that when I start the cluster with
> start-cluster-streaming.sh
> > out of the 16gb tm memory 10.6 gb becomes flink managed? (I get pretty
> much
> > the same number when I use start-cluster.sh)
> >
> > I thought that Flink would only use a very small fraction in streaming
> > mode.
> >
> > Cheers,
> > Gyula
> >
>