Hi
I remember seeing that using off-heap memory was on Flink’s roadmap as well as a related pull request https://github.com/apache/flink/pull/290 Any update on such effort? Thanks Slim Baltagi |
Hi Slim,
Off-heap memory has been postponed because it's not a pressing but rather a nice-to-have feature. I know that Stephan continued to work on the off-heap memory. I think we can get it in sometime this year. Best, Max On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <[hidden email]> wrote: > Hi > > I remember seeing that using off-heap memory was on Flink’s roadmap as well > as a related pull request https://github.com/apache/flink/pull/290 > > Any update on such effort? > > Thanks > > Slim Baltagi > > > > -- > View this message in context: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-heap-memory-in-Flink-tp7285.html > Sent from the Apache Flink Mailing List archive. mailing list archive at > Nabble.com. > |
The Pull Request is basically ready. I would like to benchmark a bit before
merging it. The on-heap Flink-managed memory classes are highly optimized to be JIT friendly. Just want to make sure that we don't loose that. I have worked a lot on streaming issues lately, so this is still in my backlog. On Fri, Jul 31, 2015 at 2:09 PM, Maximilian Michels <[hidden email]> wrote: > Hi Slim, > > Off-heap memory has been postponed because it's not a pressing but rather a > nice-to-have feature. I know that Stephan continued to work on the off-heap > memory. I think we can get it in sometime this year. > > Best, > Max > > On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <[hidden email]> wrote: > > > Hi > > > > I remember seeing that using off-heap memory was on Flink’s roadmap as > well > > as a related pull request https://github.com/apache/flink/pull/290 > > > > Any update on such effort? > > > > Thanks > > > > Slim Baltagi > > > > > > > > -- > > View this message in context: > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-heap-memory-in-Flink-tp7285.html > > Sent from the Apache Flink Mailing List archive. mailing list archive at > > Nabble.com. > > > |
Hi,
Is there some info, description about how this off-heap memory is managed and its goals? Thanks Dr. Radu Tudoran Research Engineer IT R&D Division HUAWEI TECHNOLOGIES Duesseldorf GmbH European Research Center Riesstrasse 25, 80992 München E-mail: [hidden email] Mobile: +49 15209084330 Telephone: +49 891588344173 HUAWEI TECHNOLOGIES Duesseldorf GmbH Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063, Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Stephan Ewen Sent: Friday, July 31, 2015 4:02 PM To: [hidden email] Subject: Re: Off-heap memory in Flink? The Pull Request is basically ready. I would like to benchmark a bit before merging it. The on-heap Flink-managed memory classes are highly optimized to be JIT friendly. Just want to make sure that we don't loose that. I have worked a lot on streaming issues lately, so this is still in my backlog. On Fri, Jul 31, 2015 at 2:09 PM, Maximilian Michels <[hidden email]> wrote: > Hi Slim, > > Off-heap memory has been postponed because it's not a pressing but > rather a nice-to-have feature. I know that Stephan continued to work > on the off-heap memory. I think we can get it in sometime this year. > > Best, > Max > > On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <[hidden email]> wrote: > > > Hi > > > > I remember seeing that using off-heap memory was on Flink’s roadmap > > as > well > > as a related pull request https://github.com/apache/flink/pull/290 > > > > Any update on such effort? > > > > Thanks > > > > Slim Baltagi > > > > > > > > -- > > View this message in context: > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-hea > p-memory-in-Flink-tp7285.html > > Sent from the Apache Flink Mailing List archive. mailing list > > archive at Nabble.com. > > > |
Hey!
There is the wiki on Flink's memory management: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741525 Greetings, Stephan On Fri, Jul 31, 2015 at 4:06 PM, Radu Tudoran <[hidden email]> wrote: > Hi, > > Is there some info, description about how this off-heap memory is managed > and its goals? > Thanks > > Dr. Radu Tudoran > Research Engineer > IT R&D Division > > > HUAWEI TECHNOLOGIES Duesseldorf GmbH > European Research Center > Riesstrasse 25, 80992 München > > E-mail: [hidden email] > Mobile: +49 15209084330 > Telephone: +49 891588344173 > > HUAWEI TECHNOLOGIES Duesseldorf GmbH > Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com > Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063, > Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN > Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063, > Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On Behalf Of > Stephan Ewen > Sent: Friday, July 31, 2015 4:02 PM > To: [hidden email] > Subject: Re: Off-heap memory in Flink? > > The Pull Request is basically ready. I would like to benchmark a bit > before merging it. > > The on-heap Flink-managed memory classes are highly optimized to be JIT > friendly. Just want to make sure that we don't loose that. > > I have worked a lot on streaming issues lately, so this is still in my > backlog. > > On Fri, Jul 31, 2015 at 2:09 PM, Maximilian Michels <[hidden email]> > wrote: > > > Hi Slim, > > > > Off-heap memory has been postponed because it's not a pressing but > > rather a nice-to-have feature. I know that Stephan continued to work > > on the off-heap memory. I think we can get it in sometime this year. > > > > Best, > > Max > > > > On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <[hidden email]> > wrote: > > > > > Hi > > > > > > I remember seeing that using off-heap memory was on Flink’s roadmap > > > as > > well > > > as a related pull request https://github.com/apache/flink/pull/290 > > > > > > Any update on such effort? > > > > > > Thanks > > > > > > Slim Baltagi > > > > > > > > > > > > -- > > > View this message in context: > > > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-hea > > p-memory-in-Flink-tp7285.html > > > Sent from the Apache Flink Mailing List archive. mailing list > > > archive at Nabble.com. > > > > > > |
Free forum by Nabble | Edit this page |