How to let Flink 1.7.X run Flink session cluster on YARN in Java 7 default environment

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to let Flink 1.7.X run Flink session cluster on YARN in Java 7 default environment

胡逸才
At present, all YARN clusters adopt JAVA 7 environment.

While trying to use FLINK to handle the deployment of flow processing business scenarios, it was found that FLINK ON YARN mode always failed to perform a session task. The application log of YARN shows Unsupported major. minor version 52.0.

I tried to add env. java. home: < JDK 1.8PATH > in flink-conf. yaml of the mailing list solution. And the startup command adds -yD yarn. taskmanager. env. JAVA_HOME= < JDK1.8PATH>、-yD containerized. master. env. JAVA_HOME= < JDK1.8PATH>, -yD containerized. taskmanager. env. JAVA_HOME= < JDK1.8PATH>. Flink session cluster in YARN can not run Application in JAVA 8 environment.

So can I use Flink1.7.X submit Flink session cluster application in YARN under JAVA 7 environment?
Reply | Threaded
Open this post in threaded view
|

Re: How to let Flink 1.7.X run Flink session cluster on YARN in Java 7 default environment

张军
Flink源码中用到了很多java1.8的特性,所以用jdk1.7是不行的

发自我的 iPhone

> 在 2019年4月26日,17:48,胡逸才 <[hidden email]> 写道:
>
> At present, all YARN clusters adopt JAVA 7 environment.
>
> While trying to use FLINK to handle the deployment of flow processing business scenarios, it was found that FLINK ON YARN mode always failed to perform a session task. The application log of YARN shows Unsupported major. minor version 52.0.
>
> I tried to add env. java. home: < JDK 1.8PATH > in flink-conf. yaml of the mailing list solution. And the startup command adds -yD yarn. taskmanager. env. JAVA_HOME= < JDK1.8PATH>、-yD containerized. master. env. JAVA_HOME= < JDK1.8PATH>, -yD containerized. taskmanager. env. JAVA_HOME= < JDK1.8PATH>. Flink session cluster in YARN can not run Application in JAVA 8 environment.
>
> So can I use Flink1.7.X submit Flink session cluster application in YARN under JAVA 7 environment?
>
>
>  
Reply | Threaded
Open this post in threaded view
|

Re: How to let Flink 1.7.X run Flink session cluster on YARN in Java 7 default environment

张军
Thank you for your reminder, I will pay attention to this issue in the future.

I read some flink source code and saw that there are a lot of new features of java8, such as CompletableFuture and Lambda expressions, which causes flink do not run in jdk 1.7 environment, so you may need to upgrade your jdk to 1.8.




> 在 2019年4月28日,上午8:24,126 <[hidden email]> 写道:
>
> Flink源码中用到了很多java1.8的特性,所以用jdk1.7是不行的
>
> 发自我的 iPhone
>
>> 在 2019年4月26日,17:48,胡逸才 <[hidden email]> 写道:
>>
>> At present, all YARN clusters adopt JAVA 7 environment.
>>
>> While trying to use FLINK to handle the deployment of flow processing business scenarios, it was found that FLINK ON YARN mode always failed to perform a session task. The application log of YARN shows Unsupported major. minor version 52.0.
>>
>> I tried to add env. java. home: < JDK 1.8PATH > in flink-conf. yaml of the mailing list solution. And the startup command adds -yD yarn. taskmanager. env. JAVA_HOME= < JDK1.8PATH>、-yD containerized. master. env. JAVA_HOME= < JDK1.8PATH>, -yD containerized. taskmanager. env. JAVA_HOME= < JDK1.8PATH>. Flink session cluster in YARN can not run Application in JAVA 8 environment.
>>
>> So can I use Flink1.7.X submit Flink session cluster application in YARN under JAVA 7 environment?
>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: How to let Flink 1.7.X run Flink session cluster on YARN in Java 7 default environment

Yaoting Gong
Hi, @胡逸才,  I've met the same problem.   Add some configs as blow will help
you .

env.java.home: /usr/jdk1.8.0_51containerized.master.env.JAVA_HOME:
/usr/jdk1.8.0_51containerized.taskmanager.env.JAVA_HOME:
/usr/jdk1.8.0_51yarn.taskmanager.env.JAVA_HOME:
/usr/jdk1.8.0_51



On Sun, Apr 28, 2019 at 1:58 PM 张军 <[hidden email]> wrote:

> Thank you for your reminder, I will pay attention to this issue in the
> future.
>
> I read some flink source code and saw that there are a lot of new features
> of java8, such as CompletableFuture and Lambda expressions, which causes
> flink do not run in jdk 1.7 environment, so you may need to upgrade your
> jdk to 1.8.
>
>
>
>
> > 在 2019年4月28日,上午8:24,126 <[hidden email]> 写道:
> >
> > Flink源码中用到了很多java1.8的特性,所以用jdk1.7是不行的
> >
> > 发自我的 iPhone
> >
> >> 在 2019年4月26日,17:48,胡逸才 <[hidden email]> 写道:
> >>
> >> At present, all YARN clusters adopt JAVA 7 environment.
> >>
> >> While trying to use FLINK to handle the deployment of flow processing
> business scenarios, it was found that FLINK ON YARN mode always failed to
> perform a session task. The application log of YARN shows Unsupported
> major. minor version 52.0.
> >>
> >> I tried to add env. java. home: < JDK 1.8PATH > in flink-conf. yaml of
> the mailing list solution. And the startup command adds -yD yarn.
> taskmanager. env. JAVA_HOME= < JDK1.8PATH>、-yD containerized. master. env.
> JAVA_HOME= < JDK1.8PATH>, -yD containerized. taskmanager. env. JAVA_HOME= <
> JDK1.8PATH>. Flink session cluster in YARN can not run Application in JAVA
> 8 environment.
> >>
> >> So can I use Flink1.7.X submit Flink session cluster application in
> YARN under JAVA 7 environment?
> >>
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: How to let Flink 1.7.X run Flink session cluster on YARN in Java 7 default environment

Yaoting Gong
Sorry, I didn't see the original message already mentioned in these
configurations.  But it do works for me. I put these into flink-conf.yml,
not try the way you did.

On Mon, Apr 29, 2019 at 9:45 AM Yaoting Gong <[hidden email]>
wrote:

> Hi, @胡逸才,  I've met the same problem.   Add some configs as blow will help
> you .
>
> env.java.home: /usr/jdk1.8.0_51containerized.master.env.JAVA_HOME:
> /usr/jdk1.8.0_51containerized.taskmanager.env.JAVA_HOME: /usr/jdk1.8.0_51yarn.taskmanager.env.JAVA_HOME:
> /usr/jdk1.8.0_51
>
>
>
> On Sun, Apr 28, 2019 at 1:58 PM 张军 <[hidden email]> wrote:
>
>> Thank you for your reminder, I will pay attention to this issue in the
>> future.
>>
>> I read some flink source code and saw that there are a lot of new
>> features of java8, such as CompletableFuture and Lambda expressions, which
>> causes flink do not run in jdk 1.7 environment, so you may need to upgrade
>> your jdk to 1.8.
>>
>>
>>
>>
>> > 在 2019年4月28日,上午8:24,126 <[hidden email]> 写道:
>> >
>> > Flink源码中用到了很多java1.8的特性,所以用jdk1.7是不行的
>> >
>> > 发自我的 iPhone
>> >
>> >> 在 2019年4月26日,17:48,胡逸才 <[hidden email]> 写道:
>> >>
>> >> At present, all YARN clusters adopt JAVA 7 environment.
>> >>
>> >> While trying to use FLINK to handle the deployment of flow processing
>> business scenarios, it was found that FLINK ON YARN mode always failed to
>> perform a session task. The application log of YARN shows Unsupported
>> major. minor version 52.0.
>> >>
>> >> I tried to add env. java. home: < JDK 1.8PATH > in flink-conf. yaml of
>> the mailing list solution. And the startup command adds -yD yarn.
>> taskmanager. env. JAVA_HOME= < JDK1.8PATH>、-yD containerized. master. env.
>> JAVA_HOME= < JDK1.8PATH>, -yD containerized. taskmanager. env. JAVA_HOME= <
>> JDK1.8PATH>. Flink session cluster in YARN can not run Application in JAVA
>> 8 environment.
>> >>
>> >> So can I use Flink1.7.X submit Flink session cluster application in
>> YARN under JAVA 7 environment?
>> >>
>> >>
>> >>
>>
>>