Hi Guys,
I have deployed my application on a cluster, however when I try to run the application it throws *NoClassDefFoundError for KeyedDeserializationSchema*, all the dependencies are provided correctly since I have run it on a different standalone node. Please Help Regards, Vinay Patil |
How did you “provide” the dependencies? Did you use the –C <URL> parameter when you submitted your job?
On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote: >Hi Guys, > >I have deployed my application on a cluster, however when I try to run the >application it throws *NoClassDefFoundError for KeyedDeserializationSchema*, >all the dependencies are provided correctly since I have run it on a >different standalone node. > >Please Help > >Regards, >Vinay Patil |
Are you using Maven for building your job jar?
If yes, can you post your pom file on the mailing list? On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]> wrote: > How did you “provide” the dependencies? Did you use the –C <URL> > parameter when you submitted your job? > > On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote: > > >Hi Guys, > > > >I have deployed my application on a cluster, however when I try to run the > >application it throws *NoClassDefFoundError for > KeyedDeserializationSchema*, > >all the dependencies are provided correctly since I have run it on a > >different standalone node. > > > >Please Help > > > >Regards, > >Vinay Patil > > |
Hi Robert, Yes we are using maven for building the jar, I have deployed both jar with dependencies and without dependencies. I actually cannot share the pom since it is on the client machine. But all the dependencies required are there, I have attached a sample pom file which is similar to the pom we are using Regards, Vinay Patil +91-800-728-4749 On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]> wrote: Are you using Maven for building your job jar? |
Issue resolved.
Created the uber jar (fat jar) as shown in the flink-quickstart guide. My bad, I should have checked that first. However now I am facing NullPointerException , consider the following example (this is the dummy code , since I cannot share the actual code): public class App implements MapFunction<String, List<String>> { private static TempClass tempClass; public App() { tempClass = new TempClass(); } @Override public List<String> map(String dummyString) throws Exception { *tempClass.doSomething(dummyString); // getting NPE here* } } I am not getting NPE when I run it on a single machine, however when I run on a cluster , it gives me NPE. Instantiating tempClass inside the map function will fix the issue , however my reviewer suggested not to initialize it inside the operator as it will get initialized always, so did it in the constructor. Also, when I looked at the flink examples, you have used static classes and accessed the functions inside operators, so which is the better approach instantiating inside the operators or use static class ? Regards, Vinay Patil *+91-800-728-4749* On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <[hidden email]> wrote: > Hi Robert, > > Yes we are using maven for building the jar, I have deployed both jar with > dependencies and without dependencies. > > I actually cannot share the pom since it is on the client machine. > But all the dependencies required are there, I have attached a sample pom > file which is similar to the pom we are using > > Regards, > Vinay Patil > > *+91-800-728-4749* > > On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]> > wrote: > >> Are you using Maven for building your job jar? >> If yes, can you post your pom file on the mailing list? >> >> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]> >> wrote: >> >> > How did you “provide” the dependencies? Did you use the –C <URL> >> > parameter when you submitted your job? >> > >> > On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote: >> > >> > >Hi Guys, >> > > >> > >I have deployed my application on a cluster, however when I try to run >> the >> > >application it throws *NoClassDefFoundError for >> > KeyedDeserializationSchema*, >> > >all the dependencies are provided correctly since I have run it on a >> > >different standalone node. >> > > >> > >Please Help >> > > >> > >Regards, >> > >Vinay Patil >> > >> > >> > > |
Hi,
the static field will be null after the code has been shipped to the cluster. You can use a RichMapFunction instead, there you can define an open() method to initialize such a field. Please see this part of the doc for rich functions: https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html#rich-functions Cheers, Aljoscha On Sat, 11 Jun 2016 at 21:07 Vinay Patil <[hidden email]> wrote: > Issue resolved. > > Created the uber jar (fat jar) as shown in the flink-quickstart guide. > My bad, I should have checked that first. > > However now I am facing NullPointerException , consider the following > example (this is the dummy code , since I cannot share the actual code): > > public class App implements MapFunction<String, List<String>> { > private static TempClass tempClass; > > public App() { > tempClass = new TempClass(); > } > > @Override > public List<String> map(String dummyString) throws Exception { > *tempClass.doSomething(dummyString); // getting NPE here* > } > } > > I am not getting NPE when I run it on a single machine, however when I run > on a cluster , it gives me NPE. > Instantiating tempClass inside the map function will fix the issue , > however my reviewer suggested not to initialize it inside the operator as > it will get initialized always, so did it in the constructor. > > > Also, when I looked at the flink examples, you have used static classes and > accessed the functions inside operators, so which is the better approach > instantiating inside the operators or use static class ? > > > > Regards, > Vinay Patil > > *+91-800-728-4749* > > On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <[hidden email]> > wrote: > > > Hi Robert, > > > > Yes we are using maven for building the jar, I have deployed both jar > with > > dependencies and without dependencies. > > > > I actually cannot share the pom since it is on the client machine. > > But all the dependencies required are there, I have attached a sample pom > > file which is similar to the pom we are using > > > > Regards, > > Vinay Patil > > > > *+91-800-728-4749* > > > > On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]> > > wrote: > > > >> Are you using Maven for building your job jar? > >> If yes, can you post your pom file on the mailing list? > >> > >> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]> > >> wrote: > >> > >> > How did you “provide” the dependencies? Did you use the –C <URL> > >> > parameter when you submitted your job? > >> > > >> > On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote: > >> > > >> > >Hi Guys, > >> > > > >> > >I have deployed my application on a cluster, however when I try to > run > >> the > >> > >application it throws *NoClassDefFoundError for > >> > KeyedDeserializationSchema*, > >> > >all the dependencies are provided correctly since I have run it on a > >> > >different standalone node. > >> > > > >> > >Please Help > >> > > > >> > >Regards, > >> > >Vinay Patil > >> > > >> > > >> > > > > > |
Yes correct. Thank you for this suggestion.
This is the first time I am working on distributed application, so having such basic questions Now I am getting object is not serializable exception. I have generated the serialUId for that class. On Jun 12, 2016 11:28 AM, "Aljoscha Krettek" <[hidden email]> wrote: > Hi, > the static field will be null after the code has been shipped to the > cluster. You can use a RichMapFunction instead, there you can define an > open() method to initialize such a field. Please see this part of the doc > for rich functions: > > https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html#rich-functions > > Cheers, > Aljoscha > > On Sat, 11 Jun 2016 at 21:07 Vinay Patil <[hidden email]> wrote: > > > Issue resolved. > > > > Created the uber jar (fat jar) as shown in the flink-quickstart guide. > > My bad, I should have checked that first. > > > > However now I am facing NullPointerException , consider the following > > example (this is the dummy code , since I cannot share the actual code): > > > > public class App implements MapFunction<String, List<String>> { > > private static TempClass tempClass; > > > > public App() { > > tempClass = new TempClass(); > > } > > > > @Override > > public List<String> map(String dummyString) throws Exception { > > *tempClass.doSomething(dummyString); // getting NPE > here* > > } > > } > > > > I am not getting NPE when I run it on a single machine, however when I > run > > on a cluster , it gives me NPE. > > Instantiating tempClass inside the map function will fix the issue , > > however my reviewer suggested not to initialize it inside the operator as > > it will get initialized always, so did it in the constructor. > > > > > > Also, when I looked at the flink examples, you have used static classes > and > > accessed the functions inside operators, so which is the better approach > > instantiating inside the operators or use static class ? > > > > > > > > Regards, > > Vinay Patil > > > > *+91-800-728-4749* > > > > On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <[hidden email]> > > wrote: > > > > > Hi Robert, > > > > > > Yes we are using maven for building the jar, I have deployed both jar > > with > > > dependencies and without dependencies. > > > > > > I actually cannot share the pom since it is on the client machine. > > > But all the dependencies required are there, I have attached a sample > pom > > > file which is similar to the pom we are using > > > > > > Regards, > > > Vinay Patil > > > > > > *+91-800-728-4749* > > > > > > On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]> > > > wrote: > > > > > >> Are you using Maven for building your job jar? > > >> If yes, can you post your pom file on the mailing list? > > >> > > >> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]> > > >> wrote: > > >> > > >> > How did you “provide” the dependencies? Did you use the –C <URL> > > >> > parameter when you submitted your job? > > >> > > > >> > On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> > wrote: > > >> > > > >> > >Hi Guys, > > >> > > > > >> > >I have deployed my application on a cluster, however when I try to > > run > > >> the > > >> > >application it throws *NoClassDefFoundError for > > >> > KeyedDeserializationSchema*, > > >> > >all the dependencies are provided correctly since I have run it on > a > > >> > >different standalone node. > > >> > > > > >> > >Please Help > > >> > > > > >> > >Regards, > > >> > >Vinay Patil > > >> > > > >> > > > >> > > > > > > > > > |
Free forum by Nabble | Edit this page |