[Issue] No Class Definition Found Error

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Issue] No Class Definition Found Error

Vinay Patil
Hi Guys,

I have deployed my application on a cluster, however when I try to run the
application it throws *NoClassDefFoundError for KeyedDeserializationSchema*,
all the dependencies are provided correctly since I have run it on a
different standalone node.

Please Help

Regards,
Vinay Patil
Reply | Threaded
Open this post in threaded view
|

Re: [Issue] No Class Definition Found Error

THORMAN, ROBERT D
How did you “provide” the dependencies?  Did you use the –C <URL> parameter when you submitted your job?

On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote:

>Hi Guys,
>
>I have deployed my application on a cluster, however when I try to run the
>application it throws *NoClassDefFoundError for KeyedDeserializationSchema*,
>all the dependencies are provided correctly since I have run it on a
>different standalone node.
>
>Please Help
>
>Regards,
>Vinay Patil

Reply | Threaded
Open this post in threaded view
|

Re: [Issue] No Class Definition Found Error

Robert Metzger
Are you using Maven for building your job jar?
If yes, can you post your pom file on the mailing list?

On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]> wrote:

> How did you “provide” the dependencies?  Did you use the –C <URL>
> parameter when you submitted your job?
>
> On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote:
>
> >Hi Guys,
> >
> >I have deployed my application on a cluster, however when I try to run the
> >application it throws *NoClassDefFoundError for
> KeyedDeserializationSchema*,
> >all the dependencies are provided correctly since I have run it on a
> >different standalone node.
> >
> >Please Help
> >
> >Regards,
> >Vinay Patil
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Issue] No Class Definition Found Error

Vinay Patil
Hi Robert,

Yes we are using maven for building the jar, I have deployed both jar with dependencies  and without dependencies.

I actually cannot share the pom since it is on the client machine.
But all the dependencies required are there, I have attached a sample pom file which is similar to the pom we are using

Regards,
Vinay Patil

+91-800-728-4749

On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]> wrote:
Are you using Maven for building your job jar?
If yes, can you post your pom file on the mailing list?

On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]> wrote:

> How did you “provide” the dependencies?  Did you use the –C <URL>
> parameter when you submitted your job?
>
> On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote:
>
> >Hi Guys,
> >
> >I have deployed my application on a cluster, however when I try to run the
> >application it throws *NoClassDefFoundError for
> KeyedDeserializationSchema*,
> >all the dependencies are provided correctly since I have run it on a
> >different standalone node.
> >
> >Please Help
> >
> >Regards,
> >Vinay Patil
>
>


sample_pom.txt (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Issue] No Class Definition Found Error

Vinay Patil
Issue resolved.

Created the uber jar (fat jar) as shown in the flink-quickstart guide.
My bad, I should have checked that first.

However now I am facing NullPointerException , consider the following
example (this is the dummy code , since I cannot share the actual code):

public class App implements MapFunction<String, List<String>> {
     private static TempClass tempClass;

     public App() {
          tempClass = new TempClass();
     }

     @Override
public List<String> map(String dummyString) throws Exception {
                 *tempClass.doSomething(dummyString); // getting NPE here*
      }
}

I am not getting NPE when I run it on a single machine, however when I run
on a cluster , it gives me NPE.
Instantiating tempClass inside the map function will fix the issue ,
however my reviewer suggested not to initialize it inside the operator as
it will get initialized always, so did it in the constructor.


Also, when I looked at the flink examples, you have used static classes and
accessed the functions inside operators, so which is the better approach
instantiating inside the operators or use static class ?



Regards,
Vinay Patil

*+91-800-728-4749*

On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <[hidden email]>
wrote:

> Hi Robert,
>
> Yes we are using maven for building the jar, I have deployed both jar with
> dependencies  and without dependencies.
>
> I actually cannot share the pom since it is on the client machine.
> But all the dependencies required are there, I have attached a sample pom
> file which is similar to the pom we are using
>
> Regards,
> Vinay Patil
>
> *+91-800-728-4749*
>
> On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]>
> wrote:
>
>> Are you using Maven for building your job jar?
>> If yes, can you post your pom file on the mailing list?
>>
>> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]>
>> wrote:
>>
>> > How did you “provide” the dependencies?  Did you use the –C <URL>
>> > parameter when you submitted your job?
>> >
>> > On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote:
>> >
>> > >Hi Guys,
>> > >
>> > >I have deployed my application on a cluster, however when I try to run
>> the
>> > >application it throws *NoClassDefFoundError for
>> > KeyedDeserializationSchema*,
>> > >all the dependencies are provided correctly since I have run it on a
>> > >different standalone node.
>> > >
>> > >Please Help
>> > >
>> > >Regards,
>> > >Vinay Patil
>> >
>> >
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Issue] No Class Definition Found Error

Aljoscha Krettek-2
Hi,
the static field will be null after the code has been shipped to the
cluster. You can use a RichMapFunction instead, there you can define an
open() method to initialize such a field. Please see this part of the doc
for rich functions:
https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html#rich-functions

Cheers,
Aljoscha

On Sat, 11 Jun 2016 at 21:07 Vinay Patil <[hidden email]> wrote:

> Issue resolved.
>
> Created the uber jar (fat jar) as shown in the flink-quickstart guide.
> My bad, I should have checked that first.
>
> However now I am facing NullPointerException , consider the following
> example (this is the dummy code , since I cannot share the actual code):
>
> public class App implements MapFunction<String, List<String>> {
>      private static TempClass tempClass;
>
>      public App() {
>           tempClass = new TempClass();
>      }
>
>      @Override
> public List<String> map(String dummyString) throws Exception {
>                  *tempClass.doSomething(dummyString); // getting NPE here*
>       }
> }
>
> I am not getting NPE when I run it on a single machine, however when I run
> on a cluster , it gives me NPE.
> Instantiating tempClass inside the map function will fix the issue ,
> however my reviewer suggested not to initialize it inside the operator as
> it will get initialized always, so did it in the constructor.
>
>
> Also, when I looked at the flink examples, you have used static classes and
> accessed the functions inside operators, so which is the better approach
> instantiating inside the operators or use static class ?
>
>
>
> Regards,
> Vinay Patil
>
> *+91-800-728-4749*
>
> On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <[hidden email]>
> wrote:
>
> > Hi Robert,
> >
> > Yes we are using maven for building the jar, I have deployed both jar
> with
> > dependencies  and without dependencies.
> >
> > I actually cannot share the pom since it is on the client machine.
> > But all the dependencies required are there, I have attached a sample pom
> > file which is similar to the pom we are using
> >
> > Regards,
> > Vinay Patil
> >
> > *+91-800-728-4749*
> >
> > On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]>
> > wrote:
> >
> >> Are you using Maven for building your job jar?
> >> If yes, can you post your pom file on the mailing list?
> >>
> >> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]>
> >> wrote:
> >>
> >> > How did you “provide” the dependencies?  Did you use the –C <URL>
> >> > parameter when you submitted your job?
> >> >
> >> > On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]> wrote:
> >> >
> >> > >Hi Guys,
> >> > >
> >> > >I have deployed my application on a cluster, however when I try to
> run
> >> the
> >> > >application it throws *NoClassDefFoundError for
> >> > KeyedDeserializationSchema*,
> >> > >all the dependencies are provided correctly since I have run it on a
> >> > >different standalone node.
> >> > >
> >> > >Please Help
> >> > >
> >> > >Regards,
> >> > >Vinay Patil
> >> >
> >> >
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Issue] No Class Definition Found Error

Vinay Patil
Yes correct. Thank you for this suggestion.

This is the first time I am working on distributed application, so having
such basic questions

Now I am getting object is not serializable exception. I have generated the
serialUId for that class.
On Jun 12, 2016 11:28 AM, "Aljoscha Krettek" <[hidden email]> wrote:

> Hi,
> the static field will be null after the code has been shipped to the
> cluster. You can use a RichMapFunction instead, there you can define an
> open() method to initialize such a field. Please see this part of the doc
> for rich functions:
>
> https://ci.apache.org/projects/flink/flink-docs-master/apis/common/index.html#rich-functions
>
> Cheers,
> Aljoscha
>
> On Sat, 11 Jun 2016 at 21:07 Vinay Patil <[hidden email]> wrote:
>
> > Issue resolved.
> >
> > Created the uber jar (fat jar) as shown in the flink-quickstart guide.
> > My bad, I should have checked that first.
> >
> > However now I am facing NullPointerException , consider the following
> > example (this is the dummy code , since I cannot share the actual code):
> >
> > public class App implements MapFunction<String, List<String>> {
> >      private static TempClass tempClass;
> >
> >      public App() {
> >           tempClass = new TempClass();
> >      }
> >
> >      @Override
> > public List<String> map(String dummyString) throws Exception {
> >                  *tempClass.doSomething(dummyString); // getting NPE
> here*
> >       }
> > }
> >
> > I am not getting NPE when I run it on a single machine, however when I
> run
> > on a cluster , it gives me NPE.
> > Instantiating tempClass inside the map function will fix the issue ,
> > however my reviewer suggested not to initialize it inside the operator as
> > it will get initialized always, so did it in the constructor.
> >
> >
> > Also, when I looked at the flink examples, you have used static classes
> and
> > accessed the functions inside operators, so which is the better approach
> > instantiating inside the operators or use static class ?
> >
> >
> >
> > Regards,
> > Vinay Patil
> >
> > *+91-800-728-4749*
> >
> > On Sat, Jun 11, 2016 at 9:32 AM, Vinay Patil <[hidden email]>
> > wrote:
> >
> > > Hi Robert,
> > >
> > > Yes we are using maven for building the jar, I have deployed both jar
> > with
> > > dependencies  and without dependencies.
> > >
> > > I actually cannot share the pom since it is on the client machine.
> > > But all the dependencies required are there, I have attached a sample
> pom
> > > file which is similar to the pom we are using
> > >
> > > Regards,
> > > Vinay Patil
> > >
> > > *+91-800-728-4749*
> > >
> > > On Sat, Jun 11, 2016 at 1:36 AM, Robert Metzger <[hidden email]>
> > > wrote:
> > >
> > >> Are you using Maven for building your job jar?
> > >> If yes, can you post your pom file on the mailing list?
> > >>
> > >> On Fri, Jun 10, 2016 at 7:16 PM, THORMAN, ROBERT D <[hidden email]>
> > >> wrote:
> > >>
> > >> > How did you “provide” the dependencies?  Did you use the –C <URL>
> > >> > parameter when you submitted your job?
> > >> >
> > >> > On 6/10/16, 11:35 AM, "Vinay Patil" <[hidden email]>
> wrote:
> > >> >
> > >> > >Hi Guys,
> > >> > >
> > >> > >I have deployed my application on a cluster, however when I try to
> > run
> > >> the
> > >> > >application it throws *NoClassDefFoundError for
> > >> > KeyedDeserializationSchema*,
> > >> > >all the dependencies are provided correctly since I have run it on
> a
> > >> > >different standalone node.
> > >> > >
> > >> > >Please Help
> > >> > >
> > >> > >Regards,
> > >> > >Vinay Patil
> > >> >
> > >> >
> > >>
> > >
> > >
> >
>