Janino related question

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Janino related question

Gábor Horváth
Hi!

I have created a proof of concept implementation of my GSoC project [1]:
introducing code generation to the serializers. The code is available here
[2]. Unfortunately I have run into a problem that I am unable to debug. I
generated some code that I compiled using the Janino compiler [4]. I did
not get any exception during the compilation, and I was able to get the
Class from the compiler. Unfortunately I am unable to get the list of the
constructors from the Class and can not debug what is the source of the
problem. There are no exceptions thrown, no errors displayed in the event
log, no fatal error logfile generated by the JVM, but the process
terminates (at line [3]). I suspect that the Class generated by the Janino
compiler is invalid, but it does not emit any exceptions or warnings. I am
using Oracle JDK 8, on Arch Linux. Have you seen any similar problem? Is
there a way to debug a situation like this? Is there a way to get extra
diagnostics from the Janino compiler?

@aljoscha: you have some experience with the Janino compiler, what do you
think?

Thanks in advance,
Gábor

[1] https://issues.apache.org/jira/browse/FLINK-3599
[2] https://github.com/Xazax-hun/flink/commits/serializer_codegen
[3]
https://github.com/Xazax-hun/flink/commit/af499d5bebe5c1dba6b970977852318346636a8f#diff-7a2080515bac95cec58032655867d6cfR57
[4] http://unkrig.de/w/Janino
Reply | Threaded
Open this post in threaded view
|

Re: Janino related question

Márton Balassi
Hi Gabor,

I have checked out your branch and tried debugging WordCountPojo to
reproduce the behaviour. I am on a Mac with jdk1.8.0_91. I have received
the following error when trying to access the constructors of the class in
question:

Exception in thread "main" java.lang.VerifyError: (class:
org/apache/flink/api/java/typeutils/runtime/generated/Word_GeneratedSerializer,
method: deserialize signature:
(Lorg/apache/flink/core/memory/DataInputView;)Lorg/apache/flink/examples/java/wordcount/WordCountPojo$Word;)
Register 3 contains wrong type
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getConstructors(Class.java:1651)
at
org.apache.flink.api.java.typeutils.runtime.PojoSerializerGenerator.createSerializer(PojoSerializerGenerator.java:57)
at
org.apache.flink.api.java.typeutils.PojoTypeInfo.createSerializer(PojoTypeInfo.java:306)
at org.apache.flink.api.java.DataSet.collect(DataSet.java:407)
at org.apache.flink.api.java.DataSet.print(DataSet.java:1605)
at
org.apache.flink.examples.java.wordcount.WordCountPojo.main(WordCountPojo.java:119)
Disconnected from the target VM, address: '127.0.0.1:52140', transport:
'socket'

I hope this helps.

Marton

On Sun, May 8, 2016 at 2:59 PM, Gábor Horváth <[hidden email]> wrote:

> Hi!
>
> I have created a proof of concept implementation of my GSoC project [1]:
> introducing code generation to the serializers. The code is available here
> [2]. Unfortunately I have run into a problem that I am unable to debug. I
> generated some code that I compiled using the Janino compiler [4]. I did
> not get any exception during the compilation, and I was able to get the
> Class from the compiler. Unfortunately I am unable to get the list of the
> constructors from the Class and can not debug what is the source of the
> problem. There are no exceptions thrown, no errors displayed in the event
> log, no fatal error logfile generated by the JVM, but the process
> terminates (at line [3]). I suspect that the Class generated by the Janino
> compiler is invalid, but it does not emit any exceptions or warnings. I am
> using Oracle JDK 8, on Arch Linux. Have you seen any similar problem? Is
> there a way to debug a situation like this? Is there a way to get extra
> diagnostics from the Janino compiler?
>
> @aljoscha: you have some experience with the Janino compiler, what do you
> think?
>
> Thanks in advance,
> Gábor
>
> [1] https://issues.apache.org/jira/browse/FLINK-3599
> [2] https://github.com/Xazax-hun/flink/commits/serializer_codegen
> [3]
>
> https://github.com/Xazax-hun/flink/commit/af499d5bebe5c1dba6b970977852318346636a8f#diff-7a2080515bac95cec58032655867d6cfR57
> [4] http://unkrig.de/w/Janino
>
Reply | Threaded
Open this post in threaded view
|

Re: Janino related question

Aljoscha Krettek-2
Hi Gábor,
I used it, yes, but I never encountered such a problem. Let's hope that the
error message Márton discovered helps. :-)

Cheers,
Aljoscha

On Mon, 9 May 2016 at 11:38 Márton Balassi <[hidden email]> wrote:

> Hi Gabor,
>
> I have checked out your branch and tried debugging WordCountPojo to
> reproduce the behaviour. I am on a Mac with jdk1.8.0_91. I have received
> the following error when trying to access the constructors of the class in
> question:
>
> Exception in thread "main" java.lang.VerifyError: (class:
>
> org/apache/flink/api/java/typeutils/runtime/generated/Word_GeneratedSerializer,
> method: deserialize signature:
>
> (Lorg/apache/flink/core/memory/DataInputView;)Lorg/apache/flink/examples/java/wordcount/WordCountPojo$Word;)
> Register 3 contains wrong type
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructors(Class.java:1651)
> at
>
> org.apache.flink.api.java.typeutils.runtime.PojoSerializerGenerator.createSerializer(PojoSerializerGenerator.java:57)
> at
>
> org.apache.flink.api.java.typeutils.PojoTypeInfo.createSerializer(PojoTypeInfo.java:306)
> at org.apache.flink.api.java.DataSet.collect(DataSet.java:407)
> at org.apache.flink.api.java.DataSet.print(DataSet.java:1605)
> at
>
> org.apache.flink.examples.java.wordcount.WordCountPojo.main(WordCountPojo.java:119)
> Disconnected from the target VM, address: '127.0.0.1:52140', transport:
> 'socket'
>
> I hope this helps.
>
> Marton
>
> On Sun, May 8, 2016 at 2:59 PM, Gábor Horváth <[hidden email]> wrote:
>
> > Hi!
> >
> > I have created a proof of concept implementation of my GSoC project [1]:
> > introducing code generation to the serializers. The code is available
> here
> > [2]. Unfortunately I have run into a problem that I am unable to debug. I
> > generated some code that I compiled using the Janino compiler [4]. I did
> > not get any exception during the compilation, and I was able to get the
> > Class from the compiler. Unfortunately I am unable to get the list of the
> > constructors from the Class and can not debug what is the source of the
> > problem. There are no exceptions thrown, no errors displayed in the event
> > log, no fatal error logfile generated by the JVM, but the process
> > terminates (at line [3]). I suspect that the Class generated by the
> Janino
> > compiler is invalid, but it does not emit any exceptions or warnings. I
> am
> > using Oracle JDK 8, on Arch Linux. Have you seen any similar problem? Is
> > there a way to debug a situation like this? Is there a way to get extra
> > diagnostics from the Janino compiler?
> >
> > @aljoscha: you have some experience with the Janino compiler, what do you
> > think?
> >
> > Thanks in advance,
> > Gábor
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-3599
> > [2] https://github.com/Xazax-hun/flink/commits/serializer_codegen
> > [3]
> >
> >
> https://github.com/Xazax-hun/flink/commit/af499d5bebe5c1dba6b970977852318346636a8f#diff-7a2080515bac95cec58032655867d6cfR57
> > [4] http://unkrig.de/w/Janino
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Janino related question

Gábor Horváth
Hi,

Thank you for your support, I was able to solve this issue :)

Gábor

On 9 May 2016 at 12:15, Aljoscha Krettek <[hidden email]> wrote:

> Hi Gábor,
> I used it, yes, but I never encountered such a problem. Let's hope that the
> error message Márton discovered helps. :-)
>
> Cheers,
> Aljoscha
>
> On Mon, 9 May 2016 at 11:38 Márton Balassi <[hidden email]>
> wrote:
>
> > Hi Gabor,
> >
> > I have checked out your branch and tried debugging WordCountPojo to
> > reproduce the behaviour. I am on a Mac with jdk1.8.0_91. I have received
> > the following error when trying to access the constructors of the class
> in
> > question:
> >
> > Exception in thread "main" java.lang.VerifyError: (class:
> >
> >
> org/apache/flink/api/java/typeutils/runtime/generated/Word_GeneratedSerializer,
> > method: deserialize signature:
> >
> >
> (Lorg/apache/flink/core/memory/DataInputView;)Lorg/apache/flink/examples/java/wordcount/WordCountPojo$Word;)
> > Register 3 contains wrong type
> > at java.lang.Class.getDeclaredConstructors0(Native Method)
> > at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> > at java.lang.Class.getConstructors(Class.java:1651)
> > at
> >
> >
> org.apache.flink.api.java.typeutils.runtime.PojoSerializerGenerator.createSerializer(PojoSerializerGenerator.java:57)
> > at
> >
> >
> org.apache.flink.api.java.typeutils.PojoTypeInfo.createSerializer(PojoTypeInfo.java:306)
> > at org.apache.flink.api.java.DataSet.collect(DataSet.java:407)
> > at org.apache.flink.api.java.DataSet.print(DataSet.java:1605)
> > at
> >
> >
> org.apache.flink.examples.java.wordcount.WordCountPojo.main(WordCountPojo.java:119)
> > Disconnected from the target VM, address: '127.0.0.1:52140', transport:
> > 'socket'
> >
> > I hope this helps.
> >
> > Marton
> >
> > On Sun, May 8, 2016 at 2:59 PM, Gábor Horváth <[hidden email]>
> wrote:
> >
> > > Hi!
> > >
> > > I have created a proof of concept implementation of my GSoC project
> [1]:
> > > introducing code generation to the serializers. The code is available
> > here
> > > [2]. Unfortunately I have run into a problem that I am unable to
> debug. I
> > > generated some code that I compiled using the Janino compiler [4]. I
> did
> > > not get any exception during the compilation, and I was able to get the
> > > Class from the compiler. Unfortunately I am unable to get the list of
> the
> > > constructors from the Class and can not debug what is the source of the
> > > problem. There are no exceptions thrown, no errors displayed in the
> event
> > > log, no fatal error logfile generated by the JVM, but the process
> > > terminates (at line [3]). I suspect that the Class generated by the
> > Janino
> > > compiler is invalid, but it does not emit any exceptions or warnings. I
> > am
> > > using Oracle JDK 8, on Arch Linux. Have you seen any similar problem?
> Is
> > > there a way to debug a situation like this? Is there a way to get extra
> > > diagnostics from the Janino compiler?
> > >
> > > @aljoscha: you have some experience with the Janino compiler, what do
> you
> > > think?
> > >
> > > Thanks in advance,
> > > Gábor
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-3599
> > > [2] https://github.com/Xazax-hun/flink/commits/serializer_codegen
> > > [3]
> > >
> > >
> >
> https://github.com/Xazax-hun/flink/commit/af499d5bebe5c1dba6b970977852318346636a8f#diff-7a2080515bac95cec58032655867d6cfR57
> > > [4] http://unkrig.de/w/Janino
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Janino related question

Gábor Horváth
Hi,

Unfortunately I found another mysterious issue. If I compile the generated code using Janino, the readObject method will not triggered after an object of that class is deserialized. In case I am compiling the class using Oracle JDK, the readObject method will be triggered as expected. Did any of you see similar phenomenon? What would be the best way to debug this?

I have attached the generated Java code.

Thanks in advance,
Gábor

On 9 May 2016 at 20:41, Gábor Horváth <[hidden email]> wrote:
Hi,

Thank you for your support, I was able to solve this issue :)

Gábor

On 9 May 2016 at 12:15, Aljoscha Krettek <[hidden email]> wrote:
Hi Gábor,
I used it, yes, but I never encountered such a problem. Let's hope that the
error message Márton discovered helps. :-)

Cheers,
Aljoscha

On Mon, 9 May 2016 at 11:38 Márton Balassi <[hidden email]> wrote:

> Hi Gabor,
>
> I have checked out your branch and tried debugging WordCountPojo to
> reproduce the behaviour. I am on a Mac with jdk1.8.0_91. I have received
> the following error when trying to access the constructors of the class in
> question:
>
> Exception in thread "main" java.lang.VerifyError: (class:
>
> org/apache/flink/api/java/typeutils/runtime/generated/Word_GeneratedSerializer,
> method: deserialize signature:
>
> (Lorg/apache/flink/core/memory/DataInputView;)Lorg/apache/flink/examples/java/wordcount/WordCountPojo$Word;)
> Register 3 contains wrong type
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructors(Class.java:1651)
> at
>
> org.apache.flink.api.java.typeutils.runtime.PojoSerializerGenerator.createSerializer(PojoSerializerGenerator.java:57)
> at
>
> org.apache.flink.api.java.typeutils.PojoTypeInfo.createSerializer(PojoTypeInfo.java:306)
> at org.apache.flink.api.java.DataSet.collect(DataSet.java:407)
> at org.apache.flink.api.java.DataSet.print(DataSet.java:1605)
> at
>
> org.apache.flink.examples.java.wordcount.WordCountPojo.main(WordCountPojo.java:119)
> Disconnected from the target VM, address: '127.0.0.1:52140', transport:
> 'socket'
>
> I hope this helps.
>
> Marton
>
> On Sun, May 8, 2016 at 2:59 PM, Gábor Horváth <[hidden email]> wrote:
>
> > Hi!
> >
> > I have created a proof of concept implementation of my GSoC project [1]:
> > introducing code generation to the serializers. The code is available
> here
> > [2]. Unfortunately I have run into a problem that I am unable to debug. I
> > generated some code that I compiled using the Janino compiler [4]. I did
> > not get any exception during the compilation, and I was able to get the
> > Class from the compiler. Unfortunately I am unable to get the list of the
> > constructors from the Class and can not debug what is the source of the
> > problem. There are no exceptions thrown, no errors displayed in the event
> > log, no fatal error logfile generated by the JVM, but the process
> > terminates (at line [3]). I suspect that the Class generated by the
> Janino
> > compiler is invalid, but it does not emit any exceptions or warnings. I
> am
> > using Oracle JDK 8, on Arch Linux. Have you seen any similar problem? Is
> > there a way to debug a situation like this? Is there a way to get extra
> > diagnostics from the Janino compiler?
> >
> > @aljoscha: you have some experience with the Janino compiler, what do you
> > think?
> >
> > Thanks in advance,
> > Gábor
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-3599
> > [2] https://github.com/Xazax-hun/flink/commits/serializer_codegen
> > [3]
> >
> >
> https://github.com/Xazax-hun/flink/commit/af499d5bebe5c1dba6b970977852318346636a8f#diff-7a2080515bac95cec58032655867d6cfR57
> > [4] http://unkrig.de/w/Janino
> >
>



janino_generated.java (10K) Download Attachment