Question about the ReadableConfigToConfigurationAdapter

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about the ReadableConfigToConfigurationAdapter

felixzheng
Hi everyone,

Recently I failed to run a Flink job when enabling RocksDBStateBackend on
the branch master and 1.10.

The exception stack trace is:

The program finished with the following
exception:org.apache.flink.client.program.ProgramInvocationException:
The main method caused an error: The adapter does not support this
method
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
        at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
Caused by: java.lang.UnsupportedOperationException: The adapter does
not support this method
        at org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
        at org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
        at org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
        at org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
        at java.util.Optional.map(Optional.java:215)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
        at org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
        ... 8 more

It seems that this is a BUG. Does anyone encounter such a problem?

I am wondering why we introduce ReadableConfigToConfigurationAdapter
to wrap the Configuration but leave many of the methods in it to throw
UnsupportedOperationException that causes problems.

Regards,

Canbin Zheng
Reply | Threaded
Open this post in threaded view
|

Re: Question about the ReadableConfigToConfigurationAdapter

Till Rohrmann
Hi Canbin,

this looks indeed like a bug to me. I'm pulling in Dawid who worked on this
part and might be able to tell us more about it.

If he agrees, then I would suggest to open a JIRA issue and to fix it.

Cheers,
Till

On Wed, Apr 1, 2020 at 11:26 AM Canbin Zheng <[hidden email]> wrote:

> Hi everyone,
>
> Recently I failed to run a Flink job when enabling RocksDBStateBackend on
> the branch master and 1.10.
>
> The exception stack trace is:
>
> The program finished with the following
> exception:org.apache.flink.client.program.ProgramInvocationException:
> The main method caused an error: The adapter does not support this
> method
>         at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
>         at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
>         at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
>         at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
>         at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
>         at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
>         at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
>         at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>         at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
> Caused by: java.lang.UnsupportedOperationException: The adapter does
> not support this method
>         at
> org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
>         at
> org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
>         at
> org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
>         at
> org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
>         at java.util.Optional.map(Optional.java:215)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
>         at
> org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
>         ... 8 more
>
> It seems that this is a BUG. Does anyone encounter such a problem?
>
> I am wondering why we introduce ReadableConfigToConfigurationAdapter
> to wrap the Configuration but leave many of the methods in it to throw
> UnsupportedOperationException that causes problems.
>
> Regards,
>
> Canbin Zheng
>
Reply | Threaded
Open this post in threaded view
|

Re: Question about the ReadableConfigToConfigurationAdapter

dwysakowicz

Hi,

Yes this is a bug that is tracked here: https://issues.apache.org/jira/browse/FLINK-16913. I am working on it right now.

You should expect a fix very soon.

Best,

Dawid

On 02/04/2020 17:07, Till Rohrmann wrote:
Hi Canbin,

this looks indeed like a bug to me. I'm pulling in Dawid who worked on this part and might be able to tell us more about it.

If he agrees, then I would suggest to open a JIRA issue and to fix it.

Cheers,
Till

On Wed, Apr 1, 2020 at 11:26 AM Canbin Zheng <[hidden email]> wrote:
Hi everyone,

Recently I failed to run a Flink job when enabling RocksDBStateBackend on
the branch master and 1.10.

The exception stack trace is:

The program finished with the following
exception:org.apache.flink.client.program.ProgramInvocationException:
The main method caused an error: The adapter does not support this
method
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
        at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
        at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
        at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
        at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
        at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
        at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
        at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
Caused by: java.lang.UnsupportedOperationException: The adapter does
not support this method
        at org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
        at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
        at org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
        at org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
        at org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
        at java.util.Optional.map(Optional.java:215)
        at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
        at org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
        ... 8 more

It seems that this is a BUG. Does anyone encounter such a problem?

I am wondering why we introduce ReadableConfigToConfigurationAdapter
to wrap the Configuration but leave many of the methods in it to throw
UnsupportedOperationException that causes problems.

Regards,

Canbin Zheng

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Question about the ReadableConfigToConfigurationAdapter

felixzheng
Thanks for your feedback, Till & Dawid.

Dawid Wysakowicz <[hidden email]> 于2020年4月2日周四 下午11:11写道:

> Hi,
>
> Yes this is a bug that is tracked here:
> https://issues.apache.org/jira/browse/FLINK-16913. I am working on it
> right now.
>
> You should expect a fix very soon.
>
> Best,
>
> Dawid
> On 02/04/2020 17:07, Till Rohrmann wrote:
>
> Hi Canbin,
>
> this looks indeed like a bug to me. I'm pulling in Dawid who worked on
> this part and might be able to tell us more about it.
>
> If he agrees, then I would suggest to open a JIRA issue and to fix it.
>
> Cheers,
> Till
>
> On Wed, Apr 1, 2020 at 11:26 AM Canbin Zheng <[hidden email]>
> wrote:
>
>> Hi everyone,
>>
>> Recently I failed to run a Flink job when enabling RocksDBStateBackend on
>> the branch master and 1.10.
>>
>> The exception stack trace is:
>>
>> The program finished with the following
>> exception:org.apache.flink.client.program.ProgramInvocationException:
>> The main method caused an error: The adapter does not support this
>> method
>>         at
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
>>         at
>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
>>         at
>> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
>>         at
>> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
>>         at
>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
>>         at
>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
>>         at
>> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
>>         at
>> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>         at
>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
>> Caused by: java.lang.UnsupportedOperationException: The adapter does
>> not support this method
>>         at
>> org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
>>         at
>> org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
>>         at
>> org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
>>         at
>> org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
>>         at java.util.Optional.map(Optional.java:215)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
>>         at
>> org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:498)
>>         at
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
>>         ... 8 more
>>
>> It seems that this is a BUG. Does anyone encounter such a problem?
>>
>> I am wondering why we introduce ReadableConfigToConfigurationAdapter
>> to wrap the Configuration but leave many of the methods in it to throw
>> UnsupportedOperationException that causes problems.
>>
>> Regards,
>>
>> Canbin Zheng
>>
>