Issues using StreamingFileSink (Cannot locate configuration: tried hadoop-metrics2-s3a-file-system.properties, hadoop-metrics2.properties)

[vishalovercome]

I've written a Flink job to stream updates to S3 using StreamingFileSink. I
used the presto plugin for checkpointing and the hadoop plugin for writing
part files to S3. I was going through my application logs and found these
messages which I couldn't understand and would like to know if there's some
step I might have missed.

Even if these messages are harmless, I would like to understand what these
mean.

2020-05-19 19:23:33,277 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.commons.beanutils.FluentPropertyBeanIntrospector
- Error when creating PropertyDescriptor for public final void
org.apache.flink.fs.shaded.hadoop3.org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)!
Ignoring this property.

*2020-05-19 19:23:33,310 WARN
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsConfig
- Cannot locate configuration: tried
hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties*

2020-05-19 19:23:33,397 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
- Scheduled Metric snapshot period at 10 second(s).

2020-05-19 19:23:33,397 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
- s3a-file-system metrics system started

2020-05-19 19:23:34,833 INFO
org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.conf.Configuration.deprecation
- fs.s3a.server-side-encryption-key is deprecated. Instead, use
fs.s3a.server-side-encryption.key



--
Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

[Till Rohrmann]

Hi,

for the first message I think this might be related to BEANUTILS-477 [1]
and should not be a problem.

The other messages look to me like the metric system initialization of the
Hadoop components. I don't think that there is something wrong with them.

[1] https://issues.apache.org/jira/browse/BEANUTILS-477

Cheers,
Till

On Wed, May 20, 2020 at 6:46 PM vishalovercome <[hidden email]> wrote:

> I've written a Flink job to stream updates to S3 using StreamingFileSink. I
> used the presto plugin for checkpointing and the hadoop plugin for writing
> part files to S3. I was going through my application logs and found these
> messages which I couldn't understand and would like to know if there's some
> step I might have missed.
>
> Even if these messages are harmless, I would like to understand what these
> mean.
>
> 2020-05-19 19:23:33,277 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.commons.beanutils.FluentPropertyBeanIntrospector
> - Error when creating PropertyDescriptor for public final void
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)!
> Ignoring this property.
>
> *2020-05-19 19:23:33,310 WARN
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsConfig
> - Cannot locate configuration: tried
> hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties*
>
> 2020-05-19 19:23:33,397 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
> - Scheduled Metric snapshot period at 10 second(s).
>
> 2020-05-19 19:23:33,397 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.metrics2.impl.MetricsSystemImpl
> - s3a-file-system metrics system started
>
> 2020-05-19 19:23:34,833 INFO
>
> org.apache.flink.fs.shaded.hadoop3.org.apache.hadoop.conf.Configuration.deprecation
> - fs.s3a.server-side-encryption-key is deprecated. Instead, use
> fs.s3a.server-side-encryption.key
>
>
>
> --
> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/
>