[jira] [Created] (FLINK-16684) StreamingFileSink builder does not work with Scala

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-16684) StreamingFileSink builder does not work with Scala

Shang Yuanchun (Jira)
Till Rohrmann created FLINK-16684:
-------------------------------------

             Summary: StreamingFileSink builder does not work with Scala
                 Key: FLINK-16684
                 URL: https://issues.apache.org/jira/browse/FLINK-16684
             Project: Flink
          Issue Type: Bug
          Components: API / Scala, Connectors / FileSystem
    Affects Versions: 1.10.0
            Reporter: Till Rohrmann
             Fix For: 1.10.1, 1.11.0


The {{StreamingFileSink}} builders don't work with Scala as they lose with every {{with*}} method more type information. For example, the following example does not compile:

{code}
val sink = StreamingFileSink.forRowFormat(new Path("s3a://123"), new Encoder[String] {
      override def encode(element: String, stream: OutputStream): Unit = ???
    }).withRollingPolicy(
      DefaultRollingPolicy.builder()
        .withRolloverInterval(TimeUnit.MINUTES.toMinutes(5))
        .withInactivityInterval(TimeUnit.MINUTES.toMinutes(5))
        .withMaxPartSize(128 * 1024 * 1024)
        .build()
    ).withBucketAssigner(
      new BucketAssigner[String, String] {
        override def getBucketId(element: String, context: BucketAssigner.Context): String = ???
        override def getSerializer: SimpleVersionedSerializer[String] = ???
      }
    ).build();
{code}

The problem seems to be that Scala does type inference slightly differently than Java. I believe that the unspecified {{RowFormatBuilder}} type cannot be properly resolved.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)