Scala 2.10/2.11 Maven dependencies

[mxm]

Hi Flinksters,

We have recently committed an easy way to change Flink's Scala version. The
question arises now whether we should ship Scala 2.11 as binaries and via
Maven. For the rc0, I created all binaries twice, for Scala 2.10 and 2.11.
However, I didn't create Maven artifacts. This follows our current shipping
strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven dependencies but
additionally Hadoop 2.4, 2.6, 2.7 as binaries.

Should we also upload Maven dependencies for Scala 2.11?

If so, the next question arises: What version pattern should we have for
the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to the
VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.

However, it is common practice to append the suffix to the artifactID of
the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1. This
has mostly historic reasons but is widely used.

Whatever naming pattern we choose, it should be consistent. I would be in
favor of changing our artifact names to contain the Hadoop and Scala
version. This would also imply that all Scala dependent Maven modules
receive a Scala suffix (also the default Scala 2.10 modules).

Cheers,
Max

[Till Rohrmann]

I would be in favor of deploying also Scala 2.11 artifacts to Maven since
more and more people will try out Flink with Scala 2.11. Having the
dependencies in the Maven repository makes it considerably easier for
people to get their Flink jobs running.

Furthermore, I observed that people are not aware that our deployed
artifacts, e.g. flink-runtime, are built with Scala 2.10. As a consequence,
they mix flink dependencies with other dependencies pulling in Scala 2.11
and then they wonder that the program crashes. It would be, imho, clearer
if all our dependencies which depend on a specific Scala version would have
the corresponding Scala suffix appended.

Adding the 2.10 suffix would also spare us the hassle of upgrading to a
newer Scala version in the future, because then the artifacts wouldn't
share the same artifact name.

Cheers,
Till

On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]> wrote:

> Hi Flinksters,
>
> We have recently committed an easy way to change Flink's Scala version. The
> question arises now whether we should ship Scala 2.11 as binaries and via
> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and 2.11.
> However, I didn't create Maven artifacts. This follows our current shipping
> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven dependencies but
> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>
> Should we also upload Maven dependencies for Scala 2.11?
>
> If so, the next question arises: What version pattern should we have for
> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to the
> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
>
> However, it is common practice to append the suffix to the artifactID of
> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1. This
> has mostly historic reasons but is widely used.
>
> Whatever naming pattern we choose, it should be consistent. I would be in
> favor of changing our artifact names to contain the Hadoop and Scala
> version. This would also imply that all Scala dependent Maven modules
> receive a Scala suffix (also the default Scala 2.10 modules).
>
> Cheers,
> Max
>

[Ufuk Celebi-2]

I agree with Till, but is this something you want to address in this release already?

I would postpone it to 1.0.0.

– Ufuk

> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]> wrote:
>
> I would be in favor of deploying also Scala 2.11 artifacts to Maven since
> more and more people will try out Flink with Scala 2.11. Having the
> dependencies in the Maven repository makes it considerably easier for
> people to get their Flink jobs running.
>
> Furthermore, I observed that people are not aware that our deployed
> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a consequence,
> they mix flink dependencies with other dependencies pulling in Scala 2.11
> and then they wonder that the program crashes. It would be, imho, clearer
> if all our dependencies which depend on a specific Scala version would have
> the corresponding Scala suffix appended.
>
> Adding the 2.10 suffix would also spare us the hassle of upgrading to a
> newer Scala version in the future, because then the artifacts wouldn't
> share the same artifact name.
>
> Cheers,
> Till
>
> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]> wrote:
>
>> Hi Flinksters,
>>
>> We have recently committed an easy way to change Flink's Scala version. The
>> question arises now whether we should ship Scala 2.11 as binaries and via
>> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and 2.11.
>> However, I didn't create Maven artifacts. This follows our current shipping
>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven dependencies but
>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>>
>> Should we also upload Maven dependencies for Scala 2.11?
>>
>> If so, the next question arises: What version pattern should we have for
>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to the
>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
>>
>> However, it is common practice to append the suffix to the artifactID of
>> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1. This
>> has mostly historic reasons but is widely used.
>>
>> Whatever naming pattern we choose, it should be consistent. I would be in
>> favor of changing our artifact names to contain the Hadoop and Scala
>> version. This would also imply that all Scala dependent Maven modules
>> receive a Scala suffix (also the default Scala 2.10 modules).
>>
>> Cheers,
>> Max
>>

[Theodore Vasiloudis]

+1 for having binaries, I'm working on a Spark application currently with
Scala 2.11 and having to rebuild everything when deploying e.g. to EC2 is a
pain.

On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]> wrote:

> I agree with Till, but is this something you want to address in this
> release already?
>
> I would postpone it to 1.0.0.
>
> – Ufuk
>
> > On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]> wrote:
> >
> > I would be in favor of deploying also Scala 2.11 artifacts to Maven since
> > more and more people will try out Flink with Scala 2.11. Having the
> > dependencies in the Maven repository makes it considerably easier for
> > people to get their Flink jobs running.
> >
> > Furthermore, I observed that people are not aware that our deployed
> > artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
> consequence,
> > they mix flink dependencies with other dependencies pulling in Scala 2.11
> > and then they wonder that the program crashes. It would be, imho, clearer
> > if all our dependencies which depend on a specific Scala version would
> have
> > the corresponding Scala suffix appended.
> >
> > Adding the 2.10 suffix would also spare us the hassle of upgrading to a
> > newer Scala version in the future, because then the artifacts wouldn't
> > share the same artifact name.
> >
> > Cheers,
> > Till
> >
> > On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]>
> wrote:
> >
> >> Hi Flinksters,
> >>
> >> We have recently committed an easy way to change Flink's Scala version.
> The
> >> question arises now whether we should ship Scala 2.11 as binaries and
> via
> >> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and
> 2.11.
> >> However, I didn't create Maven artifacts. This follows our current
> shipping
> >> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven dependencies
> but
> >> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> >>
> >> Should we also upload Maven dependencies for Scala 2.11?
> >>
> >> If so, the next question arises: What version pattern should we have for
> >> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to the
> >> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
> >>
> >> However, it is common practice to append the suffix to the artifactID of
> >> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1.
> This
> >> has mostly historic reasons but is widely used.
> >>
> >> Whatever naming pattern we choose, it should be consistent. I would be
> in
> >> favor of changing our artifact names to contain the Hadoop and Scala
> >> version. This would also imply that all Scala dependent Maven modules
> >> receive a Scala suffix (also the default Scala 2.10 modules).
> >>
> >> Cheers,
> >> Max
> >>
>
>

[Fabian Hueske-2]

+1 to have binaries for both versions in Maven and as build to download.

2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
[hidden email]>:

> +1 for having binaries, I'm working on a Spark application currently with
> Scala 2.11 and having to rebuild everything when deploying e.g. to EC2 is a
> pain.
>
> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]> wrote:
>
> > I agree with Till, but is this something you want to address in this
> > release already?
> >
> > I would postpone it to 1.0.0.
> >
> > – Ufuk
> >
> > > On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]> wrote:
> > >
> > > I would be in favor of deploying also Scala 2.11 artifacts to Maven
> since
> > > more and more people will try out Flink with Scala 2.11. Having the
> > > dependencies in the Maven repository makes it considerably easier for
> > > people to get their Flink jobs running.
> > >
> > > Furthermore, I observed that people are not aware that our deployed
> > > artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
> > consequence,
> > > they mix flink dependencies with other dependencies pulling in Scala
> 2.11
> > > and then they wonder that the program crashes. It would be, imho,
> clearer
> > > if all our dependencies which depend on a specific Scala version would
> > have
> > > the corresponding Scala suffix appended.
> > >
> > > Adding the 2.10 suffix would also spare us the hassle of upgrading to a
> > > newer Scala version in the future, because then the artifacts wouldn't
> > > share the same artifact name.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]>
> > wrote:
> > >
> > >> Hi Flinksters,
> > >>
> > >> We have recently committed an easy way to change Flink's Scala
> version.
> > The
> > >> question arises now whether we should ship Scala 2.11 as binaries and
> > via
> > >> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and
> > 2.11.
> > >> However, I didn't create Maven artifacts. This follows our current
> > shipping
> > >> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
> dependencies
> > but
> > >> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> > >>
> > >> Should we also upload Maven dependencies for Scala 2.11?
> > >>
> > >> If so, the next question arises: What version pattern should we have
> for
> > >> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to
> the
> > >> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
> > >>
> > >> However, it is common practice to append the suffix to the artifactID
> of
> > >> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1.
> > This
> > >> has mostly historic reasons but is widely used.
> > >>
> > >> Whatever naming pattern we choose, it should be consistent. I would be
> > in
> > >> favor of changing our artifact names to contain the Hadoop and Scala
> > >> version. This would also imply that all Scala dependent Maven modules
> > >> receive a Scala suffix (also the default Scala 2.10 modules).
> > >>
> > >> Cheers,
> > >> Max
> > >>
> >
> >
>

[Frederick F. Kautz IV]

No idea if I get a vote ;) Nevertheless, +1 to have binaries for both
versions in Maven and explicitly "scala versioned".

Some background on this for those not as familiar with scala versioning:

It's considered best practice to label what version of scala a library
uses in the artifact id.

The reason is compiled scala code is only compatible with the major
version of scala it was compiled for. For example, a library compatible
with 2.10 is not compatible with 2.11. The same will be true with 2.12
once it is released. Mixing versions will result in undefined behavior
which will likely manifest itself as runtime exceptions.

The convention to fix this problem is for all published libraries to
specify the version of scala they are compatible with. Leaving out the
scala version in a library is akin to saying "We don't depend on scala
for this library, so feel free to use whatever you want." Sbt users will
typically specify the version of scala they use and tooling is built
around ensuring consistency with the "%%" operator.

E.g.

scalaVersion := "2.11.4"

// this resolves to to artifactID: "scalacheck_2.11"
libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.0" % "test"

The most important part of this is that the scala version is explicit
which eliminates the problem for downstream users.

Cheers,
Frederick

On 10/28/2015 10:55 AM, Fabian Hueske wrote:

> +1 to have binaries for both versions in Maven and as build to download.
>
> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> [hidden email]>:
>
>> +1 for having binaries, I'm working on a Spark application currently with
>> Scala 2.11 and having to rebuild everything when deploying e.g. to EC2 is a
>> pain.
>>
>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]> wrote:
>>
>>> I agree with Till, but is this something you want to address in this
>>> release already?
>>>
>>> I would postpone it to 1.0.0.
>>>
>>> – Ufuk
>>>
>>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]> wrote:
>>>>
>>>> I would be in favor of deploying also Scala 2.11 artifacts to Maven
>> since
>>>> more and more people will try out Flink with Scala 2.11. Having the
>>>> dependencies in the Maven repository makes it considerably easier for
>>>> people to get their Flink jobs running.
>>>>
>>>> Furthermore, I observed that people are not aware that our deployed
>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
>>> consequence,
>>>> they mix flink dependencies with other dependencies pulling in Scala
>> 2.11
>>>> and then they wonder that the program crashes. It would be, imho,
>> clearer
>>>> if all our dependencies which depend on a specific Scala version would
>>> have
>>>> the corresponding Scala suffix appended.
>>>>
>>>> Adding the 2.10 suffix would also spare us the hassle of upgrading to a
>>>> newer Scala version in the future, because then the artifacts wouldn't
>>>> share the same artifact name.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]>
>>> wrote:
>>>>> Hi Flinksters,
>>>>>
>>>>> We have recently committed an easy way to change Flink's Scala
>> version.
>>> The
>>>>> question arises now whether we should ship Scala 2.11 as binaries and
>>> via
>>>>> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and
>>> 2.11.
>>>>> However, I didn't create Maven artifacts. This follows our current
>>> shipping
>>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
>> dependencies
>>> but
>>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>>>>>
>>>>> Should we also upload Maven dependencies for Scala 2.11?
>>>>>
>>>>> If so, the next question arises: What version pattern should we have
>> for
>>>>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to
>> the
>>>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
>>>>>
>>>>> However, it is common practice to append the suffix to the artifactID
>> of
>>>>> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1.
>>> This
>>>>> has mostly historic reasons but is widely used.
>>>>>
>>>>> Whatever naming pattern we choose, it should be consistent. I would be
>>> in
>>>>> favor of changing our artifact names to contain the Hadoop and Scala
>>>>> version. This would also imply that all Scala dependent Maven modules
>>>>> receive a Scala suffix (also the default Scala 2.10 modules).
>>>>>
>>>>> Cheers,
>>>>> Max
>>>>>
>>>

[mxm]

Seems like we agree that we need artifacts for different versions of Scala
on Maven. There also seems to be a preference for including the version in
the artifact name.

I've created an issue and marked it to be resolved for 1.0. For the 0.10
release, we will have binaries but no Maven artifacts. The biggest
challenge I see is to remove Scala from as many modules as possible. For
example, flink-java depends on Scala at the moment..

https://issues.apache.org/jira/browse/FLINK-2940

On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <[hidden email]>
wrote:

> No idea if I get a vote ;) Nevertheless, +1 to have binaries for both
> versions in Maven and explicitly "scala versioned".
>
> Some background on this for those not as familiar with scala versioning:
>
> It's considered best practice to label what version of scala a library
> uses in the artifact id.
>
> The reason is compiled scala code is only compatible with the major
> version of scala it was compiled for. For example, a library compatible
> with 2.10 is not compatible with 2.11. The same will be true with 2.12 once
> it is released. Mixing versions will result in undefined behavior which
> will likely manifest itself as runtime exceptions.
>
> The convention to fix this problem is for all published libraries to
> specify the version of scala they are compatible with. Leaving out the
> scala version in a library is akin to saying "We don't depend on scala for
> this library, so feel free to use whatever you want." Sbt users will
> typically specify the version of scala they use and tooling is built around
> ensuring consistency with the "%%" operator.
>
> E.g.
>
> scalaVersion := "2.11.4"
>
> // this resolves to to artifactID: "scalacheck_2.11"
> libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.0" % "test"
>
> The most important part of this is that the scala version is explicit
> which eliminates the problem for downstream users.
>
> Cheers,
> Frederick
>
>
> On 10/28/2015 10:55 AM, Fabian Hueske wrote:
>
>> +1 to have binaries for both versions in Maven and as build to download.
>>
>> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
>> [hidden email]>:
>>
>> +1 for having binaries, I'm working on a Spark application currently with
>>> Scala 2.11 and having to rebuild everything when deploying e.g. to EC2
>>> is a
>>> pain.
>>>
>>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]> wrote:
>>>
>>> I agree with Till, but is this something you want to address in this
>>>> release already?
>>>>
>>>> I would postpone it to 1.0.0.
>>>>
>>>> – Ufuk
>>>>
>>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]> wrote:
>>>>>
>>>>> I would be in favor of deploying also Scala 2.11 artifacts to Maven
>>>>>
>>>> since
>>>
>>>> more and more people will try out Flink with Scala 2.11. Having the
>>>>> dependencies in the Maven repository makes it considerably easier for
>>>>> people to get their Flink jobs running.
>>>>>
>>>>> Furthermore, I observed that people are not aware that our deployed
>>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
>>>>>
>>>> consequence,
>>>>
>>>>> they mix flink dependencies with other dependencies pulling in Scala
>>>>>
>>>> 2.11
>>>
>>>> and then they wonder that the program crashes. It would be, imho,
>>>>>
>>>> clearer
>>>
>>>> if all our dependencies which depend on a specific Scala version would
>>>>>
>>>> have
>>>>
>>>>> the corresponding Scala suffix appended.
>>>>>
>>>>> Adding the 2.10 suffix would also spare us the hassle of upgrading to a
>>>>> newer Scala version in the future, because then the artifacts wouldn't
>>>>> share the same artifact name.
>>>>>
>>>>> Cheers,
>>>>> Till
>>>>>
>>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]>
>>>>>
>>>> wrote:
>>>>
>>>>> Hi Flinksters,
>>>>>>
>>>>>> We have recently committed an easy way to change Flink's Scala
>>>>>>
>>>>> version.
>>>
>>>> The
>>>>
>>>>> question arises now whether we should ship Scala 2.11 as binaries and
>>>>>>
>>>>> via
>>>>
>>>>> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and
>>>>>>
>>>>> 2.11.
>>>>
>>>>> However, I didn't create Maven artifacts. This follows our current
>>>>>>
>>>>> shipping
>>>>
>>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
>>>>>>
>>>>> dependencies
>>>
>>>> but
>>>>
>>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>>>>>>
>>>>>> Should we also upload Maven dependencies for Scala 2.11?
>>>>>>
>>>>>> If so, the next question arises: What version pattern should we have
>>>>>>
>>>>> for
>>>
>>>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to
>>>>>>
>>>>> the
>>>
>>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
>>>>>>
>>>>>> However, it is common practice to append the suffix to the artifactID
>>>>>>
>>>>> of
>>>
>>>> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1.
>>>>>>
>>>>> This
>>>>
>>>>> has mostly historic reasons but is widely used.
>>>>>>
>>>>>> Whatever naming pattern we choose, it should be consistent. I would be
>>>>>>
>>>>> in
>>>>
>>>>> favor of changing our artifact names to contain the Hadoop and Scala
>>>>>> version. This would also imply that all Scala dependent Maven modules
>>>>>> receive a Scala suffix (also the default Scala 2.10 modules).
>>>>>>
>>>>>> Cheers,
>>>>>> Max
>>>>>>
>>>>>>
>>>>
>

[aalexandrov]

My two cents - there are already Maven artifacts deployed for 2.11 in the
SNAPSHOT repository. I think it might be confusing if they suddenly
disappear for the stable release.


2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:

> Seems like we agree that we need artifacts for different versions of Scala
> on Maven. There also seems to be a preference for including the version in
> the artifact name.
>
> I've created an issue and marked it to be resolved for 1.0. For the 0.10
> release, we will have binaries but no Maven artifacts. The biggest
> challenge I see is to remove Scala from as many modules as possible. For
> example, flink-java depends on Scala at the moment..
>
> https://issues.apache.org/jira/browse/FLINK-2940
>
> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <[hidden email]>
> wrote:
>
> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for both
> > versions in Maven and explicitly "scala versioned".
> >
> > Some background on this for those not as familiar with scala versioning:
> >
> > It's considered best practice to label what version of scala a library
> > uses in the artifact id.
> >
> > The reason is compiled scala code is only compatible with the major
> > version of scala it was compiled for. For example, a library compatible
> > with 2.10 is not compatible with 2.11. The same will be true with 2.12
> once
> > it is released. Mixing versions will result in undefined behavior which
> > will likely manifest itself as runtime exceptions.
> >
> > The convention to fix this problem is for all published libraries to
> > specify the version of scala they are compatible with. Leaving out the
> > scala version in a library is akin to saying "We don't depend on scala
> for
> > this library, so feel free to use whatever you want." Sbt users will
> > typically specify the version of scala they use and tooling is built
> around
> > ensuring consistency with the "%%" operator.
> >
> > E.g.
> >
> > scalaVersion := "2.11.4"
> >
> > // this resolves to to artifactID: "scalacheck_2.11"
> > libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.0" %
> "test"
> >
> > The most important part of this is that the scala version is explicit
> > which eliminates the problem for downstream users.
> >
> > Cheers,
> > Frederick
> >
> >
> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
> >
> >> +1 to have binaries for both versions in Maven and as build to download.
> >>
> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> >> [hidden email]>:
> >>
> >> +1 for having binaries, I'm working on a Spark application currently
> with
> >>> Scala 2.11 and having to rebuild everything when deploying e.g. to EC2
> >>> is a
> >>> pain.
> >>>
> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]> wrote:
> >>>
> >>> I agree with Till, but is this something you want to address in this
> >>>> release already?
> >>>>
> >>>> I would postpone it to 1.0.0.
> >>>>
> >>>> – Ufuk
> >>>>
> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]> wrote:
> >>>>>
> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to Maven
> >>>>>
> >>>> since
> >>>
> >>>> more and more people will try out Flink with Scala 2.11. Having the
> >>>>> dependencies in the Maven repository makes it considerably easier for
> >>>>> people to get their Flink jobs running.
> >>>>>
> >>>>> Furthermore, I observed that people are not aware that our deployed
> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
> >>>>>
> >>>> consequence,
> >>>>
> >>>>> they mix flink dependencies with other dependencies pulling in Scala
> >>>>>
> >>>> 2.11
> >>>
> >>>> and then they wonder that the program crashes. It would be, imho,
> >>>>>
> >>>> clearer
> >>>
> >>>> if all our dependencies which depend on a specific Scala version would
> >>>>>
> >>>> have
> >>>>
> >>>>> the corresponding Scala suffix appended.
> >>>>>
> >>>>> Adding the 2.10 suffix would also spare us the hassle of upgrading
> to a
> >>>>> newer Scala version in the future, because then the artifacts
> wouldn't
> >>>>> share the same artifact name.
> >>>>>
> >>>>> Cheers,
> >>>>> Till
> >>>>>
> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]>
> >>>>>
> >>>> wrote:
> >>>>
> >>>>> Hi Flinksters,
> >>>>>>
> >>>>>> We have recently committed an easy way to change Flink's Scala
> >>>>>>
> >>>>> version.
> >>>
> >>>> The
> >>>>
> >>>>> question arises now whether we should ship Scala 2.11 as binaries and
> >>>>>>
> >>>>> via
> >>>>
> >>>>> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and
> >>>>>>
> >>>>> 2.11.
> >>>>
> >>>>> However, I didn't create Maven artifacts. This follows our current
> >>>>>>
> >>>>> shipping
> >>>>
> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
> >>>>>>
> >>>>> dependencies
> >>>
> >>>> but
> >>>>
> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> >>>>>>
> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
> >>>>>>
> >>>>>> If so, the next question arises: What version pattern should we have
> >>>>>>
> >>>>> for
> >>>
> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to
> >>>>>>
> >>>>> the
> >>>
> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
> >>>>>>
> >>>>>> However, it is common practice to append the suffix to the
> artifactID
> >>>>>>
> >>>>> of
> >>>
> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1.
> >>>>>>
> >>>>> This
> >>>>
> >>>>> has mostly historic reasons but is widely used.
> >>>>>>
> >>>>>> Whatever naming pattern we choose, it should be consistent. I would
> be
> >>>>>>
> >>>>> in
> >>>>
> >>>>> favor of changing our artifact names to contain the Hadoop and Scala
> >>>>>> version. This would also imply that all Scala dependent Maven
> modules
> >>>>>> receive a Scala suffix (also the default Scala 2.10 modules).
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Max
> >>>>>>
> >>>>>>
> >>>>
> >
>

[mxm]

Good point. Didn't know that. We can still add them for the release.

On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
<[hidden email]> wrote:

> My two cents - there are already Maven artifacts deployed for 2.11 in the
> SNAPSHOT repository. I think it might be confusing if they suddenly
> disappear for the stable release.
>
>
> 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:
>
>> Seems like we agree that we need artifacts for different versions of Scala
>> on Maven. There also seems to be a preference for including the version in
>> the artifact name.
>>
>> I've created an issue and marked it to be resolved for 1.0. For the 0.10
>> release, we will have binaries but no Maven artifacts. The biggest
>> challenge I see is to remove Scala from as many modules as possible. For
>> example, flink-java depends on Scala at the moment..
>>
>> https://issues.apache.org/jira/browse/FLINK-2940
>>
>> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <[hidden email]>
>> wrote:
>>
>> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for both
>> > versions in Maven and explicitly "scala versioned".
>> >
>> > Some background on this for those not as familiar with scala versioning:
>> >
>> > It's considered best practice to label what version of scala a library
>> > uses in the artifact id.
>> >
>> > The reason is compiled scala code is only compatible with the major
>> > version of scala it was compiled for. For example, a library compatible
>> > with 2.10 is not compatible with 2.11. The same will be true with 2.12
>> once
>> > it is released. Mixing versions will result in undefined behavior which
>> > will likely manifest itself as runtime exceptions.
>> >
>> > The convention to fix this problem is for all published libraries to
>> > specify the version of scala they are compatible with. Leaving out the
>> > scala version in a library is akin to saying "We don't depend on scala
>> for
>> > this library, so feel free to use whatever you want." Sbt users will
>> > typically specify the version of scala they use and tooling is built
>> around
>> > ensuring consistency with the "%%" operator.
>> >
>> > E.g.
>> >
>> > scalaVersion := "2.11.4"
>> >
>> > // this resolves to to artifactID: "scalacheck_2.11"
>> > libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.0" %
>> "test"
>> >
>> > The most important part of this is that the scala version is explicit
>> > which eliminates the problem for downstream users.
>> >
>> > Cheers,
>> > Frederick
>> >
>> >
>> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
>> >
>> >> +1 to have binaries for both versions in Maven and as build to download.
>> >>
>> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
>> >> [hidden email]>:
>> >>
>> >> +1 for having binaries, I'm working on a Spark application currently
>> with
>> >>> Scala 2.11 and having to rebuild everything when deploying e.g. to EC2
>> >>> is a
>> >>> pain.
>> >>>
>> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]> wrote:
>> >>>
>> >>> I agree with Till, but is this something you want to address in this
>> >>>> release already?
>> >>>>
>> >>>> I would postpone it to 1.0.0.
>> >>>>
>> >>>> – Ufuk
>> >>>>
>> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]> wrote:
>> >>>>>
>> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to Maven
>> >>>>>
>> >>>> since
>> >>>
>> >>>> more and more people will try out Flink with Scala 2.11. Having the
>> >>>>> dependencies in the Maven repository makes it considerably easier for
>> >>>>> people to get their Flink jobs running.
>> >>>>>
>> >>>>> Furthermore, I observed that people are not aware that our deployed
>> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
>> >>>>>
>> >>>> consequence,
>> >>>>
>> >>>>> they mix flink dependencies with other dependencies pulling in Scala
>> >>>>>
>> >>>> 2.11
>> >>>
>> >>>> and then they wonder that the program crashes. It would be, imho,
>> >>>>>
>> >>>> clearer
>> >>>
>> >>>> if all our dependencies which depend on a specific Scala version would
>> >>>>>
>> >>>> have
>> >>>>
>> >>>>> the corresponding Scala suffix appended.
>> >>>>>
>> >>>>> Adding the 2.10 suffix would also spare us the hassle of upgrading
>> to a
>> >>>>> newer Scala version in the future, because then the artifacts
>> wouldn't
>> >>>>> share the same artifact name.
>> >>>>>
>> >>>>> Cheers,
>> >>>>> Till
>> >>>>>
>> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <[hidden email]>
>> >>>>>
>> >>>> wrote:
>> >>>>
>> >>>>> Hi Flinksters,
>> >>>>>>
>> >>>>>> We have recently committed an easy way to change Flink's Scala
>> >>>>>>
>> >>>>> version.
>> >>>
>> >>>> The
>> >>>>
>> >>>>> question arises now whether we should ship Scala 2.11 as binaries and
>> >>>>>>
>> >>>>> via
>> >>>>
>> >>>>> Maven. For the rc0, I created all binaries twice, for Scala 2.10 and
>> >>>>>>
>> >>>>> 2.11.
>> >>>>
>> >>>>> However, I didn't create Maven artifacts. This follows our current
>> >>>>>>
>> >>>>> shipping
>> >>>>
>> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
>> >>>>>>
>> >>>>> dependencies
>> >>>
>> >>>> but
>> >>>>
>> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>> >>>>>>
>> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
>> >>>>>>
>> >>>>>> If so, the next question arises: What version pattern should we have
>> >>>>>>
>> >>>>> for
>> >>>
>> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1 to
>> >>>>>>
>> >>>>> the
>> >>>
>> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
>> >>>>>>
>> >>>>>> However, it is common practice to append the suffix to the
>> artifactID
>> >>>>>>
>> >>>>> of
>> >>>
>> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11, version=0.9.1.
>> >>>>>>
>> >>>>> This
>> >>>>
>> >>>>> has mostly historic reasons but is widely used.
>> >>>>>>
>> >>>>>> Whatever naming pattern we choose, it should be consistent. I would
>> be
>> >>>>>>
>> >>>>> in
>> >>>>
>> >>>>> favor of changing our artifact names to contain the Hadoop and Scala
>> >>>>>> version. This would also imply that all Scala dependent Maven
>> modules
>> >>>>>> receive a Scala suffix (also the default Scala 2.10 modules).
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>> Max
>> >>>>>>
>> >>>>>>
>> >>>>
>> >
>>

[Stephan Ewen]

+1 for the approach discusses here, and for removing Scala dependencies
from modules that can be Scala independent.

It would be nice if pure Java users would not see any Scala versioning (on
flink-core, flink-java, later also flink-sreaming-java). I guess for any
runtime-related parts (including flink-client and currently all streaming
projects), we need the Scala versions...

On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]> wrote:

> Good point. Didn't know that. We can still add them for the release.
>
> On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
> <[hidden email]> wrote:
> > My two cents - there are already Maven artifacts deployed for 2.11 in the
> > SNAPSHOT repository. I think it might be confusing if they suddenly
> > disappear for the stable release.
> >
> >
> > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:
> >
> >> Seems like we agree that we need artifacts for different versions of
> Scala
> >> on Maven. There also seems to be a preference for including the version
> in
> >> the artifact name.
> >>
> >> I've created an issue and marked it to be resolved for 1.0. For the 0.10
> >> release, we will have binaries but no Maven artifacts. The biggest
> >> challenge I see is to remove Scala from as many modules as possible. For
> >> example, flink-java depends on Scala at the moment..
> >>
> >> https://issues.apache.org/jira/browse/FLINK-2940
> >>
> >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
> [hidden email]>
> >> wrote:
> >>
> >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for both
> >> > versions in Maven and explicitly "scala versioned".
> >> >
> >> > Some background on this for those not as familiar with scala
> versioning:
> >> >
> >> > It's considered best practice to label what version of scala a library
> >> > uses in the artifact id.
> >> >
> >> > The reason is compiled scala code is only compatible with the major
> >> > version of scala it was compiled for. For example, a library
> compatible
> >> > with 2.10 is not compatible with 2.11. The same will be true with 2.12
> >> once
> >> > it is released. Mixing versions will result in undefined behavior
> which
> >> > will likely manifest itself as runtime exceptions.
> >> >
> >> > The convention to fix this problem is for all published libraries to
> >> > specify the version of scala they are compatible with. Leaving out the
> >> > scala version in a library is akin to saying "We don't depend on scala
> >> for
> >> > this library, so feel free to use whatever you want." Sbt users will
> >> > typically specify the version of scala they use and tooling is built
> >> around
> >> > ensuring consistency with the "%%" operator.
> >> >
> >> > E.g.
> >> >
> >> > scalaVersion := "2.11.4"
> >> >
> >> > // this resolves to to artifactID: "scalacheck_2.11"
> >> > libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.0" %
> >> "test"
> >> >
> >> > The most important part of this is that the scala version is explicit
> >> > which eliminates the problem for downstream users.
> >> >
> >> > Cheers,
> >> > Frederick
> >> >
> >> >
> >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
> >> >
> >> >> +1 to have binaries for both versions in Maven and as build to
> download.
> >> >>
> >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> >> >> [hidden email]>:
> >> >>
> >> >> +1 for having binaries, I'm working on a Spark application currently
> >> with
> >> >>> Scala 2.11 and having to rebuild everything when deploying e.g. to
> EC2
> >> >>> is a
> >> >>> pain.
> >> >>>
> >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]>
> wrote:
> >> >>>
> >> >>> I agree with Till, but is this something you want to address in this
> >> >>>> release already?
> >> >>>>
> >> >>>> I would postpone it to 1.0.0.
> >> >>>>
> >> >>>> – Ufuk
> >> >>>>
> >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]>
> wrote:
> >> >>>>>
> >> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to
> Maven
> >> >>>>>
> >> >>>> since
> >> >>>
> >> >>>> more and more people will try out Flink with Scala 2.11. Having the
> >> >>>>> dependencies in the Maven repository makes it considerably easier
> for
> >> >>>>> people to get their Flink jobs running.
> >> >>>>>
> >> >>>>> Furthermore, I observed that people are not aware that our
> deployed
> >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
> >> >>>>>
> >> >>>> consequence,
> >> >>>>
> >> >>>>> they mix flink dependencies with other dependencies pulling in
> Scala
> >> >>>>>
> >> >>>> 2.11
> >> >>>
> >> >>>> and then they wonder that the program crashes. It would be, imho,
> >> >>>>>
> >> >>>> clearer
> >> >>>
> >> >>>> if all our dependencies which depend on a specific Scala version
> would
> >> >>>>>
> >> >>>> have
> >> >>>>
> >> >>>>> the corresponding Scala suffix appended.
> >> >>>>>
> >> >>>>> Adding the 2.10 suffix would also spare us the hassle of upgrading
> >> to a
> >> >>>>> newer Scala version in the future, because then the artifacts
> >> wouldn't
> >> >>>>> share the same artifact name.
> >> >>>>>
> >> >>>>> Cheers,
> >> >>>>> Till
> >> >>>>>
> >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
> [hidden email]>
> >> >>>>>
> >> >>>> wrote:
> >> >>>>
> >> >>>>> Hi Flinksters,
> >> >>>>>>
> >> >>>>>> We have recently committed an easy way to change Flink's Scala
> >> >>>>>>
> >> >>>>> version.
> >> >>>
> >> >>>> The
> >> >>>>
> >> >>>>> question arises now whether we should ship Scala 2.11 as binaries
> and
> >> >>>>>>
> >> >>>>> via
> >> >>>>
> >> >>>>> Maven. For the rc0, I created all binaries twice, for Scala 2.10
> and
> >> >>>>>>
> >> >>>>> 2.11.
> >> >>>>
> >> >>>>> However, I didn't create Maven artifacts. This follows our current
> >> >>>>>>
> >> >>>>> shipping
> >> >>>>
> >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
> >> >>>>>>
> >> >>>>> dependencies
> >> >>>
> >> >>>> but
> >> >>>>
> >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> >> >>>>>>
> >> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
> >> >>>>>>
> >> >>>>>> If so, the next question arises: What version pattern should we
> have
> >> >>>>>>
> >> >>>>> for
> >> >>>
> >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1
> to
> >> >>>>>>
> >> >>>>> the
> >> >>>
> >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
> >> >>>>>>
> >> >>>>>> However, it is common practice to append the suffix to the
> >> artifactID
> >> >>>>>>
> >> >>>>> of
> >> >>>
> >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
> version=0.9.1.
> >> >>>>>>
> >> >>>>> This
> >> >>>>
> >> >>>>> has mostly historic reasons but is widely used.
> >> >>>>>>
> >> >>>>>> Whatever naming pattern we choose, it should be consistent. I
> would
> >> be
> >> >>>>>>
> >> >>>>> in
> >> >>>>
> >> >>>>> favor of changing our artifact names to contain the Hadoop and
> Scala
> >> >>>>>> version. This would also imply that all Scala dependent Maven
> >> modules
> >> >>>>>> receive a Scala suffix (also the default Scala 2.10 modules).
> >> >>>>>>
> >> >>>>>> Cheers,
> >> >>>>>> Max
> >> >>>>>>
> >> >>>>>>
> >> >>>>
> >> >
> >>
>

[Fabian Hueske-2]

OK, let me try to summarize the discussion (and please correct me if I got
something wrong).

1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have to
release 2.11 artifacts for the 0.10.0 release version as well.

2) Everybody agrees to appropriately tag all artifacts that have a
(transitive) Scala dependency. ATM, that would also include flink-java
which is a bit awkward. The Scala dependency in flink-java originates from
the Chill library which is used to obtain a Kryo serializer which is
initialized with serializers for Scala classes. We could resolve this issue
by providing Java and Scala specific implementations of the Kryo
serializers and have KryoTypeInfos for Java and Scala.

The question to answer right now is, do we want to have "correctly" labeled
artifacts for the next 0.10.0 release or do we defer that for 1.0?
If we want to solve it for 0.10.0 we need to cancel the current RC and
provide a fix to remove the Scala dependency in flink-java, IMO.

Opinions?

Cheers, Fabian

2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:

> +1 for the approach discusses here, and for removing Scala dependencies
> from modules that can be Scala independent.
>
> It would be nice if pure Java users would not see any Scala versioning (on
> flink-core, flink-java, later also flink-sreaming-java). I guess for any
> runtime-related parts (including flink-client and currently all streaming
> projects), we need the Scala versions...
>
> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]> wrote:
>
> > Good point. Didn't know that. We can still add them for the release.
> >
> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
> > <[hidden email]> wrote:
> > > My two cents - there are already Maven artifacts deployed for 2.11 in
> the
> > > SNAPSHOT repository. I think it might be confusing if they suddenly
> > > disappear for the stable release.
> > >
> > >
> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:
> > >
> > >> Seems like we agree that we need artifacts for different versions of
> > Scala
> > >> on Maven. There also seems to be a preference for including the
> version
> > in
> > >> the artifact name.
> > >>
> > >> I've created an issue and marked it to be resolved for 1.0. For the
> 0.10
> > >> release, we will have binaries but no Maven artifacts. The biggest
> > >> challenge I see is to remove Scala from as many modules as possible.
> For
> > >> example, flink-java depends on Scala at the moment..
> > >>
> > >> https://issues.apache.org/jira/browse/FLINK-2940
> > >>
> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
> > [hidden email]>
> > >> wrote:
> > >>
> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for
> both
> > >> > versions in Maven and explicitly "scala versioned".
> > >> >
> > >> > Some background on this for those not as familiar with scala
> > versioning:
> > >> >
> > >> > It's considered best practice to label what version of scala a
> library
> > >> > uses in the artifact id.
> > >> >
> > >> > The reason is compiled scala code is only compatible with the major
> > >> > version of scala it was compiled for. For example, a library
> > compatible
> > >> > with 2.10 is not compatible with 2.11. The same will be true with
> 2.12
> > >> once
> > >> > it is released. Mixing versions will result in undefined behavior
> > which
> > >> > will likely manifest itself as runtime exceptions.
> > >> >
> > >> > The convention to fix this problem is for all published libraries to
> > >> > specify the version of scala they are compatible with. Leaving out
> the
> > >> > scala version in a library is akin to saying "We don't depend on
> scala
> > >> for
> > >> > this library, so feel free to use whatever you want." Sbt users will
> > >> > typically specify the version of scala they use and tooling is built
> > >> around
> > >> > ensuring consistency with the "%%" operator.
> > >> >
> > >> > E.g.
> > >> >
> > >> > scalaVersion := "2.11.4"
> > >> >
> > >> > // this resolves to to artifactID: "scalacheck_2.11"
> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.0" %
> > >> "test"
> > >> >
> > >> > The most important part of this is that the scala version is
> explicit
> > >> > which eliminates the problem for downstream users.
> > >> >
> > >> > Cheers,
> > >> > Frederick
> > >> >
> > >> >
> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
> > >> >
> > >> >> +1 to have binaries for both versions in Maven and as build to
> > download.
> > >> >>
> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> > >> >> [hidden email]>:
> > >> >>
> > >> >> +1 for having binaries, I'm working on a Spark application
> currently
> > >> with
> > >> >>> Scala 2.11 and having to rebuild everything when deploying e.g. to
> > EC2
> > >> >>> is a
> > >> >>> pain.
> > >> >>>
> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]>
> > wrote:
> > >> >>>
> > >> >>> I agree with Till, but is this something you want to address in
> this
> > >> >>>> release already?
> > >> >>>>
> > >> >>>> I would postpone it to 1.0.0.
> > >> >>>>
> > >> >>>> – Ufuk
> > >> >>>>
> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]>
> > wrote:
> > >> >>>>>
> > >> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to
> > Maven
> > >> >>>>>
> > >> >>>> since
> > >> >>>
> > >> >>>> more and more people will try out Flink with Scala 2.11. Having
> the
> > >> >>>>> dependencies in the Maven repository makes it considerably
> easier
> > for
> > >> >>>>> people to get their Flink jobs running.
> > >> >>>>>
> > >> >>>>> Furthermore, I observed that people are not aware that our
> > deployed
> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
> > >> >>>>>
> > >> >>>> consequence,
> > >> >>>>
> > >> >>>>> they mix flink dependencies with other dependencies pulling in
> > Scala
> > >> >>>>>
> > >> >>>> 2.11
> > >> >>>
> > >> >>>> and then they wonder that the program crashes. It would be, imho,
> > >> >>>>>
> > >> >>>> clearer
> > >> >>>
> > >> >>>> if all our dependencies which depend on a specific Scala version
> > would
> > >> >>>>>
> > >> >>>> have
> > >> >>>>
> > >> >>>>> the corresponding Scala suffix appended.
> > >> >>>>>
> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of
> upgrading
> > >> to a
> > >> >>>>> newer Scala version in the future, because then the artifacts
> > >> wouldn't
> > >> >>>>> share the same artifact name.
> > >> >>>>>
> > >> >>>>> Cheers,
> > >> >>>>> Till
> > >> >>>>>
> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
> > [hidden email]>
> > >> >>>>>
> > >> >>>> wrote:
> > >> >>>>
> > >> >>>>> Hi Flinksters,
> > >> >>>>>>
> > >> >>>>>> We have recently committed an easy way to change Flink's Scala
> > >> >>>>>>
> > >> >>>>> version.
> > >> >>>
> > >> >>>> The
> > >> >>>>
> > >> >>>>> question arises now whether we should ship Scala 2.11 as
> binaries
> > and
> > >> >>>>>>
> > >> >>>>> via
> > >> >>>>
> > >> >>>>> Maven. For the rc0, I created all binaries twice, for Scala 2.10
> > and
> > >> >>>>>>
> > >> >>>>> 2.11.
> > >> >>>>
> > >> >>>>> However, I didn't create Maven artifacts. This follows our
> current
> > >> >>>>>>
> > >> >>>>> shipping
> > >> >>>>
> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
> > >> >>>>>>
> > >> >>>>> dependencies
> > >> >>>
> > >> >>>> but
> > >> >>>>
> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> > >> >>>>>>
> > >> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
> > >> >>>>>>
> > >> >>>>>> If so, the next question arises: What version pattern should we
> > have
> > >> >>>>>>
> > >> >>>>> for
> > >> >>>
> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1
> > to
> > >> >>>>>>
> > >> >>>>> the
> > >> >>>
> > >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
> > >> >>>>>>
> > >> >>>>>> However, it is common practice to append the suffix to the
> > >> artifactID
> > >> >>>>>>
> > >> >>>>> of
> > >> >>>
> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
> > version=0.9.1.
> > >> >>>>>>
> > >> >>>>> This
> > >> >>>>
> > >> >>>>> has mostly historic reasons but is widely used.
> > >> >>>>>>
> > >> >>>>>> Whatever naming pattern we choose, it should be consistent. I
> > would
> > >> be
> > >> >>>>>>
> > >> >>>>> in
> > >> >>>>
> > >> >>>>> favor of changing our artifact names to contain the Hadoop and
> > Scala
> > >> >>>>>> version. This would also imply that all Scala dependent Maven
> > >> modules
> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10 modules).
> > >> >>>>>>
> > >> >>>>>> Cheers,
> > >> >>>>>> Max
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>
> > >> >
> > >>
> >
>

[mxm]

I'm for leaving it as-is and renaming all artifacts which depend on
Scala for the release following 0.10.

On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]> wrote:

> OK, let me try to summarize the discussion (and please correct me if I got
> something wrong).
>
> 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have to
> release 2.11 artifacts for the 0.10.0 release version as well.
>
> 2) Everybody agrees to appropriately tag all artifacts that have a
> (transitive) Scala dependency. ATM, that would also include flink-java
> which is a bit awkward. The Scala dependency in flink-java originates from
> the Chill library which is used to obtain a Kryo serializer which is
> initialized with serializers for Scala classes. We could resolve this issue
> by providing Java and Scala specific implementations of the Kryo
> serializers and have KryoTypeInfos for Java and Scala.
>
> The question to answer right now is, do we want to have "correctly" labeled
> artifacts for the next 0.10.0 release or do we defer that for 1.0?
> If we want to solve it for 0.10.0 we need to cancel the current RC and
> provide a fix to remove the Scala dependency in flink-java, IMO.
>
> Opinions?
>
> Cheers, Fabian
>
> 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:
>
>> +1 for the approach discusses here, and for removing Scala dependencies
>> from modules that can be Scala independent.
>>
>> It would be nice if pure Java users would not see any Scala versioning (on
>> flink-core, flink-java, later also flink-sreaming-java). I guess for any
>> runtime-related parts (including flink-client and currently all streaming
>> projects), we need the Scala versions...
>>
>> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]> wrote:
>>
>> > Good point. Didn't know that. We can still add them for the release.
>> >
>> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
>> > <[hidden email]> wrote:
>> > > My two cents - there are already Maven artifacts deployed for 2.11 in
>> the
>> > > SNAPSHOT repository. I think it might be confusing if they suddenly
>> > > disappear for the stable release.
>> > >
>> > >
>> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:
>> > >
>> > >> Seems like we agree that we need artifacts for different versions of
>> > Scala
>> > >> on Maven. There also seems to be a preference for including the
>> version
>> > in
>> > >> the artifact name.
>> > >>
>> > >> I've created an issue and marked it to be resolved for 1.0. For the
>> 0.10
>> > >> release, we will have binaries but no Maven artifacts. The biggest
>> > >> challenge I see is to remove Scala from as many modules as possible.
>> For
>> > >> example, flink-java depends on Scala at the moment..
>> > >>
>> > >> https://issues.apache.org/jira/browse/FLINK-2940
>> > >>
>> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
>> > [hidden email]>
>> > >> wrote:
>> > >>
>> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for
>> both
>> > >> > versions in Maven and explicitly "scala versioned".
>> > >> >
>> > >> > Some background on this for those not as familiar with scala
>> > versioning:
>> > >> >
>> > >> > It's considered best practice to label what version of scala a
>> library
>> > >> > uses in the artifact id.
>> > >> >
>> > >> > The reason is compiled scala code is only compatible with the major
>> > >> > version of scala it was compiled for. For example, a library
>> > compatible
>> > >> > with 2.10 is not compatible with 2.11. The same will be true with
>> 2.12
>> > >> once
>> > >> > it is released. Mixing versions will result in undefined behavior
>> > which
>> > >> > will likely manifest itself as runtime exceptions.
>> > >> >
>> > >> > The convention to fix this problem is for all published libraries to
>> > >> > specify the version of scala they are compatible with. Leaving out
>> the
>> > >> > scala version in a library is akin to saying "We don't depend on
>> scala
>> > >> for
>> > >> > this library, so feel free to use whatever you want." Sbt users will
>> > >> > typically specify the version of scala they use and tooling is built
>> > >> around
>> > >> > ensuring consistency with the "%%" operator.
>> > >> >
>> > >> > E.g.
>> > >> >
>> > >> > scalaVersion := "2.11.4"
>> > >> >
>> > >> > // this resolves to to artifactID: "scalacheck_2.11"
>> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" % "1.12.0" %
>> > >> "test"
>> > >> >
>> > >> > The most important part of this is that the scala version is
>> explicit
>> > >> > which eliminates the problem for downstream users.
>> > >> >
>> > >> > Cheers,
>> > >> > Frederick
>> > >> >
>> > >> >
>> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
>> > >> >
>> > >> >> +1 to have binaries for both versions in Maven and as build to
>> > download.
>> > >> >>
>> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
>> > >> >> [hidden email]>:
>> > >> >>
>> > >> >> +1 for having binaries, I'm working on a Spark application
>> currently
>> > >> with
>> > >> >>> Scala 2.11 and having to rebuild everything when deploying e.g. to
>> > EC2
>> > >> >>> is a
>> > >> >>> pain.
>> > >> >>>
>> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]>
>> > wrote:
>> > >> >>>
>> > >> >>> I agree with Till, but is this something you want to address in
>> this
>> > >> >>>> release already?
>> > >> >>>>
>> > >> >>>> I would postpone it to 1.0.0.
>> > >> >>>>
>> > >> >>>> – Ufuk
>> > >> >>>>
>> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]>
>> > wrote:
>> > >> >>>>>
>> > >> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to
>> > Maven
>> > >> >>>>>
>> > >> >>>> since
>> > >> >>>
>> > >> >>>> more and more people will try out Flink with Scala 2.11. Having
>> the
>> > >> >>>>> dependencies in the Maven repository makes it considerably
>> easier
>> > for
>> > >> >>>>> people to get their Flink jobs running.
>> > >> >>>>>
>> > >> >>>>> Furthermore, I observed that people are not aware that our
>> > deployed
>> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As a
>> > >> >>>>>
>> > >> >>>> consequence,
>> > >> >>>>
>> > >> >>>>> they mix flink dependencies with other dependencies pulling in
>> > Scala
>> > >> >>>>>
>> > >> >>>> 2.11
>> > >> >>>
>> > >> >>>> and then they wonder that the program crashes. It would be, imho,
>> > >> >>>>>
>> > >> >>>> clearer
>> > >> >>>
>> > >> >>>> if all our dependencies which depend on a specific Scala version
>> > would
>> > >> >>>>>
>> > >> >>>> have
>> > >> >>>>
>> > >> >>>>> the corresponding Scala suffix appended.
>> > >> >>>>>
>> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of
>> upgrading
>> > >> to a
>> > >> >>>>> newer Scala version in the future, because then the artifacts
>> > >> wouldn't
>> > >> >>>>> share the same artifact name.
>> > >> >>>>>
>> > >> >>>>> Cheers,
>> > >> >>>>> Till
>> > >> >>>>>
>> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
>> > [hidden email]>
>> > >> >>>>>
>> > >> >>>> wrote:
>> > >> >>>>
>> > >> >>>>> Hi Flinksters,
>> > >> >>>>>>
>> > >> >>>>>> We have recently committed an easy way to change Flink's Scala
>> > >> >>>>>>
>> > >> >>>>> version.
>> > >> >>>
>> > >> >>>> The
>> > >> >>>>
>> > >> >>>>> question arises now whether we should ship Scala 2.11 as
>> binaries
>> > and
>> > >> >>>>>>
>> > >> >>>>> via
>> > >> >>>>
>> > >> >>>>> Maven. For the rc0, I created all binaries twice, for Scala 2.10
>> > and
>> > >> >>>>>>
>> > >> >>>>> 2.11.
>> > >> >>>>
>> > >> >>>>> However, I didn't create Maven artifacts. This follows our
>> current
>> > >> >>>>>>
>> > >> >>>>> shipping
>> > >> >>>>
>> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
>> > >> >>>>>>
>> > >> >>>>> dependencies
>> > >> >>>
>> > >> >>>> but
>> > >> >>>>
>> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>> > >> >>>>>>
>> > >> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
>> > >> >>>>>>
>> > >> >>>>>> If so, the next question arises: What version pattern should we
>> > have
>> > >> >>>>>>
>> > >> >>>>> for
>> > >> >>>
>> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append -hadoop1
>> > to
>> > >> >>>>>>
>> > >> >>>>> the
>> > >> >>>
>> > >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
>> > >> >>>>>>
>> > >> >>>>>> However, it is common practice to append the suffix to the
>> > >> artifactID
>> > >> >>>>>>
>> > >> >>>>> of
>> > >> >>>
>> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
>> > version=0.9.1.
>> > >> >>>>>>
>> > >> >>>>> This
>> > >> >>>>
>> > >> >>>>> has mostly historic reasons but is widely used.
>> > >> >>>>>>
>> > >> >>>>>> Whatever naming pattern we choose, it should be consistent. I
>> > would
>> > >> be
>> > >> >>>>>>
>> > >> >>>>> in
>> > >> >>>>
>> > >> >>>>> favor of changing our artifact names to contain the Hadoop and
>> > Scala
>> > >> >>>>>> version. This would also imply that all Scala dependent Maven
>> > >> modules
>> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10 modules).
>> > >> >>>>>>
>> > >> >>>>>> Cheers,
>> > >> >>>>>> Max
>> > >> >>>>>>
>> > >> >>>>>>
>> > >> >>>>
>> > >> >
>> > >>
>> >
>>

[Fabian Hueske-2]

That would mean to have "flink-java_2.10" and "flink-java_2.11" artifacts
(and others that depend on flink-java and have no other Scala dependency)
in the 0.10.0 release and only "flink-java" in the next 1.0 release.

Do we want that?

2015-11-02 11:37 GMT+01:00 Maximilian Michels <[hidden email]>:

> I'm for leaving it as-is and renaming all artifacts which depend on
> Scala for the release following 0.10.
>
> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]> wrote:
> > OK, let me try to summarize the discussion (and please correct me if I
> got
> > something wrong).
> >
> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have to
> > release 2.11 artifacts for the 0.10.0 release version as well.
> >
> > 2) Everybody agrees to appropriately tag all artifacts that have a
> > (transitive) Scala dependency. ATM, that would also include flink-java
> > which is a bit awkward. The Scala dependency in flink-java originates
> from
> > the Chill library which is used to obtain a Kryo serializer which is
> > initialized with serializers for Scala classes. We could resolve this
> issue
> > by providing Java and Scala specific implementations of the Kryo
> > serializers and have KryoTypeInfos for Java and Scala.
> >
> > The question to answer right now is, do we want to have "correctly"
> labeled
> > artifacts for the next 0.10.0 release or do we defer that for 1.0?
> > If we want to solve it for 0.10.0 we need to cancel the current RC and
> > provide a fix to remove the Scala dependency in flink-java, IMO.
> >
> > Opinions?
> >
> > Cheers, Fabian
> >
> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:
> >
> >> +1 for the approach discusses here, and for removing Scala dependencies
> >> from modules that can be Scala independent.
> >>
> >> It would be nice if pure Java users would not see any Scala versioning
> (on
> >> flink-core, flink-java, later also flink-sreaming-java). I guess for any
> >> runtime-related parts (including flink-client and currently all
> streaming
> >> projects), we need the Scala versions...
> >>
> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]>
> wrote:
> >>
> >> > Good point. Didn't know that. We can still add them for the release.
> >> >
> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
> >> > <[hidden email]> wrote:
> >> > > My two cents - there are already Maven artifacts deployed for 2.11
> in
> >> the
> >> > > SNAPSHOT repository. I think it might be confusing if they suddenly
> >> > > disappear for the stable release.
> >> > >
> >> > >
> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:
> >> > >
> >> > >> Seems like we agree that we need artifacts for different versions
> of
> >> > Scala
> >> > >> on Maven. There also seems to be a preference for including the
> >> version
> >> > in
> >> > >> the artifact name.
> >> > >>
> >> > >> I've created an issue and marked it to be resolved for 1.0. For the
> >> 0.10
> >> > >> release, we will have binaries but no Maven artifacts. The biggest
> >> > >> challenge I see is to remove Scala from as many modules as
> possible.
> >> For
> >> > >> example, flink-java depends on Scala at the moment..
> >> > >>
> >> > >> https://issues.apache.org/jira/browse/FLINK-2940
> >> > >>
> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
> >> > [hidden email]>
> >> > >> wrote:
> >> > >>
> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for
> >> both
> >> > >> > versions in Maven and explicitly "scala versioned".
> >> > >> >
> >> > >> > Some background on this for those not as familiar with scala
> >> > versioning:
> >> > >> >
> >> > >> > It's considered best practice to label what version of scala a
> >> library
> >> > >> > uses in the artifact id.
> >> > >> >
> >> > >> > The reason is compiled scala code is only compatible with the
> major
> >> > >> > version of scala it was compiled for. For example, a library
> >> > compatible
> >> > >> > with 2.10 is not compatible with 2.11. The same will be true with
> >> 2.12
> >> > >> once
> >> > >> > it is released. Mixing versions will result in undefined behavior
> >> > which
> >> > >> > will likely manifest itself as runtime exceptions.
> >> > >> >
> >> > >> > The convention to fix this problem is for all published
> libraries to
> >> > >> > specify the version of scala they are compatible with. Leaving
> out
> >> the
> >> > >> > scala version in a library is akin to saying "We don't depend on
> >> scala
> >> > >> for
> >> > >> > this library, so feel free to use whatever you want." Sbt users
> will
> >> > >> > typically specify the version of scala they use and tooling is
> built
> >> > >> around
> >> > >> > ensuring consistency with the "%%" operator.
> >> > >> >
> >> > >> > E.g.
> >> > >> >
> >> > >> > scalaVersion := "2.11.4"
> >> > >> >
> >> > >> > // this resolves to to artifactID: "scalacheck_2.11"
> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" %
> "1.12.0" %
> >> > >> "test"
> >> > >> >
> >> > >> > The most important part of this is that the scala version is
> >> explicit
> >> > >> > which eliminates the problem for downstream users.
> >> > >> >
> >> > >> > Cheers,
> >> > >> > Frederick
> >> > >> >
> >> > >> >
> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
> >> > >> >
> >> > >> >> +1 to have binaries for both versions in Maven and as build to
> >> > download.
> >> > >> >>
> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> >> > >> >> [hidden email]>:
> >> > >> >>
> >> > >> >> +1 for having binaries, I'm working on a Spark application
> >> currently
> >> > >> with
> >> > >> >>> Scala 2.11 and having to rebuild everything when deploying
> e.g. to
> >> > EC2
> >> > >> >>> is a
> >> > >> >>> pain.
> >> > >> >>>
> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]>
> >> > wrote:
> >> > >> >>>
> >> > >> >>> I agree with Till, but is this something you want to address in
> >> this
> >> > >> >>>> release already?
> >> > >> >>>>
> >> > >> >>>> I would postpone it to 1.0.0.
> >> > >> >>>>
> >> > >> >>>> – Ufuk
> >> > >> >>>>
> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]
> >
> >> > wrote:
> >> > >> >>>>>
> >> > >> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to
> >> > Maven
> >> > >> >>>>>
> >> > >> >>>> since
> >> > >> >>>
> >> > >> >>>> more and more people will try out Flink with Scala 2.11.
> Having
> >> the
> >> > >> >>>>> dependencies in the Maven repository makes it considerably
> >> easier
> >> > for
> >> > >> >>>>> people to get their Flink jobs running.
> >> > >> >>>>>
> >> > >> >>>>> Furthermore, I observed that people are not aware that our
> >> > deployed
> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As
> a
> >> > >> >>>>>
> >> > >> >>>> consequence,
> >> > >> >>>>
> >> > >> >>>>> they mix flink dependencies with other dependencies pulling
> in
> >> > Scala
> >> > >> >>>>>
> >> > >> >>>> 2.11
> >> > >> >>>
> >> > >> >>>> and then they wonder that the program crashes. It would be,
> imho,
> >> > >> >>>>>
> >> > >> >>>> clearer
> >> > >> >>>
> >> > >> >>>> if all our dependencies which depend on a specific Scala
> version
> >> > would
> >> > >> >>>>>
> >> > >> >>>> have
> >> > >> >>>>
> >> > >> >>>>> the corresponding Scala suffix appended.
> >> > >> >>>>>
> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of
> >> upgrading
> >> > >> to a
> >> > >> >>>>> newer Scala version in the future, because then the artifacts
> >> > >> wouldn't
> >> > >> >>>>> share the same artifact name.
> >> > >> >>>>>
> >> > >> >>>>> Cheers,
> >> > >> >>>>> Till
> >> > >> >>>>>
> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
> >> > [hidden email]>
> >> > >> >>>>>
> >> > >> >>>> wrote:
> >> > >> >>>>
> >> > >> >>>>> Hi Flinksters,
> >> > >> >>>>>>
> >> > >> >>>>>> We have recently committed an easy way to change Flink's
> Scala
> >> > >> >>>>>>
> >> > >> >>>>> version.
> >> > >> >>>
> >> > >> >>>> The
> >> > >> >>>>
> >> > >> >>>>> question arises now whether we should ship Scala 2.11 as
> >> binaries
> >> > and
> >> > >> >>>>>>
> >> > >> >>>>> via
> >> > >> >>>>
> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for Scala
> 2.10
> >> > and
> >> > >> >>>>>>
> >> > >> >>>>> 2.11.
> >> > >> >>>>
> >> > >> >>>>> However, I didn't create Maven artifacts. This follows our
> >> current
> >> > >> >>>>>>
> >> > >> >>>>> shipping
> >> > >> >>>>
> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
> >> > >> >>>>>>
> >> > >> >>>>> dependencies
> >> > >> >>>
> >> > >> >>>> but
> >> > >> >>>>
> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> >> > >> >>>>>>
> >> > >> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
> >> > >> >>>>>>
> >> > >> >>>>>> If so, the next question arises: What version pattern
> should we
> >> > have
> >> > >> >>>>>>
> >> > >> >>>>> for
> >> > >> >>>
> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append
> -hadoop1
> >> > to
> >> > >> >>>>>>
> >> > >> >>>>> the
> >> > >> >>>
> >> > >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
> >> > >> >>>>>>
> >> > >> >>>>>> However, it is common practice to append the suffix to the
> >> > >> artifactID
> >> > >> >>>>>>
> >> > >> >>>>> of
> >> > >> >>>
> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
> >> > version=0.9.1.
> >> > >> >>>>>>
> >> > >> >>>>> This
> >> > >> >>>>
> >> > >> >>>>> has mostly historic reasons but is widely used.
> >> > >> >>>>>>
> >> > >> >>>>>> Whatever naming pattern we choose, it should be consistent.
> I
> >> > would
> >> > >> be
> >> > >> >>>>>>
> >> > >> >>>>> in
> >> > >> >>>>
> >> > >> >>>>> favor of changing our artifact names to contain the Hadoop
> and
> >> > Scala
> >> > >> >>>>>> version. This would also imply that all Scala dependent
> Maven
> >> > >> modules
> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10
> modules).
> >> > >> >>>>>>
> >> > >> >>>>>> Cheers,
> >> > >> >>>>>> Max
> >> > >> >>>>>>
> >> > >> >>>>>>
> >> > >> >>>>
> >> > >> >
> >> > >>
> >> >
> >>
>

[Chiwan Park-2]

If we choose selective Scala version suffix for artifacts, we have to tell which artifacts have the version suffix to newcomers. Some artifacts such as "flink-java”, "flink-streaming-java" are easily recognized. But IMO, knowing whether artifacts such as "flink-ml", "flink-clients", "flink-table" have the version suffix or not is difficult for newcomers.

This is why we are adding the version suffix to all Scala 2.11 artifacts currently. For Scala 2.10 artifacts, we aren’t adding the version suffix for Flink with Java users.

I’m for adding the version suffix to Scala 2.10 artifacts also. But I’m not sure that removing the version suffix from Java-only artifacts would be good. As I said above, It seems difficult for newcomers.

Regards,
Chiwan Park

On November 2, 2015 at 8:19:15 PM, Fabian Hueske ([hidden email]) wrote:

That would mean to have "flink-java_2.10" and "flink-java_2.11" artifacts  
(and others that depend on flink-java and have no other Scala dependency)  
in the 0.10.0 release and only "flink-java" in the next 1.0 release.  

Do we want that?  

2015-11-02 11:37 GMT+01:00 Maximilian Michels <[hidden email]>:  

> I'm for leaving it as-is and renaming all artifacts which depend on  
> Scala for the release following 0.10.  
>  
> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]> wrote:  
> > OK, let me try to summarize the discussion (and please correct me if I  
> got  
> > something wrong).  
> >  
> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have to  
> > release 2.11 artifacts for the 0.10.0 release version as well.  
> >  
> > 2) Everybody agrees to appropriately tag all artifacts that have a  
> > (transitive) Scala dependency. ATM, that would also include flink-java  
> > which is a bit awkward. The Scala dependency in flink-java originates  
> from  
> > the Chill library which is used to obtain a Kryo serializer which is  
> > initialized with serializers for Scala classes. We could resolve this  
> issue  
> > by providing Java and Scala specific implementations of the Kryo  
> > serializers and have KryoTypeInfos for Java and Scala.  
> >  
> > The question to answer right now is, do we want to have "correctly"  
> labeled  
> > artifacts for the next 0.10.0 release or do we defer that for 1.0?  
> > If we want to solve it for 0.10.0 we need to cancel the current RC and  
> > provide a fix to remove the Scala dependency in flink-java, IMO.  
> >  
> > Opinions?  
> >  
> > Cheers, Fabian  
> >  
> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:  
> >  
> >> +1 for the approach discusses here, and for removing Scala dependencies  
> >> from modules that can be Scala independent.  
> >>  
> >> It would be nice if pure Java users would not see any Scala versioning  
> (on  
> >> flink-core, flink-java, later also flink-sreaming-java). I guess for any  
> >> runtime-related parts (including flink-client and currently all  
> streaming  
> >> projects), we need the Scala versions...  
> >>  
> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]>  
> wrote:  
> >>  
> >> > Good point. Didn't know that. We can still add them for the release.  
> >> >  
> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov  
> >> > <[hidden email]> wrote:  
> >> > > My two cents - there are already Maven artifacts deployed for 2.11  
> in  
> >> the  
> >> > > SNAPSHOT repository. I think it might be confusing if they suddenly  
> >> > > disappear for the stable release.  
> >> > >  
> >> > >  
> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:  
> >> > >  
> >> > >> Seems like we agree that we need artifacts for different versions  
> of  
> >> > Scala  
> >> > >> on Maven. There also seems to be a preference for including the  
> >> version  
> >> > in  
> >> > >> the artifact name.  
> >> > >>  
> >> > >> I've created an issue and marked it to be resolved for 1.0. For the  
> >> 0.10  
> >> > >> release, we will have binaries but no Maven artifacts. The biggest  
> >> > >> challenge I see is to remove Scala from as many modules as  
> possible.  
> >> For  
> >> > >> example, flink-java depends on Scala at the moment..  
> >> > >>  
> >> > >> https://issues.apache.org/jira/browse/FLINK-2940 
> >> > >>  
> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <  
> >> > [hidden email]>  
> >> > >> wrote:  
> >> > >>  
> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for  
> >> both  
> >> > >> > versions in Maven and explicitly "scala versioned".  
> >> > >> >  
> >> > >> > Some background on this for those not as familiar with scala  
> >> > versioning:  
> >> > >> >  
> >> > >> > It's considered best practice to label what version of scala a  
> >> library  
> >> > >> > uses in the artifact id.  
> >> > >> >  
> >> > >> > The reason is compiled scala code is only compatible with the  
> major  
> >> > >> > version of scala it was compiled for. For example, a library  
> >> > compatible  
> >> > >> > with 2.10 is not compatible with 2.11. The same will be true with  
> >> 2.12  
> >> > >> once  
> >> > >> > it is released. Mixing versions will result in undefined behavior  
> >> > which  
> >> > >> > will likely manifest itself as runtime exceptions.  
> >> > >> >  
> >> > >> > The convention to fix this problem is for all published  
> libraries to  
> >> > >> > specify the version of scala they are compatible with. Leaving  
> out  
> >> the  
> >> > >> > scala version in a library is akin to saying "We don't depend on  
> >> scala  
> >> > >> for  
> >> > >> > this library, so feel free to use whatever you want." Sbt users  
> will  
> >> > >> > typically specify the version of scala they use and tooling is  
> built  
> >> > >> around  
> >> > >> > ensuring consistency with the "%%" operator.  
> >> > >> >  
> >> > >> > E.g.  
> >> > >> >  
> >> > >> > scalaVersion := "2.11.4"  
> >> > >> >  
> >> > >> > // this resolves to to artifactID: "scalacheck_2.11"  
> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" %  
> "1.12.0" %  
> >> > >> "test"  
> >> > >> >  
> >> > >> > The most important part of this is that the scala version is  
> >> explicit  
> >> > >> > which eliminates the problem for downstream users.  
> >> > >> >  
> >> > >> > Cheers,  
> >> > >> > Frederick  
> >> > >> >  
> >> > >> >  
> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:  
> >> > >> >  
> >> > >> >> +1 to have binaries for both versions in Maven and as build to  
> >> > download.  
> >> > >> >>  
> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <  
> >> > >> >> [hidden email]>:  
> >> > >> >>  
> >> > >> >> +1 for having binaries, I'm working on a Spark application  
> >> currently  
> >> > >> with  
> >> > >> >>> Scala 2.11 and having to rebuild everything when deploying  
> e.g. to  
> >> > EC2  
> >> > >> >>> is a  
> >> > >> >>> pain.  
> >> > >> >>>  
> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]>  
> >> > wrote:  
> >> > >> >>>  
> >> > >> >>> I agree with Till, but is this something you want to address in  
> >> this  
> >> > >> >>>> release already?  
> >> > >> >>>>  
> >> > >> >>>> I would postpone it to 1.0.0.  
> >> > >> >>>>  
> >> > >> >>>> – Ufuk  
> >> > >> >>>>  
> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]  
> >  
> >> > wrote:  
> >> > >> >>>>>  
> >> > >> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to  
> >> > Maven  
> >> > >> >>>>>  
> >> > >> >>>> since  
> >> > >> >>>  
> >> > >> >>>> more and more people will try out Flink with Scala 2.11.  
> Having  
> >> the  
> >> > >> >>>>> dependencies in the Maven repository makes it considerably  
> >> easier  
> >> > for  
> >> > >> >>>>> people to get their Flink jobs running.  
> >> > >> >>>>>  
> >> > >> >>>>> Furthermore, I observed that people are not aware that our  
> >> > deployed  
> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As  
> a  
> >> > >> >>>>>  
> >> > >> >>>> consequence,  
> >> > >> >>>>  
> >> > >> >>>>> they mix flink dependencies with other dependencies pulling  
> in  
> >> > Scala  
> >> > >> >>>>>  
> >> > >> >>>> 2.11  
> >> > >> >>>  
> >> > >> >>>> and then they wonder that the program crashes. It would be,  
> imho,  
> >> > >> >>>>>  
> >> > >> >>>> clearer  
> >> > >> >>>  
> >> > >> >>>> if all our dependencies which depend on a specific Scala  
> version  
> >> > would  
> >> > >> >>>>>  
> >> > >> >>>> have  
> >> > >> >>>>  
> >> > >> >>>>> the corresponding Scala suffix appended.  
> >> > >> >>>>>  
> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of  
> >> upgrading  
> >> > >> to a  
> >> > >> >>>>> newer Scala version in the future, because then the artifacts  
> >> > >> wouldn't  
> >> > >> >>>>> share the same artifact name.  
> >> > >> >>>>>  
> >> > >> >>>>> Cheers,  
> >> > >> >>>>> Till  
> >> > >> >>>>>  
> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <  
> >> > [hidden email]>  
> >> > >> >>>>>  
> >> > >> >>>> wrote:  
> >> > >> >>>>  
> >> > >> >>>>> Hi Flinksters,  
> >> > >> >>>>>>  
> >> > >> >>>>>> We have recently committed an easy way to change Flink's  
> Scala  
> >> > >> >>>>>>  
> >> > >> >>>>> version.  
> >> > >> >>>  
> >> > >> >>>> The  
> >> > >> >>>>  
> >> > >> >>>>> question arises now whether we should ship Scala 2.11 as  
> >> binaries  
> >> > and  
> >> > >> >>>>>>  
> >> > >> >>>>> via  
> >> > >> >>>>  
> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for Scala  
> 2.10  
> >> > and  
> >> > >> >>>>>>  
> >> > >> >>>>> 2.11.  
> >> > >> >>>>  
> >> > >> >>>>> However, I didn't create Maven artifacts. This follows our  
> >> current  
> >> > >> >>>>>>  
> >> > >> >>>>> shipping  
> >> > >> >>>>  
> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven  
> >> > >> >>>>>>  
> >> > >> >>>>> dependencies  
> >> > >> >>>  
> >> > >> >>>> but  
> >> > >> >>>>  
> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.  
> >> > >> >>>>>>  
> >> > >> >>>>>> Should we also upload Maven dependencies for Scala 2.11?  
> >> > >> >>>>>>  
> >> > >> >>>>>> If so, the next question arises: What version pattern  
> should we  
> >> > have  
> >> > >> >>>>>>  
> >> > >> >>>>> for  
> >> > >> >>>  
> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append  
> -hadoop1  
> >> > to  
> >> > >> >>>>>>  
> >> > >> >>>>> the  
> >> > >> >>>  
> >> > >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.  
> >> > >> >>>>>>  
> >> > >> >>>>>> However, it is common practice to append the suffix to the  
> >> > >> artifactID  
> >> > >> >>>>>>  
> >> > >> >>>>> of  
> >> > >> >>>  
> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,  
> >> > version=0.9.1.  
> >> > >> >>>>>>  
> >> > >> >>>>> This  
> >> > >> >>>>  
> >> > >> >>>>> has mostly historic reasons but is widely used.  
> >> > >> >>>>>>  
> >> > >> >>>>>> Whatever naming pattern we choose, it should be consistent.  
> I  
> >> > would  
> >> > >> be  
> >> > >> >>>>>>  
> >> > >> >>>>> in  
> >> > >> >>>>  
> >> > >> >>>>> favor of changing our artifact names to contain the Hadoop  
> and  
> >> > Scala  
> >> > >> >>>>>> version. This would also imply that all Scala dependent  
> Maven  
> >> > >> modules  
> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10  
> modules).  
> >> > >> >>>>>>  
> >> > >> >>>>>> Cheers,  
> >> > >> >>>>>> Max  
> >> > >> >>>>>>  
> >> > >> >>>>>>  
> >> > >> >>>>  
> >> > >> >  
> >> > >>  
> >> >  
> >>  
>  

[mxm]

> That would mean to have "flink-java_2.10" and "flink-java_2.11" artifacts
> (and others that depend on flink-java and have no other Scala dependency)
> in the 0.10.0 release and only "flink-java" in the next 1.0 release.


My suggestion was to keep the Scala unsuffixed Scala 2.10 and add a
suffix for Scala 2.11. That's the way we are currently doing it (also
deployed on Maven like this). For the next release after 0.10, we can
do it properly.

On Mon, Nov 2, 2015 at 1:30 PM, Chiwan Park <[hidden email]> wrote:

> If we choose selective Scala version suffix for artifacts, we have to tell which artifacts have the version suffix to newcomers. Some artifacts such as "flink-java”, "flink-streaming-java" are easily recognized. But IMO, knowing whether artifacts such as "flink-ml", "flink-clients", "flink-table" have the version suffix or not is difficult for newcomers.
>
> This is why we are adding the version suffix to all Scala 2.11 artifacts currently. For Scala 2.10 artifacts, we aren’t adding the version suffix for Flink with Java users.
>
> I’m for adding the version suffix to Scala 2.10 artifacts also. But I’m not sure that removing the version suffix from Java-only artifacts would be good. As I said above, It seems difficult for newcomers.
>
> Regards,
> Chiwan Park
>
> On November 2, 2015 at 8:19:15 PM, Fabian Hueske ([hidden email]) wrote:
>
> That would mean to have "flink-java_2.10" and "flink-java_2.11" artifacts
> (and others that depend on flink-java and have no other Scala dependency)
> in the 0.10.0 release and only "flink-java" in the next 1.0 release.
>
> Do we want that?
>
> 2015-11-02 11:37 GMT+01:00 Maximilian Michels <[hidden email]>:
>
>> I'm for leaving it as-is and renaming all artifacts which depend on
>> Scala for the release following 0.10.
>>
>> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]> wrote:
>> > OK, let me try to summarize the discussion (and please correct me if I
>> got
>> > something wrong).
>> >
>> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have to
>> > release 2.11 artifacts for the 0.10.0 release version as well.
>> >
>> > 2) Everybody agrees to appropriately tag all artifacts that have a
>> > (transitive) Scala dependency. ATM, that would also include flink-java
>> > which is a bit awkward. The Scala dependency in flink-java originates
>> from
>> > the Chill library which is used to obtain a Kryo serializer which is
>> > initialized with serializers for Scala classes. We could resolve this
>> issue
>> > by providing Java and Scala specific implementations of the Kryo
>> > serializers and have KryoTypeInfos for Java and Scala.
>> >
>> > The question to answer right now is, do we want to have "correctly"
>> labeled
>> > artifacts for the next 0.10.0 release or do we defer that for 1.0?
>> > If we want to solve it for 0.10.0 we need to cancel the current RC and
>> > provide a fix to remove the Scala dependency in flink-java, IMO.
>> >
>> > Opinions?
>> >
>> > Cheers, Fabian
>> >
>> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:
>> >
>> >> +1 for the approach discusses here, and for removing Scala dependencies
>> >> from modules that can be Scala independent.
>> >>
>> >> It would be nice if pure Java users would not see any Scala versioning
>> (on
>> >> flink-core, flink-java, later also flink-sreaming-java). I guess for any
>> >> runtime-related parts (including flink-client and currently all
>> streaming
>> >> projects), we need the Scala versions...
>> >>
>> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]>
>> wrote:
>> >>
>> >> > Good point. Didn't know that. We can still add them for the release.
>> >> >
>> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
>> >> > <[hidden email]> wrote:
>> >> > > My two cents - there are already Maven artifacts deployed for 2.11
>> in
>> >> the
>> >> > > SNAPSHOT repository. I think it might be confusing if they suddenly
>> >> > > disappear for the stable release.
>> >> > >
>> >> > >
>> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:
>> >> > >
>> >> > >> Seems like we agree that we need artifacts for different versions
>> of
>> >> > Scala
>> >> > >> on Maven. There also seems to be a preference for including the
>> >> version
>> >> > in
>> >> > >> the artifact name.
>> >> > >>
>> >> > >> I've created an issue and marked it to be resolved for 1.0. For the
>> >> 0.10
>> >> > >> release, we will have binaries but no Maven artifacts. The biggest
>> >> > >> challenge I see is to remove Scala from as many modules as
>> possible.
>> >> For
>> >> > >> example, flink-java depends on Scala at the moment..
>> >> > >>
>> >> > >> https://issues.apache.org/jira/browse/FLINK-2940
>> >> > >>
>> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
>> >> > [hidden email]>
>> >> > >> wrote:
>> >> > >>
>> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries for
>> >> both
>> >> > >> > versions in Maven and explicitly "scala versioned".
>> >> > >> >
>> >> > >> > Some background on this for those not as familiar with scala
>> >> > versioning:
>> >> > >> >
>> >> > >> > It's considered best practice to label what version of scala a
>> >> library
>> >> > >> > uses in the artifact id.
>> >> > >> >
>> >> > >> > The reason is compiled scala code is only compatible with the
>> major
>> >> > >> > version of scala it was compiled for. For example, a library
>> >> > compatible
>> >> > >> > with 2.10 is not compatible with 2.11. The same will be true with
>> >> 2.12
>> >> > >> once
>> >> > >> > it is released. Mixing versions will result in undefined behavior
>> >> > which
>> >> > >> > will likely manifest itself as runtime exceptions.
>> >> > >> >
>> >> > >> > The convention to fix this problem is for all published
>> libraries to
>> >> > >> > specify the version of scala they are compatible with. Leaving
>> out
>> >> the
>> >> > >> > scala version in a library is akin to saying "We don't depend on
>> >> scala
>> >> > >> for
>> >> > >> > this library, so feel free to use whatever you want." Sbt users
>> will
>> >> > >> > typically specify the version of scala they use and tooling is
>> built
>> >> > >> around
>> >> > >> > ensuring consistency with the "%%" operator.
>> >> > >> >
>> >> > >> > E.g.
>> >> > >> >
>> >> > >> > scalaVersion := "2.11.4"
>> >> > >> >
>> >> > >> > // this resolves to to artifactID: "scalacheck_2.11"
>> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" %
>> "1.12.0" %
>> >> > >> "test"
>> >> > >> >
>> >> > >> > The most important part of this is that the scala version is
>> >> explicit
>> >> > >> > which eliminates the problem for downstream users.
>> >> > >> >
>> >> > >> > Cheers,
>> >> > >> > Frederick
>> >> > >> >
>> >> > >> >
>> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
>> >> > >> >
>> >> > >> >> +1 to have binaries for both versions in Maven and as build to
>> >> > download.
>> >> > >> >>
>> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
>> >> > >> >> [hidden email]>:
>> >> > >> >>
>> >> > >> >> +1 for having binaries, I'm working on a Spark application
>> >> currently
>> >> > >> with
>> >> > >> >>> Scala 2.11 and having to rebuild everything when deploying
>> e.g. to
>> >> > EC2
>> >> > >> >>> is a
>> >> > >> >>> pain.
>> >> > >> >>>
>> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <[hidden email]>
>> >> > wrote:
>> >> > >> >>>
>> >> > >> >>> I agree with Till, but is this something you want to address in
>> >> this
>> >> > >> >>>> release already?
>> >> > >> >>>>
>> >> > >> >>>> I would postpone it to 1.0.0.
>> >> > >> >>>>
>> >> > >> >>>> – Ufuk
>> >> > >> >>>>
>> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <[hidden email]
>> >
>> >> > wrote:
>> >> > >> >>>>>
>> >> > >> >>>>> I would be in favor of deploying also Scala 2.11 artifacts to
>> >> > Maven
>> >> > >> >>>>>
>> >> > >> >>>> since
>> >> > >> >>>
>> >> > >> >>>> more and more people will try out Flink with Scala 2.11.
>> Having
>> >> the
>> >> > >> >>>>> dependencies in the Maven repository makes it considerably
>> >> easier
>> >> > for
>> >> > >> >>>>> people to get their Flink jobs running.
>> >> > >> >>>>>
>> >> > >> >>>>> Furthermore, I observed that people are not aware that our
>> >> > deployed
>> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10. As
>> a
>> >> > >> >>>>>
>> >> > >> >>>> consequence,
>> >> > >> >>>>
>> >> > >> >>>>> they mix flink dependencies with other dependencies pulling
>> in
>> >> > Scala
>> >> > >> >>>>>
>> >> > >> >>>> 2.11
>> >> > >> >>>
>> >> > >> >>>> and then they wonder that the program crashes. It would be,
>> imho,
>> >> > >> >>>>>
>> >> > >> >>>> clearer
>> >> > >> >>>
>> >> > >> >>>> if all our dependencies which depend on a specific Scala
>> version
>> >> > would
>> >> > >> >>>>>
>> >> > >> >>>> have
>> >> > >> >>>>
>> >> > >> >>>>> the corresponding Scala suffix appended.
>> >> > >> >>>>>
>> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of
>> >> upgrading
>> >> > >> to a
>> >> > >> >>>>> newer Scala version in the future, because then the artifacts
>> >> > >> wouldn't
>> >> > >> >>>>> share the same artifact name.
>> >> > >> >>>>>
>> >> > >> >>>>> Cheers,
>> >> > >> >>>>> Till
>> >> > >> >>>>>
>> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
>> >> > [hidden email]>
>> >> > >> >>>>>
>> >> > >> >>>> wrote:
>> >> > >> >>>>
>> >> > >> >>>>> Hi Flinksters,
>> >> > >> >>>>>>
>> >> > >> >>>>>> We have recently committed an easy way to change Flink's
>> Scala
>> >> > >> >>>>>>
>> >> > >> >>>>> version.
>> >> > >> >>>
>> >> > >> >>>> The
>> >> > >> >>>>
>> >> > >> >>>>> question arises now whether we should ship Scala 2.11 as
>> >> binaries
>> >> > and
>> >> > >> >>>>>>
>> >> > >> >>>>> via
>> >> > >> >>>>
>> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for Scala
>> 2.10
>> >> > and
>> >> > >> >>>>>>
>> >> > >> >>>>> 2.11.
>> >> > >> >>>>
>> >> > >> >>>>> However, I didn't create Maven artifacts. This follows our
>> >> current
>> >> > >> >>>>>>
>> >> > >> >>>>> shipping
>> >> > >> >>>>
>> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
>> >> > >> >>>>>>
>> >> > >> >>>>> dependencies
>> >> > >> >>>
>> >> > >> >>>> but
>> >> > >> >>>>
>> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>> >> > >> >>>>>>
>> >> > >> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
>> >> > >> >>>>>>
>> >> > >> >>>>>> If so, the next question arises: What version pattern
>> should we
>> >> > have
>> >> > >> >>>>>>
>> >> > >> >>>>> for
>> >> > >> >>>
>> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append
>> -hadoop1
>> >> > to
>> >> > >> >>>>>>
>> >> > >> >>>>> the
>> >> > >> >>>
>> >> > >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
>> >> > >> >>>>>>
>> >> > >> >>>>>> However, it is common practice to append the suffix to the
>> >> > >> artifactID
>> >> > >> >>>>>>
>> >> > >> >>>>> of
>> >> > >> >>>
>> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
>> >> > version=0.9.1.
>> >> > >> >>>>>>
>> >> > >> >>>>> This
>> >> > >> >>>>
>> >> > >> >>>>> has mostly historic reasons but is widely used.
>> >> > >> >>>>>>
>> >> > >> >>>>>> Whatever naming pattern we choose, it should be consistent.
>> I
>> >> > would
>> >> > >> be
>> >> > >> >>>>>>
>> >> > >> >>>>> in
>> >> > >> >>>>
>> >> > >> >>>>> favor of changing our artifact names to contain the Hadoop
>> and
>> >> > Scala
>> >> > >> >>>>>> version. This would also imply that all Scala dependent
>> Maven
>> >> > >> modules
>> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10
>> modules).
>> >> > >> >>>>>>
>> >> > >> >>>>>> Cheers,
>> >> > >> >>>>>> Max
>> >> > >> >>>>>>
>> >> > >> >>>>>>
>> >> > >> >>>>
>> >> > >> >
>> >> > >>
>> >> >
>> >>
>>

[Fabian Hueske-2]

Ah OK. Sorry, I misunderstood your intention.

2015-11-02 14:07 GMT+01:00 Maximilian Michels <[hidden email]>:

> > That would mean to have "flink-java_2.10" and "flink-java_2.11" artifacts
> > (and others that depend on flink-java and have no other Scala dependency)
> > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
>
>
> My suggestion was to keep the Scala unsuffixed Scala 2.10 and add a
> suffix for Scala 2.11. That's the way we are currently doing it (also
> deployed on Maven like this). For the next release after 0.10, we can
> do it properly.
>
> On Mon, Nov 2, 2015 at 1:30 PM, Chiwan Park <[hidden email]> wrote:
> > If we choose selective Scala version suffix for artifacts, we have to
> tell which artifacts have the version suffix to newcomers. Some artifacts
> such as "flink-java”, "flink-streaming-java" are easily recognized. But
> IMO, knowing whether artifacts such as "flink-ml", "flink-clients",
> "flink-table" have the version suffix or not is difficult for newcomers.
> >
> > This is why we are adding the version suffix to all Scala 2.11 artifacts
> currently. For Scala 2.10 artifacts, we aren’t adding the version suffix
> for Flink with Java users.
> >
> > I’m for adding the version suffix to Scala 2.10 artifacts also. But I’m
> not sure that removing the version suffix from Java-only artifacts would be
> good. As I said above, It seems difficult for newcomers.
> >
> > Regards,
> > Chiwan Park
> >
> > On November 2, 2015 at 8:19:15 PM, Fabian Hueske ([hidden email])
> wrote:
> >
> > That would mean to have "flink-java_2.10" and "flink-java_2.11" artifacts
> > (and others that depend on flink-java and have no other Scala dependency)
> > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
> >
> > Do we want that?
> >
> > 2015-11-02 11:37 GMT+01:00 Maximilian Michels <[hidden email]>:
> >
> >> I'm for leaving it as-is and renaming all artifacts which depend on
> >> Scala for the release following 0.10.
> >>
> >> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]>
> wrote:
> >> > OK, let me try to summarize the discussion (and please correct me if I
> >> got
> >> > something wrong).
> >> >
> >> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have to
> >> > release 2.11 artifacts for the 0.10.0 release version as well.
> >> >
> >> > 2) Everybody agrees to appropriately tag all artifacts that have a
> >> > (transitive) Scala dependency. ATM, that would also include flink-java
> >> > which is a bit awkward. The Scala dependency in flink-java originates
> >> from
> >> > the Chill library which is used to obtain a Kryo serializer which is
> >> > initialized with serializers for Scala classes. We could resolve this
> >> issue
> >> > by providing Java and Scala specific implementations of the Kryo
> >> > serializers and have KryoTypeInfos for Java and Scala.
> >> >
> >> > The question to answer right now is, do we want to have "correctly"
> >> labeled
> >> > artifacts for the next 0.10.0 release or do we defer that for 1.0?
> >> > If we want to solve it for 0.10.0 we need to cancel the current RC and
> >> > provide a fix to remove the Scala dependency in flink-java, IMO.
> >> >
> >> > Opinions?
> >> >
> >> > Cheers, Fabian
> >> >
> >> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:
> >> >
> >> >> +1 for the approach discusses here, and for removing Scala
> dependencies
> >> >> from modules that can be Scala independent.
> >> >>
> >> >> It would be nice if pure Java users would not see any Scala
> versioning
> >> (on
> >> >> flink-core, flink-java, later also flink-sreaming-java). I guess for
> any
> >> >> runtime-related parts (including flink-client and currently all
> >> streaming
> >> >> projects), we need the Scala versions...
> >> >>
> >> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]>
> >> wrote:
> >> >>
> >> >> > Good point. Didn't know that. We can still add them for the
> release.
> >> >> >
> >> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
> >> >> > <[hidden email]> wrote:
> >> >> > > My two cents - there are already Maven artifacts deployed for
> 2.11
> >> in
> >> >> the
> >> >> > > SNAPSHOT repository. I think it might be confusing if they
> suddenly
> >> >> > > disappear for the stable release.
> >> >> > >
> >> >> > >
> >> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]>:
> >> >> > >
> >> >> > >> Seems like we agree that we need artifacts for different
> versions
> >> of
> >> >> > Scala
> >> >> > >> on Maven. There also seems to be a preference for including the
> >> >> version
> >> >> > in
> >> >> > >> the artifact name.
> >> >> > >>
> >> >> > >> I've created an issue and marked it to be resolved for 1.0. For
> the
> >> >> 0.10
> >> >> > >> release, we will have binaries but no Maven artifacts. The
> biggest
> >> >> > >> challenge I see is to remove Scala from as many modules as
> >> possible.
> >> >> For
> >> >> > >> example, flink-java depends on Scala at the moment..
> >> >> > >>
> >> >> > >> https://issues.apache.org/jira/browse/FLINK-2940
> >> >> > >>
> >> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
> >> >> > [hidden email]>
> >> >> > >> wrote:
> >> >> > >>
> >> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries
> for
> >> >> both
> >> >> > >> > versions in Maven and explicitly "scala versioned".
> >> >> > >> >
> >> >> > >> > Some background on this for those not as familiar with scala
> >> >> > versioning:
> >> >> > >> >
> >> >> > >> > It's considered best practice to label what version of scala a
> >> >> library
> >> >> > >> > uses in the artifact id.
> >> >> > >> >
> >> >> > >> > The reason is compiled scala code is only compatible with the
> >> major
> >> >> > >> > version of scala it was compiled for. For example, a library
> >> >> > compatible
> >> >> > >> > with 2.10 is not compatible with 2.11. The same will be true
> with
> >> >> 2.12
> >> >> > >> once
> >> >> > >> > it is released. Mixing versions will result in undefined
> behavior
> >> >> > which
> >> >> > >> > will likely manifest itself as runtime exceptions.
> >> >> > >> >
> >> >> > >> > The convention to fix this problem is for all published
> >> libraries to
> >> >> > >> > specify the version of scala they are compatible with. Leaving
> >> out
> >> >> the
> >> >> > >> > scala version in a library is akin to saying "We don't depend
> on
> >> >> scala
> >> >> > >> for
> >> >> > >> > this library, so feel free to use whatever you want." Sbt
> users
> >> will
> >> >> > >> > typically specify the version of scala they use and tooling is
> >> built
> >> >> > >> around
> >> >> > >> > ensuring consistency with the "%%" operator.
> >> >> > >> >
> >> >> > >> > E.g.
> >> >> > >> >
> >> >> > >> > scalaVersion := "2.11.4"
> >> >> > >> >
> >> >> > >> > // this resolves to to artifactID: "scalacheck_2.11"
> >> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" %
> >> "1.12.0" %
> >> >> > >> "test"
> >> >> > >> >
> >> >> > >> > The most important part of this is that the scala version is
> >> >> explicit
> >> >> > >> > which eliminates the problem for downstream users.
> >> >> > >> >
> >> >> > >> > Cheers,
> >> >> > >> > Frederick
> >> >> > >> >
> >> >> > >> >
> >> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
> >> >> > >> >
> >> >> > >> >> +1 to have binaries for both versions in Maven and as build
> to
> >> >> > download.
> >> >> > >> >>
> >> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> >> >> > >> >> [hidden email]>:
> >> >> > >> >>
> >> >> > >> >> +1 for having binaries, I'm working on a Spark application
> >> >> currently
> >> >> > >> with
> >> >> > >> >>> Scala 2.11 and having to rebuild everything when deploying
> >> e.g. to
> >> >> > EC2
> >> >> > >> >>> is a
> >> >> > >> >>> pain.
> >> >> > >> >>>
> >> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <
> [hidden email]>
> >> >> > wrote:
> >> >> > >> >>>
> >> >> > >> >>> I agree with Till, but is this something you want to
> address in
> >> >> this
> >> >> > >> >>>> release already?
> >> >> > >> >>>>
> >> >> > >> >>>> I would postpone it to 1.0.0.
> >> >> > >> >>>>
> >> >> > >> >>>> – Ufuk
> >> >> > >> >>>>
> >> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <
> [hidden email]
> >> >
> >> >> > wrote:
> >> >> > >> >>>>>
> >> >> > >> >>>>> I would be in favor of deploying also Scala 2.11
> artifacts to
> >> >> > Maven
> >> >> > >> >>>>>
> >> >> > >> >>>> since
> >> >> > >> >>>
> >> >> > >> >>>> more and more people will try out Flink with Scala 2.11.
> >> Having
> >> >> the
> >> >> > >> >>>>> dependencies in the Maven repository makes it considerably
> >> >> easier
> >> >> > for
> >> >> > >> >>>>> people to get their Flink jobs running.
> >> >> > >> >>>>>
> >> >> > >> >>>>> Furthermore, I observed that people are not aware that our
> >> >> > deployed
> >> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala 2.10.
> As
> >> a
> >> >> > >> >>>>>
> >> >> > >> >>>> consequence,
> >> >> > >> >>>>
> >> >> > >> >>>>> they mix flink dependencies with other dependencies
> pulling
> >> in
> >> >> > Scala
> >> >> > >> >>>>>
> >> >> > >> >>>> 2.11
> >> >> > >> >>>
> >> >> > >> >>>> and then they wonder that the program crashes. It would be,
> >> imho,
> >> >> > >> >>>>>
> >> >> > >> >>>> clearer
> >> >> > >> >>>
> >> >> > >> >>>> if all our dependencies which depend on a specific Scala
> >> version
> >> >> > would
> >> >> > >> >>>>>
> >> >> > >> >>>> have
> >> >> > >> >>>>
> >> >> > >> >>>>> the corresponding Scala suffix appended.
> >> >> > >> >>>>>
> >> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of
> >> >> upgrading
> >> >> > >> to a
> >> >> > >> >>>>> newer Scala version in the future, because then the
> artifacts
> >> >> > >> wouldn't
> >> >> > >> >>>>> share the same artifact name.
> >> >> > >> >>>>>
> >> >> > >> >>>>> Cheers,
> >> >> > >> >>>>> Till
> >> >> > >> >>>>>
> >> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
> >> >> > [hidden email]>
> >> >> > >> >>>>>
> >> >> > >> >>>> wrote:
> >> >> > >> >>>>
> >> >> > >> >>>>> Hi Flinksters,
> >> >> > >> >>>>>>
> >> >> > >> >>>>>> We have recently committed an easy way to change Flink's
> >> Scala
> >> >> > >> >>>>>>
> >> >> > >> >>>>> version.
> >> >> > >> >>>
> >> >> > >> >>>> The
> >> >> > >> >>>>
> >> >> > >> >>>>> question arises now whether we should ship Scala 2.11 as
> >> >> binaries
> >> >> > and
> >> >> > >> >>>>>>
> >> >> > >> >>>>> via
> >> >> > >> >>>>
> >> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for
> Scala
> >> 2.10
> >> >> > and
> >> >> > >> >>>>>>
> >> >> > >> >>>>> 2.11.
> >> >> > >> >>>>
> >> >> > >> >>>>> However, I didn't create Maven artifacts. This follows our
> >> >> current
> >> >> > >> >>>>>>
> >> >> > >> >>>>> shipping
> >> >> > >> >>>>
> >> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0 Maven
> >> >> > >> >>>>>>
> >> >> > >> >>>>> dependencies
> >> >> > >> >>>
> >> >> > >> >>>> but
> >> >> > >> >>>>
> >> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> >> >> > >> >>>>>>
> >> >> > >> >>>>>> Should we also upload Maven dependencies for Scala 2.11?
> >> >> > >> >>>>>>
> >> >> > >> >>>>>> If so, the next question arises: What version pattern
> >> should we
> >> >> > have
> >> >> > >> >>>>>>
> >> >> > >> >>>>> for
> >> >> > >> >>>
> >> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append
> >> -hadoop1
> >> >> > to
> >> >> > >> >>>>>>
> >> >> > >> >>>>> the
> >> >> > >> >>>
> >> >> > >> >>>> VERSION, e.g. artifactID=flink-core, version=0.9.1-hadoop1.
> >> >> > >> >>>>>>
> >> >> > >> >>>>>> However, it is common practice to append the suffix to
> the
> >> >> > >> artifactID
> >> >> > >> >>>>>>
> >> >> > >> >>>>> of
> >> >> > >> >>>
> >> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
> >> >> > version=0.9.1.
> >> >> > >> >>>>>>
> >> >> > >> >>>>> This
> >> >> > >> >>>>
> >> >> > >> >>>>> has mostly historic reasons but is widely used.
> >> >> > >> >>>>>>
> >> >> > >> >>>>>> Whatever naming pattern we choose, it should be
> consistent.
> >> I
> >> >> > would
> >> >> > >> be
> >> >> > >> >>>>>>
> >> >> > >> >>>>> in
> >> >> > >> >>>>
> >> >> > >> >>>>> favor of changing our artifact names to contain the Hadoop
> >> and
> >> >> > Scala
> >> >> > >> >>>>>> version. This would also imply that all Scala dependent
> >> Maven
> >> >> > >> modules
> >> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10
> >> modules).
> >> >> > >> >>>>>>
> >> >> > >> >>>>>> Cheers,
> >> >> > >> >>>>>> Max
> >> >> > >> >>>>>>
> >> >> > >> >>>>>>
> >> >> > >> >>>>
> >> >> > >> >
> >> >> > >>
> >> >> >
> >> >>
> >>
>

[Stephan Ewen]

+1 for Max' suggestion to fix that for 1.0 (next release).

Hot fixing of this thing so short before a release is a bit risky in my
opinion. It is easy to make errors (overlooking something, error not
visible because of cached older dependencies, ...), it happened more than
once during version upgrades, maven project re-organizations, etc.

Doing it after 0.10 and having a few week to let it sink in and errors
surface would probably be much safer...

On Mon, Nov 2, 2015 at 5:19 AM, Fabian Hueske <[hidden email]> wrote:

> Ah OK. Sorry, I misunderstood your intention.
>
> 2015-11-02 14:07 GMT+01:00 Maximilian Michels <[hidden email]>:
>
> > > That would mean to have "flink-java_2.10" and "flink-java_2.11"
> artifacts
> > > (and others that depend on flink-java and have no other Scala
> dependency)
> > > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
> >
> >
> > My suggestion was to keep the Scala unsuffixed Scala 2.10 and add a
> > suffix for Scala 2.11. That's the way we are currently doing it (also
> > deployed on Maven like this). For the next release after 0.10, we can
> > do it properly.
> >
> > On Mon, Nov 2, 2015 at 1:30 PM, Chiwan Park <[hidden email]>
> wrote:
> > > If we choose selective Scala version suffix for artifacts, we have to
> > tell which artifacts have the version suffix to newcomers. Some artifacts
> > such as "flink-java”, "flink-streaming-java" are easily recognized. But
> > IMO, knowing whether artifacts such as "flink-ml", "flink-clients",
> > "flink-table" have the version suffix or not is difficult for newcomers.
> > >
> > > This is why we are adding the version suffix to all Scala 2.11
> artifacts
> > currently. For Scala 2.10 artifacts, we aren’t adding the version suffix
> > for Flink with Java users.
> > >
> > > I’m for adding the version suffix to Scala 2.10 artifacts also. But I’m
> > not sure that removing the version suffix from Java-only artifacts would
> be
> > good. As I said above, It seems difficult for newcomers.
> > >
> > > Regards,
> > > Chiwan Park
> > >
> > > On November 2, 2015 at 8:19:15 PM, Fabian Hueske ([hidden email])
> > wrote:
> > >
> > > That would mean to have "flink-java_2.10" and "flink-java_2.11"
> artifacts
> > > (and others that depend on flink-java and have no other Scala
> dependency)
> > > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
> > >
> > > Do we want that?
> > >
> > > 2015-11-02 11:37 GMT+01:00 Maximilian Michels <[hidden email]>:
> > >
> > >> I'm for leaving it as-is and renaming all artifacts which depend on
> > >> Scala for the release following 0.10.
> > >>
> > >> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]>
> > wrote:
> > >> > OK, let me try to summarize the discussion (and please correct me
> if I
> > >> got
> > >> > something wrong).
> > >> >
> > >> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have
> to
> > >> > release 2.11 artifacts for the 0.10.0 release version as well.
> > >> >
> > >> > 2) Everybody agrees to appropriately tag all artifacts that have a
> > >> > (transitive) Scala dependency. ATM, that would also include
> flink-java
> > >> > which is a bit awkward. The Scala dependency in flink-java
> originates
> > >> from
> > >> > the Chill library which is used to obtain a Kryo serializer which is
> > >> > initialized with serializers for Scala classes. We could resolve
> this
> > >> issue
> > >> > by providing Java and Scala specific implementations of the Kryo
> > >> > serializers and have KryoTypeInfos for Java and Scala.
> > >> >
> > >> > The question to answer right now is, do we want to have "correctly"
> > >> labeled
> > >> > artifacts for the next 0.10.0 release or do we defer that for 1.0?
> > >> > If we want to solve it for 0.10.0 we need to cancel the current RC
> and
> > >> > provide a fix to remove the Scala dependency in flink-java, IMO.
> > >> >
> > >> > Opinions?
> > >> >
> > >> > Cheers, Fabian
> > >> >
> > >> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:
> > >> >
> > >> >> +1 for the approach discusses here, and for removing Scala
> > dependencies
> > >> >> from modules that can be Scala independent.
> > >> >>
> > >> >> It would be nice if pure Java users would not see any Scala
> > versioning
> > >> (on
> > >> >> flink-core, flink-java, later also flink-sreaming-java). I guess
> for
> > any
> > >> >> runtime-related parts (including flink-client and currently all
> > >> streaming
> > >> >> projects), we need the Scala versions...
> > >> >>
> > >> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <[hidden email]
> >
> > >> wrote:
> > >> >>
> > >> >> > Good point. Didn't know that. We can still add them for the
> > release.
> > >> >> >
> > >> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
> > >> >> > <[hidden email]> wrote:
> > >> >> > > My two cents - there are already Maven artifacts deployed for
> > 2.11
> > >> in
> > >> >> the
> > >> >> > > SNAPSHOT repository. I think it might be confusing if they
> > suddenly
> > >> >> > > disappear for the stable release.
> > >> >> > >
> > >> >> > >
> > >> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <[hidden email]
> >:
> > >> >> > >
> > >> >> > >> Seems like we agree that we need artifacts for different
> > versions
> > >> of
> > >> >> > Scala
> > >> >> > >> on Maven. There also seems to be a preference for including
> the
> > >> >> version
> > >> >> > in
> > >> >> > >> the artifact name.
> > >> >> > >>
> > >> >> > >> I've created an issue and marked it to be resolved for 1.0.
> For
> > the
> > >> >> 0.10
> > >> >> > >> release, we will have binaries but no Maven artifacts. The
> > biggest
> > >> >> > >> challenge I see is to remove Scala from as many modules as
> > >> possible.
> > >> >> For
> > >> >> > >> example, flink-java depends on Scala at the moment..
> > >> >> > >>
> > >> >> > >> https://issues.apache.org/jira/browse/FLINK-2940
> > >> >> > >>
> > >> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
> > >> >> > [hidden email]>
> > >> >> > >> wrote:
> > >> >> > >>
> > >> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have binaries
> > for
> > >> >> both
> > >> >> > >> > versions in Maven and explicitly "scala versioned".
> > >> >> > >> >
> > >> >> > >> > Some background on this for those not as familiar with scala
> > >> >> > versioning:
> > >> >> > >> >
> > >> >> > >> > It's considered best practice to label what version of
> scala a
> > >> >> library
> > >> >> > >> > uses in the artifact id.
> > >> >> > >> >
> > >> >> > >> > The reason is compiled scala code is only compatible with
> the
> > >> major
> > >> >> > >> > version of scala it was compiled for. For example, a library
> > >> >> > compatible
> > >> >> > >> > with 2.10 is not compatible with 2.11. The same will be true
> > with
> > >> >> 2.12
> > >> >> > >> once
> > >> >> > >> > it is released. Mixing versions will result in undefined
> > behavior
> > >> >> > which
> > >> >> > >> > will likely manifest itself as runtime exceptions.
> > >> >> > >> >
> > >> >> > >> > The convention to fix this problem is for all published
> > >> libraries to
> > >> >> > >> > specify the version of scala they are compatible with.
> Leaving
> > >> out
> > >> >> the
> > >> >> > >> > scala version in a library is akin to saying "We don't
> depend
> > on
> > >> >> scala
> > >> >> > >> for
> > >> >> > >> > this library, so feel free to use whatever you want." Sbt
> > users
> > >> will
> > >> >> > >> > typically specify the version of scala they use and tooling
> is
> > >> built
> > >> >> > >> around
> > >> >> > >> > ensuring consistency with the "%%" operator.
> > >> >> > >> >
> > >> >> > >> > E.g.
> > >> >> > >> >
> > >> >> > >> > scalaVersion := "2.11.4"
> > >> >> > >> >
> > >> >> > >> > // this resolves to to artifactID: "scalacheck_2.11"
> > >> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" %
> > >> "1.12.0" %
> > >> >> > >> "test"
> > >> >> > >> >
> > >> >> > >> > The most important part of this is that the scala version is
> > >> >> explicit
> > >> >> > >> > which eliminates the problem for downstream users.
> > >> >> > >> >
> > >> >> > >> > Cheers,
> > >> >> > >> > Frederick
> > >> >> > >> >
> > >> >> > >> >
> > >> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
> > >> >> > >> >
> > >> >> > >> >> +1 to have binaries for both versions in Maven and as build
> > to
> > >> >> > download.
> > >> >> > >> >>
> > >> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> > >> >> > >> >> [hidden email]>:
> > >> >> > >> >>
> > >> >> > >> >> +1 for having binaries, I'm working on a Spark application
> > >> >> currently
> > >> >> > >> with
> > >> >> > >> >>> Scala 2.11 and having to rebuild everything when deploying
> > >> e.g. to
> > >> >> > EC2
> > >> >> > >> >>> is a
> > >> >> > >> >>> pain.
> > >> >> > >> >>>
> > >> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <
> > [hidden email]>
> > >> >> > wrote:
> > >> >> > >> >>>
> > >> >> > >> >>> I agree with Till, but is this something you want to
> > address in
> > >> >> this
> > >> >> > >> >>>> release already?
> > >> >> > >> >>>>
> > >> >> > >> >>>> I would postpone it to 1.0.0.
> > >> >> > >> >>>>
> > >> >> > >> >>>> – Ufuk
> > >> >> > >> >>>>
> > >> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <
> > [hidden email]
> > >> >
> > >> >> > wrote:
> > >> >> > >> >>>>>
> > >> >> > >> >>>>> I would be in favor of deploying also Scala 2.11
> > artifacts to
> > >> >> > Maven
> > >> >> > >> >>>>>
> > >> >> > >> >>>> since
> > >> >> > >> >>>
> > >> >> > >> >>>> more and more people will try out Flink with Scala 2.11.
> > >> Having
> > >> >> the
> > >> >> > >> >>>>> dependencies in the Maven repository makes it
> considerably
> > >> >> easier
> > >> >> > for
> > >> >> > >> >>>>> people to get their Flink jobs running.
> > >> >> > >> >>>>>
> > >> >> > >> >>>>> Furthermore, I observed that people are not aware that
> our
> > >> >> > deployed
> > >> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala
> 2.10.
> > As
> > >> a
> > >> >> > >> >>>>>
> > >> >> > >> >>>> consequence,
> > >> >> > >> >>>>
> > >> >> > >> >>>>> they mix flink dependencies with other dependencies
> > pulling
> > >> in
> > >> >> > Scala
> > >> >> > >> >>>>>
> > >> >> > >> >>>> 2.11
> > >> >> > >> >>>
> > >> >> > >> >>>> and then they wonder that the program crashes. It would
> be,
> > >> imho,
> > >> >> > >> >>>>>
> > >> >> > >> >>>> clearer
> > >> >> > >> >>>
> > >> >> > >> >>>> if all our dependencies which depend on a specific Scala
> > >> version
> > >> >> > would
> > >> >> > >> >>>>>
> > >> >> > >> >>>> have
> > >> >> > >> >>>>
> > >> >> > >> >>>>> the corresponding Scala suffix appended.
> > >> >> > >> >>>>>
> > >> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle of
> > >> >> upgrading
> > >> >> > >> to a
> > >> >> > >> >>>>> newer Scala version in the future, because then the
> > artifacts
> > >> >> > >> wouldn't
> > >> >> > >> >>>>> share the same artifact name.
> > >> >> > >> >>>>>
> > >> >> > >> >>>>> Cheers,
> > >> >> > >> >>>>> Till
> > >> >> > >> >>>>>
> > >> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
> > >> >> > [hidden email]>
> > >> >> > >> >>>>>
> > >> >> > >> >>>> wrote:
> > >> >> > >> >>>>
> > >> >> > >> >>>>> Hi Flinksters,
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>>> We have recently committed an easy way to change
> Flink's
> > >> Scala
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> version.
> > >> >> > >> >>>
> > >> >> > >> >>>> The
> > >> >> > >> >>>>
> > >> >> > >> >>>>> question arises now whether we should ship Scala 2.11 as
> > >> >> binaries
> > >> >> > and
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> via
> > >> >> > >> >>>>
> > >> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for
> > Scala
> > >> 2.10
> > >> >> > and
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> 2.11.
> > >> >> > >> >>>>
> > >> >> > >> >>>>> However, I didn't create Maven artifacts. This follows
> our
> > >> >> current
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> shipping
> > >> >> > >> >>>>
> > >> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0
> Maven
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> dependencies
> > >> >> > >> >>>
> > >> >> > >> >>>> but
> > >> >> > >> >>>>
> > >> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>>> Should we also upload Maven dependencies for Scala
> 2.11?
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>>> If so, the next question arises: What version pattern
> > >> should we
> > >> >> > have
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> for
> > >> >> > >> >>>
> > >> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we append
> > >> -hadoop1
> > >> >> > to
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> the
> > >> >> > >> >>>
> > >> >> > >> >>>> VERSION, e.g. artifactID=flink-core,
> version=0.9.1-hadoop1.
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>>> However, it is common practice to append the suffix to
> > the
> > >> >> > >> artifactID
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> of
> > >> >> > >> >>>
> > >> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
> > >> >> > version=0.9.1.
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> This
> > >> >> > >> >>>>
> > >> >> > >> >>>>> has mostly historic reasons but is widely used.
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>>> Whatever naming pattern we choose, it should be
> > consistent.
> > >> I
> > >> >> > would
> > >> >> > >> be
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>> in
> > >> >> > >> >>>>
> > >> >> > >> >>>>> favor of changing our artifact names to contain the
> Hadoop
> > >> and
> > >> >> > Scala
> > >> >> > >> >>>>>> version. This would also imply that all Scala dependent
> > >> Maven
> > >> >> > >> modules
> > >> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10
> > >> modules).
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>>> Cheers,
> > >> >> > >> >>>>>> Max
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>>>
> > >> >> > >> >>>>
> > >> >> > >> >
> > >> >> > >>
> > >> >> >
> > >> >>
> > >>
> >
>

[Fabian Hueske-2]

+1 for that.

2015-11-02 20:52 GMT+01:00 Stephan Ewen <[hidden email]>:

> +1 for Max' suggestion to fix that for 1.0 (next release).
>
> Hot fixing of this thing so short before a release is a bit risky in my
> opinion. It is easy to make errors (overlooking something, error not
> visible because of cached older dependencies, ...), it happened more than
> once during version upgrades, maven project re-organizations, etc.
>
> Doing it after 0.10 and having a few week to let it sink in and errors
> surface would probably be much safer...
>
> On Mon, Nov 2, 2015 at 5:19 AM, Fabian Hueske <[hidden email]> wrote:
>
> > Ah OK. Sorry, I misunderstood your intention.
> >
> > 2015-11-02 14:07 GMT+01:00 Maximilian Michels <[hidden email]>:
> >
> > > > That would mean to have "flink-java_2.10" and "flink-java_2.11"
> > artifacts
> > > > (and others that depend on flink-java and have no other Scala
> > dependency)
> > > > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
> > >
> > >
> > > My suggestion was to keep the Scala unsuffixed Scala 2.10 and add a
> > > suffix for Scala 2.11. That's the way we are currently doing it (also
> > > deployed on Maven like this). For the next release after 0.10, we can
> > > do it properly.
> > >
> > > On Mon, Nov 2, 2015 at 1:30 PM, Chiwan Park <[hidden email]>
> > wrote:
> > > > If we choose selective Scala version suffix for artifacts, we have to
> > > tell which artifacts have the version suffix to newcomers. Some
> artifacts
> > > such as "flink-java”, "flink-streaming-java" are easily recognized. But
> > > IMO, knowing whether artifacts such as "flink-ml", "flink-clients",
> > > "flink-table" have the version suffix or not is difficult for
> newcomers.
> > > >
> > > > This is why we are adding the version suffix to all Scala 2.11
> > artifacts
> > > currently. For Scala 2.10 artifacts, we aren’t adding the version
> suffix
> > > for Flink with Java users.
> > > >
> > > > I’m for adding the version suffix to Scala 2.10 artifacts also. But
> I’m
> > > not sure that removing the version suffix from Java-only artifacts
> would
> > be
> > > good. As I said above, It seems difficult for newcomers.
> > > >
> > > > Regards,
> > > > Chiwan Park
> > > >
> > > > On November 2, 2015 at 8:19:15 PM, Fabian Hueske ([hidden email])
> > > wrote:
> > > >
> > > > That would mean to have "flink-java_2.10" and "flink-java_2.11"
> > artifacts
> > > > (and others that depend on flink-java and have no other Scala
> > dependency)
> > > > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
> > > >
> > > > Do we want that?
> > > >
> > > > 2015-11-02 11:37 GMT+01:00 Maximilian Michels <[hidden email]>:
> > > >
> > > >> I'm for leaving it as-is and renaming all artifacts which depend on
> > > >> Scala for the release following 0.10.
> > > >>
> > > >> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]>
> > > wrote:
> > > >> > OK, let me try to summarize the discussion (and please correct me
> > if I
> > > >> got
> > > >> > something wrong).
> > > >> >
> > > >> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have
> > to
> > > >> > release 2.11 artifacts for the 0.10.0 release version as well.
> > > >> >
> > > >> > 2) Everybody agrees to appropriately tag all artifacts that have a
> > > >> > (transitive) Scala dependency. ATM, that would also include
> > flink-java
> > > >> > which is a bit awkward. The Scala dependency in flink-java
> > originates
> > > >> from
> > > >> > the Chill library which is used to obtain a Kryo serializer which
> is
> > > >> > initialized with serializers for Scala classes. We could resolve
> > this
> > > >> issue
> > > >> > by providing Java and Scala specific implementations of the Kryo
> > > >> > serializers and have KryoTypeInfos for Java and Scala.
> > > >> >
> > > >> > The question to answer right now is, do we want to have
> "correctly"
> > > >> labeled
> > > >> > artifacts for the next 0.10.0 release or do we defer that for 1.0?
> > > >> > If we want to solve it for 0.10.0 we need to cancel the current RC
> > and
> > > >> > provide a fix to remove the Scala dependency in flink-java, IMO.
> > > >> >
> > > >> > Opinions?
> > > >> >
> > > >> > Cheers, Fabian
> > > >> >
> > > >> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:
> > > >> >
> > > >> >> +1 for the approach discusses here, and for removing Scala
> > > dependencies
> > > >> >> from modules that can be Scala independent.
> > > >> >>
> > > >> >> It would be nice if pure Java users would not see any Scala
> > > versioning
> > > >> (on
> > > >> >> flink-core, flink-java, later also flink-sreaming-java). I guess
> > for
> > > any
> > > >> >> runtime-related parts (including flink-client and currently all
> > > >> streaming
> > > >> >> projects), we need the Scala versions...
> > > >> >>
> > > >> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <
> [hidden email]
> > >
> > > >> wrote:
> > > >> >>
> > > >> >> > Good point. Didn't know that. We can still add them for the
> > > release.
> > > >> >> >
> > > >> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
> > > >> >> > <[hidden email]> wrote:
> > > >> >> > > My two cents - there are already Maven artifacts deployed for
> > > 2.11
> > > >> in
> > > >> >> the
> > > >> >> > > SNAPSHOT repository. I think it might be confusing if they
> > > suddenly
> > > >> >> > > disappear for the stable release.
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <
> [hidden email]
> > >:
> > > >> >> > >
> > > >> >> > >> Seems like we agree that we need artifacts for different
> > > versions
> > > >> of
> > > >> >> > Scala
> > > >> >> > >> on Maven. There also seems to be a preference for including
> > the
> > > >> >> version
> > > >> >> > in
> > > >> >> > >> the artifact name.
> > > >> >> > >>
> > > >> >> > >> I've created an issue and marked it to be resolved for 1.0.
> > For
> > > the
> > > >> >> 0.10
> > > >> >> > >> release, we will have binaries but no Maven artifacts. The
> > > biggest
> > > >> >> > >> challenge I see is to remove Scala from as many modules as
> > > >> possible.
> > > >> >> For
> > > >> >> > >> example, flink-java depends on Scala at the moment..
> > > >> >> > >>
> > > >> >> > >> https://issues.apache.org/jira/browse/FLINK-2940
> > > >> >> > >>
> > > >> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
> > > >> >> > [hidden email]>
> > > >> >> > >> wrote:
> > > >> >> > >>
> > > >> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have
> binaries
> > > for
> > > >> >> both
> > > >> >> > >> > versions in Maven and explicitly "scala versioned".
> > > >> >> > >> >
> > > >> >> > >> > Some background on this for those not as familiar with
> scala
> > > >> >> > versioning:
> > > >> >> > >> >
> > > >> >> > >> > It's considered best practice to label what version of
> > scala a
> > > >> >> library
> > > >> >> > >> > uses in the artifact id.
> > > >> >> > >> >
> > > >> >> > >> > The reason is compiled scala code is only compatible with
> > the
> > > >> major
> > > >> >> > >> > version of scala it was compiled for. For example, a
> library
> > > >> >> > compatible
> > > >> >> > >> > with 2.10 is not compatible with 2.11. The same will be
> true
> > > with
> > > >> >> 2.12
> > > >> >> > >> once
> > > >> >> > >> > it is released. Mixing versions will result in undefined
> > > behavior
> > > >> >> > which
> > > >> >> > >> > will likely manifest itself as runtime exceptions.
> > > >> >> > >> >
> > > >> >> > >> > The convention to fix this problem is for all published
> > > >> libraries to
> > > >> >> > >> > specify the version of scala they are compatible with.
> > Leaving
> > > >> out
> > > >> >> the
> > > >> >> > >> > scala version in a library is akin to saying "We don't
> > depend
> > > on
> > > >> >> scala
> > > >> >> > >> for
> > > >> >> > >> > this library, so feel free to use whatever you want." Sbt
> > > users
> > > >> will
> > > >> >> > >> > typically specify the version of scala they use and
> tooling
> > is
> > > >> built
> > > >> >> > >> around
> > > >> >> > >> > ensuring consistency with the "%%" operator.
> > > >> >> > >> >
> > > >> >> > >> > E.g.
> > > >> >> > >> >
> > > >> >> > >> > scalaVersion := "2.11.4"
> > > >> >> > >> >
> > > >> >> > >> > // this resolves to to artifactID: "scalacheck_2.11"
> > > >> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" %
> > > >> "1.12.0" %
> > > >> >> > >> "test"
> > > >> >> > >> >
> > > >> >> > >> > The most important part of this is that the scala version
> is
> > > >> >> explicit
> > > >> >> > >> > which eliminates the problem for downstream users.
> > > >> >> > >> >
> > > >> >> > >> > Cheers,
> > > >> >> > >> > Frederick
> > > >> >> > >> >
> > > >> >> > >> >
> > > >> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
> > > >> >> > >> >
> > > >> >> > >> >> +1 to have binaries for both versions in Maven and as
> build
> > > to
> > > >> >> > download.
> > > >> >> > >> >>
> > > >> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
> > > >> >> > >> >> [hidden email]>:
> > > >> >> > >> >>
> > > >> >> > >> >> +1 for having binaries, I'm working on a Spark
> application
> > > >> >> currently
> > > >> >> > >> with
> > > >> >> > >> >>> Scala 2.11 and having to rebuild everything when
> deploying
> > > >> e.g. to
> > > >> >> > EC2
> > > >> >> > >> >>> is a
> > > >> >> > >> >>> pain.
> > > >> >> > >> >>>
> > > >> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <
> > > [hidden email]>
> > > >> >> > wrote:
> > > >> >> > >> >>>
> > > >> >> > >> >>> I agree with Till, but is this something you want to
> > > address in
> > > >> >> this
> > > >> >> > >> >>>> release already?
> > > >> >> > >> >>>>
> > > >> >> > >> >>>> I would postpone it to 1.0.0.
> > > >> >> > >> >>>>
> > > >> >> > >> >>>> – Ufuk
> > > >> >> > >> >>>>
> > > >> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <
> > > [hidden email]
> > > >> >
> > > >> >> > wrote:
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>>> I would be in favor of deploying also Scala 2.11
> > > artifacts to
> > > >> >> > Maven
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>> since
> > > >> >> > >> >>>
> > > >> >> > >> >>>> more and more people will try out Flink with Scala
> 2.11.
> > > >> Having
> > > >> >> the
> > > >> >> > >> >>>>> dependencies in the Maven repository makes it
> > considerably
> > > >> >> easier
> > > >> >> > for
> > > >> >> > >> >>>>> people to get their Flink jobs running.
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>>> Furthermore, I observed that people are not aware that
> > our
> > > >> >> > deployed
> > > >> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala
> > 2.10.
> > > As
> > > >> a
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>> consequence,
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> they mix flink dependencies with other dependencies
> > > pulling
> > > >> in
> > > >> >> > Scala
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>> 2.11
> > > >> >> > >> >>>
> > > >> >> > >> >>>> and then they wonder that the program crashes. It would
> > be,
> > > >> imho,
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>> clearer
> > > >> >> > >> >>>
> > > >> >> > >> >>>> if all our dependencies which depend on a specific
> Scala
> > > >> version
> > > >> >> > would
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>> have
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> the corresponding Scala suffix appended.
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle
> of
> > > >> >> upgrading
> > > >> >> > >> to a
> > > >> >> > >> >>>>> newer Scala version in the future, because then the
> > > artifacts
> > > >> >> > >> wouldn't
> > > >> >> > >> >>>>> share the same artifact name.
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>>> Cheers,
> > > >> >> > >> >>>>> Till
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
> > > >> >> > [hidden email]>
> > > >> >> > >> >>>>>
> > > >> >> > >> >>>> wrote:
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> Hi Flinksters,
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>>> We have recently committed an easy way to change
> > Flink's
> > > >> Scala
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> version.
> > > >> >> > >> >>>
> > > >> >> > >> >>>> The
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> question arises now whether we should ship Scala 2.11
> as
> > > >> >> binaries
> > > >> >> > and
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> via
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for
> > > Scala
> > > >> 2.10
> > > >> >> > and
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> 2.11.
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> However, I didn't create Maven artifacts. This follows
> > our
> > > >> >> current
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> shipping
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0
> > Maven
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> dependencies
> > > >> >> > >> >>>
> > > >> >> > >> >>>> but
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>>> Should we also upload Maven dependencies for Scala
> > 2.11?
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>>> If so, the next question arises: What version pattern
> > > >> should we
> > > >> >> > have
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> for
> > > >> >> > >> >>>
> > > >> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we
> append
> > > >> -hadoop1
> > > >> >> > to
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> the
> > > >> >> > >> >>>
> > > >> >> > >> >>>> VERSION, e.g. artifactID=flink-core,
> > version=0.9.1-hadoop1.
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>>> However, it is common practice to append the suffix
> to
> > > the
> > > >> >> > >> artifactID
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> of
> > > >> >> > >> >>>
> > > >> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
> > > >> >> > version=0.9.1.
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> This
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> has mostly historic reasons but is widely used.
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>>> Whatever naming pattern we choose, it should be
> > > consistent.
> > > >> I
> > > >> >> > would
> > > >> >> > >> be
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>> in
> > > >> >> > >> >>>>
> > > >> >> > >> >>>>> favor of changing our artifact names to contain the
> > Hadoop
> > > >> and
> > > >> >> > Scala
> > > >> >> > >> >>>>>> version. This would also imply that all Scala
> dependent
> > > >> Maven
> > > >> >> > >> modules
> > > >> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10
> > > >> modules).
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>>> Cheers,
> > > >> >> > >> >>>>>> Max
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>>>
> > > >> >> > >> >>>>
> > > >> >> > >> >
> > > >> >> > >>
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
>

[mxm]

https://issues.apache.org/jira/browse/FLINK-2940

There is now a pending pull request: https://github.com/apache/flink/pull/1529

As I was working on the changes, I discovered we have some more
modules which have a Scala dependency which could be avoided. Namely
these are "flink-java8", and "flink-streaming-java". The latter
probably affects a lot of users. So it would be nice to make it
Scala-free.

Do you think that would be feasible?

Cheers,
Max

On Mon, Nov 2, 2015 at 9:01 PM, Fabian Hueske <[hidden email]> wrote:

> +1 for that.
>
> 2015-11-02 20:52 GMT+01:00 Stephan Ewen <[hidden email]>:
>
>> +1 for Max' suggestion to fix that for 1.0 (next release).
>>
>> Hot fixing of this thing so short before a release is a bit risky in my
>> opinion. It is easy to make errors (overlooking something, error not
>> visible because of cached older dependencies, ...), it happened more than
>> once during version upgrades, maven project re-organizations, etc.
>>
>> Doing it after 0.10 and having a few week to let it sink in and errors
>> surface would probably be much safer...
>>
>> On Mon, Nov 2, 2015 at 5:19 AM, Fabian Hueske <[hidden email]> wrote:
>>
>> > Ah OK. Sorry, I misunderstood your intention.
>> >
>> > 2015-11-02 14:07 GMT+01:00 Maximilian Michels <[hidden email]>:
>> >
>> > > > That would mean to have "flink-java_2.10" and "flink-java_2.11"
>> > artifacts
>> > > > (and others that depend on flink-java and have no other Scala
>> > dependency)
>> > > > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
>> > >
>> > >
>> > > My suggestion was to keep the Scala unsuffixed Scala 2.10 and add a
>> > > suffix for Scala 2.11. That's the way we are currently doing it (also
>> > > deployed on Maven like this). For the next release after 0.10, we can
>> > > do it properly.
>> > >
>> > > On Mon, Nov 2, 2015 at 1:30 PM, Chiwan Park <[hidden email]>
>> > wrote:
>> > > > If we choose selective Scala version suffix for artifacts, we have to
>> > > tell which artifacts have the version suffix to newcomers. Some
>> artifacts
>> > > such as "flink-java”, "flink-streaming-java" are easily recognized. But
>> > > IMO, knowing whether artifacts such as "flink-ml", "flink-clients",
>> > > "flink-table" have the version suffix or not is difficult for
>> newcomers.
>> > > >
>> > > > This is why we are adding the version suffix to all Scala 2.11
>> > artifacts
>> > > currently. For Scala 2.10 artifacts, we aren’t adding the version
>> suffix
>> > > for Flink with Java users.
>> > > >
>> > > > I’m for adding the version suffix to Scala 2.10 artifacts also. But
>> I’m
>> > > not sure that removing the version suffix from Java-only artifacts
>> would
>> > be
>> > > good. As I said above, It seems difficult for newcomers.
>> > > >
>> > > > Regards,
>> > > > Chiwan Park
>> > > >
>> > > > On November 2, 2015 at 8:19:15 PM, Fabian Hueske ([hidden email])
>> > > wrote:
>> > > >
>> > > > That would mean to have "flink-java_2.10" and "flink-java_2.11"
>> > artifacts
>> > > > (and others that depend on flink-java and have no other Scala
>> > dependency)
>> > > > in the 0.10.0 release and only "flink-java" in the next 1.0 release.
>> > > >
>> > > > Do we want that?
>> > > >
>> > > > 2015-11-02 11:37 GMT+01:00 Maximilian Michels <[hidden email]>:
>> > > >
>> > > >> I'm for leaving it as-is and renaming all artifacts which depend on
>> > > >> Scala for the release following 0.10.
>> > > >>
>> > > >> On Mon, Nov 2, 2015 at 11:32 AM, Fabian Hueske <[hidden email]>
>> > > wrote:
>> > > >> > OK, let me try to summarize the discussion (and please correct me
>> > if I
>> > > >> got
>> > > >> > something wrong).
>> > > >> >
>> > > >> > 1) Flink deploys Scala 2.11 snapshot artifacts. Therefore, we have
>> > to
>> > > >> > release 2.11 artifacts for the 0.10.0 release version as well.
>> > > >> >
>> > > >> > 2) Everybody agrees to appropriately tag all artifacts that have a
>> > > >> > (transitive) Scala dependency. ATM, that would also include
>> > flink-java
>> > > >> > which is a bit awkward. The Scala dependency in flink-java
>> > originates
>> > > >> from
>> > > >> > the Chill library which is used to obtain a Kryo serializer which
>> is
>> > > >> > initialized with serializers for Scala classes. We could resolve
>> > this
>> > > >> issue
>> > > >> > by providing Java and Scala specific implementations of the Kryo
>> > > >> > serializers and have KryoTypeInfos for Java and Scala.
>> > > >> >
>> > > >> > The question to answer right now is, do we want to have
>> "correctly"
>> > > >> labeled
>> > > >> > artifacts for the next 0.10.0 release or do we defer that for 1.0?
>> > > >> > If we want to solve it for 0.10.0 we need to cancel the current RC
>> > and
>> > > >> > provide a fix to remove the Scala dependency in flink-java, IMO.
>> > > >> >
>> > > >> > Opinions?
>> > > >> >
>> > > >> > Cheers, Fabian
>> > > >> >
>> > > >> > 2015-11-02 8:55 GMT+01:00 Stephan Ewen <[hidden email]>:
>> > > >> >
>> > > >> >> +1 for the approach discusses here, and for removing Scala
>> > > dependencies
>> > > >> >> from modules that can be Scala independent.
>> > > >> >>
>> > > >> >> It would be nice if pure Java users would not see any Scala
>> > > versioning
>> > > >> (on
>> > > >> >> flink-core, flink-java, later also flink-sreaming-java). I guess
>> > for
>> > > any
>> > > >> >> runtime-related parts (including flink-client and currently all
>> > > >> streaming
>> > > >> >> projects), we need the Scala versions...
>> > > >> >>
>> > > >> >> On Sun, Nov 1, 2015 at 9:29 AM, Maximilian Michels <
>> [hidden email]
>> > >
>> > > >> wrote:
>> > > >> >>
>> > > >> >> > Good point. Didn't know that. We can still add them for the
>> > > release.
>> > > >> >> >
>> > > >> >> > On Sat, Oct 31, 2015 at 1:51 PM, Alexander Alexandrov
>> > > >> >> > <[hidden email]> wrote:
>> > > >> >> > > My two cents - there are already Maven artifacts deployed for
>> > > 2.11
>> > > >> in
>> > > >> >> the
>> > > >> >> > > SNAPSHOT repository. I think it might be confusing if they
>> > > suddenly
>> > > >> >> > > disappear for the stable release.
>> > > >> >> > >
>> > > >> >> > >
>> > > >> >> > > 2015-10-29 11:58 GMT+01:00 Maximilian Michels <
>> [hidden email]
>> > >:
>> > > >> >> > >
>> > > >> >> > >> Seems like we agree that we need artifacts for different
>> > > versions
>> > > >> of
>> > > >> >> > Scala
>> > > >> >> > >> on Maven. There also seems to be a preference for including
>> > the
>> > > >> >> version
>> > > >> >> > in
>> > > >> >> > >> the artifact name.
>> > > >> >> > >>
>> > > >> >> > >> I've created an issue and marked it to be resolved for 1.0.
>> > For
>> > > the
>> > > >> >> 0.10
>> > > >> >> > >> release, we will have binaries but no Maven artifacts. The
>> > > biggest
>> > > >> >> > >> challenge I see is to remove Scala from as many modules as
>> > > >> possible.
>> > > >> >> For
>> > > >> >> > >> example, flink-java depends on Scala at the moment..
>> > > >> >> > >>
>> > > >> >> > >> https://issues.apache.org/jira/browse/FLINK-2940
>> > > >> >> > >>
>> > > >> >> > >> On Wed, Oct 28, 2015 at 7:31 PM, Frederick F. Kautz IV <
>> > > >> >> > [hidden email]>
>> > > >> >> > >> wrote:
>> > > >> >> > >>
>> > > >> >> > >> > No idea if I get a vote ;) Nevertheless, +1 to have
>> binaries
>> > > for
>> > > >> >> both
>> > > >> >> > >> > versions in Maven and explicitly "scala versioned".
>> > > >> >> > >> >
>> > > >> >> > >> > Some background on this for those not as familiar with
>> scala
>> > > >> >> > versioning:
>> > > >> >> > >> >
>> > > >> >> > >> > It's considered best practice to label what version of
>> > scala a
>> > > >> >> library
>> > > >> >> > >> > uses in the artifact id.
>> > > >> >> > >> >
>> > > >> >> > >> > The reason is compiled scala code is only compatible with
>> > the
>> > > >> major
>> > > >> >> > >> > version of scala it was compiled for. For example, a
>> library
>> > > >> >> > compatible
>> > > >> >> > >> > with 2.10 is not compatible with 2.11. The same will be
>> true
>> > > with
>> > > >> >> 2.12
>> > > >> >> > >> once
>> > > >> >> > >> > it is released. Mixing versions will result in undefined
>> > > behavior
>> > > >> >> > which
>> > > >> >> > >> > will likely manifest itself as runtime exceptions.
>> > > >> >> > >> >
>> > > >> >> > >> > The convention to fix this problem is for all published
>> > > >> libraries to
>> > > >> >> > >> > specify the version of scala they are compatible with.
>> > Leaving
>> > > >> out
>> > > >> >> the
>> > > >> >> > >> > scala version in a library is akin to saying "We don't
>> > depend
>> > > on
>> > > >> >> scala
>> > > >> >> > >> for
>> > > >> >> > >> > this library, so feel free to use whatever you want." Sbt
>> > > users
>> > > >> will
>> > > >> >> > >> > typically specify the version of scala they use and
>> tooling
>> > is
>> > > >> built
>> > > >> >> > >> around
>> > > >> >> > >> > ensuring consistency with the "%%" operator.
>> > > >> >> > >> >
>> > > >> >> > >> > E.g.
>> > > >> >> > >> >
>> > > >> >> > >> > scalaVersion := "2.11.4"
>> > > >> >> > >> >
>> > > >> >> > >> > // this resolves to to artifactID: "scalacheck_2.11"
>> > > >> >> > >> > libraryDependencies += "org.scalacheck" %% "scalacheck" %
>> > > >> "1.12.0" %
>> > > >> >> > >> "test"
>> > > >> >> > >> >
>> > > >> >> > >> > The most important part of this is that the scala version
>> is
>> > > >> >> explicit
>> > > >> >> > >> > which eliminates the problem for downstream users.
>> > > >> >> > >> >
>> > > >> >> > >> > Cheers,
>> > > >> >> > >> > Frederick
>> > > >> >> > >> >
>> > > >> >> > >> >
>> > > >> >> > >> > On 10/28/2015 10:55 AM, Fabian Hueske wrote:
>> > > >> >> > >> >
>> > > >> >> > >> >> +1 to have binaries for both versions in Maven and as
>> build
>> > > to
>> > > >> >> > download.
>> > > >> >> > >> >>
>> > > >> >> > >> >> 2015-10-26 17:11 GMT+01:00 Theodore Vasiloudis <
>> > > >> >> > >> >> [hidden email]>:
>> > > >> >> > >> >>
>> > > >> >> > >> >> +1 for having binaries, I'm working on a Spark
>> application
>> > > >> >> currently
>> > > >> >> > >> with
>> > > >> >> > >> >>> Scala 2.11 and having to rebuild everything when
>> deploying
>> > > >> e.g. to
>> > > >> >> > EC2
>> > > >> >> > >> >>> is a
>> > > >> >> > >> >>> pain.
>> > > >> >> > >> >>>
>> > > >> >> > >> >>> On Mon, Oct 26, 2015 at 4:22 PM, Ufuk Celebi <
>> > > [hidden email]>
>> > > >> >> > wrote:
>> > > >> >> > >> >>>
>> > > >> >> > >> >>> I agree with Till, but is this something you want to
>> > > address in
>> > > >> >> this
>> > > >> >> > >> >>>> release already?
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>> I would postpone it to 1.0.0.
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>> – Ufuk
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>> On 26 Oct 2015, at 16:17, Till Rohrmann <
>> > > [hidden email]
>> > > >> >
>> > > >> >> > wrote:
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>>> I would be in favor of deploying also Scala 2.11
>> > > artifacts to
>> > > >> >> > Maven
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>> since
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> more and more people will try out Flink with Scala
>> 2.11.
>> > > >> Having
>> > > >> >> the
>> > > >> >> > >> >>>>> dependencies in the Maven repository makes it
>> > considerably
>> > > >> >> easier
>> > > >> >> > for
>> > > >> >> > >> >>>>> people to get their Flink jobs running.
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>>> Furthermore, I observed that people are not aware that
>> > our
>> > > >> >> > deployed
>> > > >> >> > >> >>>>> artifacts, e.g. flink-runtime, are built with Scala
>> > 2.10.
>> > > As
>> > > >> a
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>> consequence,
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> they mix flink dependencies with other dependencies
>> > > pulling
>> > > >> in
>> > > >> >> > Scala
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>> 2.11
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> and then they wonder that the program crashes. It would
>> > be,
>> > > >> imho,
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>> clearer
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> if all our dependencies which depend on a specific
>> Scala
>> > > >> version
>> > > >> >> > would
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>> have
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> the corresponding Scala suffix appended.
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>>> Adding the 2.10 suffix would also spare us the hassle
>> of
>> > > >> >> upgrading
>> > > >> >> > >> to a
>> > > >> >> > >> >>>>> newer Scala version in the future, because then the
>> > > artifacts
>> > > >> >> > >> wouldn't
>> > > >> >> > >> >>>>> share the same artifact name.
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>>> Cheers,
>> > > >> >> > >> >>>>> Till
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>>> On Mon, Oct 26, 2015 at 4:04 PM, Maximilian Michels <
>> > > >> >> > [hidden email]>
>> > > >> >> > >> >>>>>
>> > > >> >> > >> >>>> wrote:
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> Hi Flinksters,
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>>> We have recently committed an easy way to change
>> > Flink's
>> > > >> Scala
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> version.
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> The
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> question arises now whether we should ship Scala 2.11
>> as
>> > > >> >> binaries
>> > > >> >> > and
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> via
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> Maven. For the rc0, I created all binaries twice, for
>> > > Scala
>> > > >> 2.10
>> > > >> >> > and
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> 2.11.
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> However, I didn't create Maven artifacts. This follows
>> > our
>> > > >> >> current
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> shipping
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> strategy where we only ship Hadoop1 and Hadoop 2.3.0
>> > Maven
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> dependencies
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> but
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> additionally Hadoop 2.4, 2.6, 2.7 as binaries.
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>>> Should we also upload Maven dependencies for Scala
>> > 2.11?
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>>> If so, the next question arises: What version pattern
>> > > >> should we
>> > > >> >> > have
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> for
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> the Flink Scala 2.11 dependencies? For Hadoop, we
>> append
>> > > >> -hadoop1
>> > > >> >> > to
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> the
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> VERSION, e.g. artifactID=flink-core,
>> > version=0.9.1-hadoop1.
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>>> However, it is common practice to append the suffix
>> to
>> > > the
>> > > >> >> > >> artifactID
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> of
>> > > >> >> > >> >>>
>> > > >> >> > >> >>>> the Maven dependency, e.g. artifactID=flink-core_2.11,
>> > > >> >> > version=0.9.1.
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> This
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> has mostly historic reasons but is widely used.
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>>> Whatever naming pattern we choose, it should be
>> > > consistent.
>> > > >> I
>> > > >> >> > would
>> > > >> >> > >> be
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>> in
>> > > >> >> > >> >>>>
>> > > >> >> > >> >>>>> favor of changing our artifact names to contain the
>> > Hadoop
>> > > >> and
>> > > >> >> > Scala
>> > > >> >> > >> >>>>>> version. This would also imply that all Scala
>> dependent
>> > > >> Maven
>> > > >> >> > >> modules
>> > > >> >> > >> >>>>>> receive a Scala suffix (also the default Scala 2.10
>> > > >> modules).
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>>> Cheers,
>> > > >> >> > >> >>>>>> Max
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>>>
>> > > >> >> > >> >>>>
>> > > >> >> > >> >
>> > > >> >> > >>
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>>

[Ufuk Celebi-2]

> On 21 Jan 2016, at 17:51, Maximilian Michels <[hidden email]> wrote:
>
> https://issues.apache.org/jira/browse/FLINK-2940
>
> There is now a pending pull request: https://github.com/apache/flink/pull/1529
>
> As I was working on the changes, I discovered we have some more
> modules which have a Scala dependency which could be avoided. Namely
> these are "flink-java8", and "flink-streaming-java". The latter
> probably affects a lot of users. So it would be nice to make it
> Scala-free.
>
> Do you think that would be feasible?

Thank you very much for taking care of this. :)

I would like to have it changed as well before merging for the same reason Fabian brought up in the PR.

Since this will be quite a breaking change, we should consider adding a big notice to the 1.0-SNAPSHOT docs regarding this (basically on the landing page) for the few weeks until the release.

– Ufuk