[DISCUSS] Dropping Scala 2.10

[Aljoscha Krettek-2]

Hi,

Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.

Are there any concerns about this?

Best,
Aljoscha

[Till Rohrmann]

Given that the last maintenance release Scala 2.10.6 is from about 2 years
ago, I would also be in favour of dropping Scala 2.10 support from Flink.
This will make maintenance easier for us and allow us to drop artifacts
like Flakka.

Cheers,
Till

On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]>
wrote:

> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite
> outdated by now. I would like to drop support for Scala 2.10 and my main
> motivation is that this would allow us to drop our custom Flakka build of
> Akka that we use because newer Akka versions only support Scala 2.11/2.12
> and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

[Márton Balassi]

Hi Aljoscha,

I am in favor of the change. No concerns on my side, just one remark that I
have talked to Sean last week (ccd) and he mentioned that he has faced some
technical issues while driving the transition from 2.10 to 2.12 for Spark.
It had to do with changes in the scope of implicits. You might end up
hitting the same.

Best,
Marton

On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]>
wrote:

> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite
> outdated by now. I would like to drop support for Scala 2.10 and my main
> motivation is that this would allow us to drop our custom Flakka build of
> Akka that we use because newer Akka versions only support Scala 2.11/2.12
> and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

[Sean Owen]

For the curious, here's the overall task in Spark:

https://issues.apache.org/jira/browse/SPARK-14220

and  most of the code-related changes:

https://github.com/apache/spark/pull/18645

and where it's stuck at the moment:

https://mail-archives.apache.org/mod_mbox/spark-dev/201709.mbox/%3CCAMAsSdKe7Os80mX7jYaD2vNWLGWioBgCb4GG55eaN_iotFxZvw%40mail.gmail.com%3E



On Tue, Sep 19, 2017 at 11:07 AM Márton Balassi <[hidden email]>
wrote:

> Hi Aljoscha,
>
> I am in favor of the change. No concerns on my side, just one remark that
> I have talked to Sean last week (ccd) and he mentioned that he has faced
> some technical issues while driving the transition from 2.10 to 2.12 for
> Spark. It had to do with changes in the scope of implicits. You might end
> up hitting the same.
>
> Best,
> Marton
>
> On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]>
> wrote:
>
>> Hi,
>>
>> Talking to some people I get the impression that Scala 2.10 is quite
>> outdated by now. I would like to drop support for Scala 2.10 and my main
>> motivation is that this would allow us to drop our custom Flakka build of
>> Akka that we use because newer Akka versions only support Scala 2.11/2.12
>> and we need a backported feature.
>>
>> Are there any concerns about this?
>>
>> Best,
>> Aljoscha
>
>
>

[Bowen Li]

+1 for dropping support for Scala 2.10

On Tue, Sep 19, 2017 at 3:29 AM, Sean Owen <[hidden email]> wrote:

> For the curious, here's the overall task in Spark:
>
> https://issues.apache.org/jira/browse/SPARK-14220
>
> and  most of the code-related changes:
>
> https://github.com/apache/spark/pull/18645
>
> and where it's stuck at the moment:
>
> <a href="https://mail-archives.apache.org/mod_mbox/spark-dev/201709.mbox/%">https://mail-archives.apache.org/mod_mbox/spark-dev/201709.mbox/%
> 3CCAMAsSdKe7Os80mX7jYaD2vNWLGWioBgCb4GG55eaN_iotFxZvw%40mail.gmail.com%3E
>
>
>
> On Tue, Sep 19, 2017 at 11:07 AM Márton Balassi <[hidden email]>
> wrote:
>
>> Hi Aljoscha,
>>
>> I am in favor of the change. No concerns on my side, just one remark that
>> I have talked to Sean last week (ccd) and he mentioned that he has faced
>> some technical issues while driving the transition from 2.10 to 2.12 for
>> Spark. It had to do with changes in the scope of implicits. You might end
>> up hitting the same.
>>
>> Best,
>> Marton
>>
>> On Tue, Sep 19, 2017 at 11:56 AM, Aljoscha Krettek <[hidden email]>
>> wrote:
>>
>>> Hi,
>>>
>>> Talking to some people I get the impression that Scala 2.10 is quite
>>> outdated by now. I would like to drop support for Scala 2.10 and my main
>>> motivation is that this would allow us to drop our custom Flakka build of
>>> Akka that we use because newer Akka versions only support Scala 2.11/2.12
>>> and we need a backported feature.
>>>
>>> Are there any concerns about this?
>>>
>>> Best,
>>> Aljoscha
>>
>>
>>

[yew1eb]

+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

[Ted Yu]

+1
-------- Original message --------From: Hai Zhou <[hidden email]> Date: 9/20/17  12:44 AM  (GMT-08:00) To: Aljoscha Krettek <[hidden email]>, [hidden email], user <[hidden email]> Subject: Re: [DISCUSS] Dropping Scala 2.10
+1

> 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>
> Hi,
>
> Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
>
> Are there any concerns about this?
>
> Best,
> Aljoscha

[Stefan Richter]

+1

[Kostas Kloudas]

+1

> On Sep 20, 2017, at 2:26 PM, Stefan Richter <[hidden email]> wrote:
>
> +1

[Federico D'Ambrosio]

Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10 as
part of their hadoop distribution.
Could the use of a different scala version cause issues with the Kafka
connector? I'm asking because we are using HDP 2.6 and we once already had
some issue with conflicting scala versions concerning Kafka (though, we
were using Storm, I still haven't tested the Flink connector in this
context).

Regards,
Federico

2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email]>:

> +1
>
> -------- Original message --------
> From: Hai Zhou <[hidden email]>
> Date: 9/20/17 12:44 AM (GMT-08:00)
> To: Aljoscha Krettek <[hidden email]>, [hidden email], user <
> [hidden email]>
> Subject: Re: [DISCUSS] Dropping Scala 2.10
>
> +1
>
> > 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
> >
> > Hi,
> >
> > Talking to some people I get the impression that Scala 2.10 is quite
> outdated by now. I would like to drop support for Scala 2.10 and my main
> motivation is that this would allow us to drop our custom Flakka build of
> Akka that we use because newer Akka versions only support Scala 2.11/2.12
> and we need a backported feature.
> >
> > Are there any concerns about this?
> >
> > Best,
> > Aljoscha
>
>

--
Federico D'Ambrosio

[Driesprong, Fokko]

+1

2017-09-20 14:30 GMT+02:00 Kostas Kloudas <[hidden email]>:

> +1
>
> > On Sep 20, 2017, at 2:26 PM, Stefan Richter <[hidden email]>
> wrote:
> >
> > +1
>
>

[Kostas Kloudas]

+ 1

> On Sep 20, 2017, at 2:26 PM, Stefan Richter <[hidden email]> wrote:
>
> +1

[Aljoscha Krettek-2]

Hi Federico,

As far as I know, the Kafka client code has been rewritten in Java for version 0.9, meaning there is no more Scala dependency in there. Only the server (broker) code still contains Scala but it doesn't matter what Scala version a client uses, if any.

Best,
Aljoscha

> On 20. Sep 2017, at 14:32, Federico D'Ambrosio <[hidden email]> wrote:
>
> Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10 as part of their hadoop distribution.
> Could the use of a different scala version cause issues with the Kafka connector? I'm asking because we are using HDP 2.6 and we once already had some issue with conflicting scala versions concerning Kafka (though, we were using Storm, I still haven't tested the Flink connector in this context).
>
> Regards,
> Federico
>
> 2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email] <mailto:[hidden email]>>:
> +1
>
> -------- Original message --------
> From: Hai Zhou <[hidden email] <mailto:[hidden email]>>
> Date: 9/20/17 12:44 AM (GMT-08:00)
> To: Aljoscha Krettek <[hidden email] <mailto:[hidden email]>>, [hidden email] <mailto:[hidden email]>, user <[hidden email] <mailto:[hidden email]>>
> Subject: Re: [DISCUSS] Dropping Scala 2.10
>
> +1
>
> > 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email] <mailto:[hidden email]>> 写道：
> >
> > Hi,
> >
> > Talking to some people I get the impression that Scala 2.10 is quite outdated by now. I would like to drop support for Scala 2.10 and my main motivation is that this would allow us to drop our custom Flakka build of Akka that we use because newer Akka versions only support Scala 2.11/2.12 and we need a backported feature.
> >
> > Are there any concerns about this?
> >
> > Best,
> > Aljoscha
>
>
>
>
> --
> Federico D'Ambrosio

[Stephan Ewen]

+1

@ Frederico: I think Aljoscha is right - Flink only executes Kafka client
code, which is Scala independent from 0.9 on. Do you use Kafka 0.8 still?

On Wed, Sep 20, 2017 at 10:00 PM, Aljoscha Krettek <[hidden email]>
wrote:

> Hi Federico,
>
> As far as I know, the Kafka client code has been rewritten in Java for
> version 0.9, meaning there is no more Scala dependency in there. Only the
> server (broker) code still contains Scala but it doesn't matter what Scala
> version a client uses, if any.
>
> Best,
> Aljoscha
>
> On 20. Sep 2017, at 14:32, Federico D'Ambrosio <[hidden email]> wrote:
>
> Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10 as
> part of their hadoop distribution.
> Could the use of a different scala version cause issues with the Kafka
> connector? I'm asking because we are using HDP 2.6 and we once already had
> some issue with conflicting scala versions concerning Kafka (though, we
> were using Storm, I still haven't tested the Flink connector in this
> context).
>
> Regards,
> Federico
>
> 2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email]>:
>
>> +1
>>
>> -------- Original message --------
>> From: Hai Zhou <[hidden email]>
>> Date: 9/20/17 12:44 AM (GMT-08:00)
>> To: Aljoscha Krettek <[hidden email]>, [hidden email], user <
>> [hidden email]>
>> Subject: Re: [DISCUSS] Dropping Scala 2.10
>>
>> +1
>>
>> > 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>> >
>> > Hi,
>> >
>> > Talking to some people I get the impression that Scala 2.10 is quite
>> outdated by now. I would like to drop support for Scala 2.10 and my main
>> motivation is that this would allow us to drop our custom Flakka build of
>> Akka that we use because newer Akka versions only support Scala 2.11/2.12
>> and we need a backported feature.
>> >
>> > Are there any concerns about this?
>> >
>> > Best,
>> > Aljoscha
>>
>>
>
>
> --
> Federico D'Ambrosio
>
>
>

[Federico D'Ambrosio]

Ok, thank you all for the clarification.

@Stephan: I'm using Kafka 0.10, I guess the problem I had then was actually
unrelated to specific Kafka version

Federico D'Ambrosio

Il 21 set 2017 16:30, "Stephan Ewen" <[hidden email]> ha scritto:

> +1
>
> @ Frederico: I think Aljoscha is right - Flink only executes Kafka client
> code, which is Scala independent from 0.9 on. Do you use Kafka 0.8 still?
>
> On Wed, Sep 20, 2017 at 10:00 PM, Aljoscha Krettek <[hidden email]>
> wrote:
>
>> Hi Federico,
>>
>> As far as I know, the Kafka client code has been rewritten in Java for
>> version 0.9, meaning there is no more Scala dependency in there. Only the
>> server (broker) code still contains Scala but it doesn't matter what Scala
>> version a client uses, if any.
>>
>> Best,
>> Aljoscha
>>
>> On 20. Sep 2017, at 14:32, Federico D'Ambrosio <[hidden email]>
>> wrote:
>>
>> Hi, as far as I know some vendors like Hortonworks still use Kafka_2.10
>> as part of their hadoop distribution.
>> Could the use of a different scala version cause issues with the Kafka
>> connector? I'm asking because we are using HDP 2.6 and we once already had
>> some issue with conflicting scala versions concerning Kafka (though, we
>> were using Storm, I still haven't tested the Flink connector in this
>> context).
>>
>> Regards,
>> Federico
>>
>> 2017-09-20 14:19 GMT+02:00 Ted Yu <[hidden email]>:
>>
>>> +1
>>>
>>> -------- Original message --------
>>> From: Hai Zhou <[hidden email]>
>>> Date: 9/20/17 12:44 AM (GMT-08:00)
>>> To: Aljoscha Krettek <[hidden email]>, [hidden email], user <
>>> [hidden email]>
>>> Subject: Re: [DISCUSS] Dropping Scala 2.10
>>>
>>> +1
>>>
>>> > 在 2017年9月19日，17:56，Aljoscha Krettek <[hidden email]> 写道：
>>> >
>>> > Hi,
>>> >
>>> > Talking to some people I get the impression that Scala 2.10 is quite
>>> outdated by now. I would like to drop support for Scala 2.10 and my main
>>> motivation is that this would allow us to drop our custom Flakka build of
>>> Akka that we use because newer Akka versions only support Scala 2.11/2.12
>>> and we need a backported feature.
>>> >
>>> > Are there any concerns about this?
>>> >
>>> > Best,
>>> > Aljoscha
>>>
>>>
>>
>>
>> --
>> Federico D'Ambrosio
>>
>>
>>
>