[DISCUSS] Dropping flink-storm?

[Kostas Kloudas-2]

+1 to drop as well.

On Thu, Jan 10, 2019 at 10:15 AM Ufuk Celebi <[hidden email]> wrote:

> +1 to drop.
>
> I totally agree with your reasoning. I like that we tried to keep it,
> but I don't think the maintenance overhead would be justified.
>
> – Ufuk
>
> On Wed, Jan 9, 2019 at 4:09 PM Till Rohrmann <[hidden email]> wrote:
> >
> > With https://issues.apache.org/jira/browse/FLINK-10571, we will remove
> the
> > Storm topologies from Flink and keep the wrappers for the moment.
> >
> > However, looking at the FlinkTopologyContext [1], it becomes quite
> obvious
> > that Flink's compatibility with Storm is really limited. Almost all of
> the
> > context methods are not supported which makes me wonder how useful these
> > wrappers really are. Given the additional maintenance overhead of having
> > them in the code base and no indication that someone is actively using
> > them, I would still be in favour of removing them. This will reduce our
> > maintenance burden in the future. What do you think?
> >
> > [1]
> >
> https://github.com/apache/flink/blob/master/flink-contrib/flink-storm/src/main/java/org/apache/flink/storm/wrappers/FlinkTopologyContext.java
> >
> > Cheers,
> > Till
> >
> > On Tue, Oct 9, 2018 at 10:08 AM Fabian Hueske <[hidden email]> wrote:
> >
> > > Yes, let's do it this way.
> > > The wrapper classes are probably not too complex and can be easily
> tested.
> > > We have the same for the Hadoop interfaces, although I think only the
> > > Input- and OutputFormatWrappers are actually used.
> > >
> > >
> > > Am Di., 9. Okt. 2018 um 09:46 Uhr schrieb Chesnay Schepler <
> > > [hidden email]>:
> > >
> > >> That sounds very good to me.
> > >>
> > >> On 08.10.2018 11:36, Till Rohrmann wrote:
> > >> > Good point. The initial idea of this thread was to remove the storm
> > >> > compatibility layer completely.
> > >> >
> > >> > During the discussion I realized that it might be useful for our
> users
> > >> > to not completely remove it in one go. Instead for those who still
> > >> > want to use some Bolt and Spout code in Flink, it could be nice to
> > >> > keep the wrappers. At least, we could remove flink-storm in a more
> > >> > graceful way by first removing the Topology and client parts and
> then
> > >> > the wrappers. What do you think?
> > >> >
> > >> > Cheers,
> > >> > Till
> > >> >
> > >> > On Mon, Oct 8, 2018 at 11:13 AM Chesnay Schepler <
> [hidden email]
> > >> > <mailto:[hidden email]>> wrote:
> > >> >
> > >> >     I don't believe that to be the consensus. For starters it is
> > >> >     contradictory; we can't /drop /flink-storm yet still /keep
> //some
> > >> >     parts/.
> > >> >
> > >> >     From my understanding we drop flink-storm completely, and put a
> > >> >     note in the docs that the bolt/spout wrappers of previous
> versions
> > >> >     will continue to work.
> > >> >
> > >> >     On 08.10.2018 11:04, Till Rohrmann wrote:
> > >> >>     Thanks for opening the issue Chesnay. I think the overall
> > >> >>     consensus is to drop flink-storm and only keep the Bolt and
> Spout
> > >> >>     wrappers. Thanks for your feedback!
> > >> >>
> > >> >>     Cheers,
> > >> >>     Till
> > >> >>
> > >> >>     On Mon, Oct 8, 2018 at 9:37 AM Chesnay Schepler
> > >> >>     <[hidden email] <mailto:[hidden email]>> wrote:
> > >> >>
> > >> >>         I've created
> > >> >>         https://issues.apache.org/jira/browse/FLINK-10509 for
> > >> >>         removing flink-storm.
> > >> >>
> > >> >>         On 28.09.2018 15:22, Till Rohrmann wrote:
> > >> >>         > Hi everyone,
> > >> >>         >
> > >> >>         > I would like to discuss how to proceed with Flink's storm
> > >> >>         compatibility
> > >> >>         > layer flink-strom.
> > >> >>         >
> > >> >>         > While working on removing Flink's legacy mode, I noticed
> > >> >>         that some parts of
> > >> >>         > flink-storm rely on the legacy Flink client. In fact, at
> > >> >>         the moment
> > >> >>         > flink-storm does not work together with Flink's new
> > >> distributed
> > >> >>         > architecture.
> > >> >>         >
> > >> >>         > I'm also wondering how many people are actually using
> > >> >>         Flink's Storm
> > >> >>         > compatibility layer and whether it would be worth
> porting it.
> > >> >>         >
> > >> >>         > I see two options how to proceed:
> > >> >>         >
> > >> >>         > 1) Commit to maintain flink-storm and port it to Flink's
> > >> >>         new architecture
> > >> >>         > 2) Drop flink-storm
> > >> >>         >
> > >> >>         > I doubt that we can contribute it to Apache Bahir [1],
> > >> >>         because once we
> > >> >>         > remove the legacy mode, this module will no longer work
> > >> >>         with all newer
> > >> >>         > Flink versions.
> > >> >>         >
> > >> >>         > Therefore, I would like to hear your opinion on this and
> in
> > >> >>         particular if
> > >> >>         > you are using or planning to use flink-storm in the
> future.
> > >> >>         >
> > >> >>         > [1] https://github.com/apache/bahir-flink
> > >> >>         >
> > >> >>         > Cheers,
> > >> >>         > Till
> > >> >>         >
> > >> >>
> > >> >
> > >>
> > >>
>

[Fabian Hueske-2]

+1 from my side as well.

I would assume that most Bolts that are supported by our current wrappers
can be easily converted into respective Flink functions.

Fabian



Am Do., 10. Jan. 2019 um 10:35 Uhr schrieb Kostas Kloudas <
[hidden email]>:

> +1 to drop as well.
>
> On Thu, Jan 10, 2019 at 10:15 AM Ufuk Celebi <[hidden email]> wrote:
>
>> +1 to drop.
>>
>> I totally agree with your reasoning. I like that we tried to keep it,
>> but I don't think the maintenance overhead would be justified.
>>
>> – Ufuk
>>
>> On Wed, Jan 9, 2019 at 4:09 PM Till Rohrmann <[hidden email]>
>> wrote:
>> >
>> > With https://issues.apache.org/jira/browse/FLINK-10571, we will remove
>> the
>> > Storm topologies from Flink and keep the wrappers for the moment.
>> >
>> > However, looking at the FlinkTopologyContext [1], it becomes quite
>> obvious
>> > that Flink's compatibility with Storm is really limited. Almost all of
>> the
>> > context methods are not supported which makes me wonder how useful these
>> > wrappers really are. Given the additional maintenance overhead of having
>> > them in the code base and no indication that someone is actively using
>> > them, I would still be in favour of removing them. This will reduce our
>> > maintenance burden in the future. What do you think?
>> >
>> > [1]
>> >
>> https://github.com/apache/flink/blob/master/flink-contrib/flink-storm/src/main/java/org/apache/flink/storm/wrappers/FlinkTopologyContext.java
>> >
>> > Cheers,
>> > Till
>> >
>> > On Tue, Oct 9, 2018 at 10:08 AM Fabian Hueske <[hidden email]>
>> wrote:
>> >
>> > > Yes, let's do it this way.
>> > > The wrapper classes are probably not too complex and can be easily
>> tested.
>> > > We have the same for the Hadoop interfaces, although I think only the
>> > > Input- and OutputFormatWrappers are actually used.
>> > >
>> > >
>> > > Am Di., 9. Okt. 2018 um 09:46 Uhr schrieb Chesnay Schepler <
>> > > [hidden email]>:
>> > >
>> > >> That sounds very good to me.
>> > >>
>> > >> On 08.10.2018 11:36, Till Rohrmann wrote:
>> > >> > Good point. The initial idea of this thread was to remove the storm
>> > >> > compatibility layer completely.
>> > >> >
>> > >> > During the discussion I realized that it might be useful for our
>> users
>> > >> > to not completely remove it in one go. Instead for those who still
>> > >> > want to use some Bolt and Spout code in Flink, it could be nice to
>> > >> > keep the wrappers. At least, we could remove flink-storm in a more
>> > >> > graceful way by first removing the Topology and client parts and
>> then
>> > >> > the wrappers. What do you think?
>> > >> >
>> > >> > Cheers,
>> > >> > Till
>> > >> >
>> > >> > On Mon, Oct 8, 2018 at 11:13 AM Chesnay Schepler <
>> [hidden email]
>> > >> > <mailto:[hidden email]>> wrote:
>> > >> >
>> > >> >     I don't believe that to be the consensus. For starters it is
>> > >> >     contradictory; we can't /drop /flink-storm yet still /keep
>> //some
>> > >> >     parts/.
>> > >> >
>> > >> >     From my understanding we drop flink-storm completely, and put a
>> > >> >     note in the docs that the bolt/spout wrappers of previous
>> versions
>> > >> >     will continue to work.
>> > >> >
>> > >> >     On 08.10.2018 11:04, Till Rohrmann wrote:
>> > >> >>     Thanks for opening the issue Chesnay. I think the overall
>> > >> >>     consensus is to drop flink-storm and only keep the Bolt and
>> Spout
>> > >> >>     wrappers. Thanks for your feedback!
>> > >> >>
>> > >> >>     Cheers,
>> > >> >>     Till
>> > >> >>
>> > >> >>     On Mon, Oct 8, 2018 at 9:37 AM Chesnay Schepler
>> > >> >>     <[hidden email] <mailto:[hidden email]>> wrote:
>> > >> >>
>> > >> >>         I've created
>> > >> >>         https://issues.apache.org/jira/browse/FLINK-10509 for
>> > >> >>         removing flink-storm.
>> > >> >>
>> > >> >>         On 28.09.2018 15:22, Till Rohrmann wrote:
>> > >> >>         > Hi everyone,
>> > >> >>         >
>> > >> >>         > I would like to discuss how to proceed with Flink's
>> storm
>> > >> >>         compatibility
>> > >> >>         > layer flink-strom.
>> > >> >>         >
>> > >> >>         > While working on removing Flink's legacy mode, I noticed
>> > >> >>         that some parts of
>> > >> >>         > flink-storm rely on the legacy Flink client. In fact, at
>> > >> >>         the moment
>> > >> >>         > flink-storm does not work together with Flink's new
>> > >> distributed
>> > >> >>         > architecture.
>> > >> >>         >
>> > >> >>         > I'm also wondering how many people are actually using
>> > >> >>         Flink's Storm
>> > >> >>         > compatibility layer and whether it would be worth
>> porting it.
>> > >> >>         >
>> > >> >>         > I see two options how to proceed:
>> > >> >>         >
>> > >> >>         > 1) Commit to maintain flink-storm and port it to Flink's
>> > >> >>         new architecture
>> > >> >>         > 2) Drop flink-storm
>> > >> >>         >
>> > >> >>         > I doubt that we can contribute it to Apache Bahir [1],
>> > >> >>         because once we
>> > >> >>         > remove the legacy mode, this module will no longer work
>> > >> >>         with all newer
>> > >> >>         > Flink versions.
>> > >> >>         >
>> > >> >>         > Therefore, I would like to hear your opinion on this
>> and in
>> > >> >>         particular if
>> > >> >>         > you are using or planning to use flink-storm in the
>> future.
>> > >> >>         >
>> > >> >>         > [1] https://github.com/apache/bahir-flink
>> > >> >>         >
>> > >> >>         > Cheers,
>> > >> >>         > Till
>> > >> >>         >
>> > >> >>
>> > >> >
>> > >>
>> > >>
>>
>