[SURVEY] Remove Mesos support

[Matthias]

Thanks for everyone's feedback. I'm gonna initiate a vote in a separate
thread.

On Mon, Mar 29, 2021 at 9:18 AM Robert Metzger <[hidden email]> wrote:

> +1
>
>
>
> On Mon, Mar 29, 2021 at 5:44 AM Yangze Guo <[hidden email]> wrote:
>
> > +1
> >
> > Best,
> > Yangze Guo
> >
> > On Mon, Mar 29, 2021 at 11:31 AM Xintong Song <[hidden email]>
> > wrote:
> > >
> > > +1
> > > It's already a matter of fact for a while that we no longer port new
> > features to the Mesos deployment.
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > >
> > > On Fri, Mar 26, 2021 at 10:37 PM Till Rohrmann <[hidden email]>
> > wrote:
> > >>
> > >> +1 for officially deprecating this component for the 1.13 release.
> > >>
> > >> Cheers,
> > >> Till
> > >>
> > >> On Thu, Mar 25, 2021 at 1:49 PM Konstantin Knauf <[hidden email]>
> > wrote:
> > >>>
> > >>> Hi Matthias,
> > >>>
> > >>> Thank you for following up on this. +1 to officially deprecate Mesos
> > in the code and documentation, too. It will be confusing for users if
> this
> > diverges from the roadmap.
> > >>>
> > >>> Cheers,
> > >>>
> > >>> Konstantin
> > >>>
> > >>> On Thu, Mar 25, 2021 at 12:23 PM Matthias Pohl <
> [hidden email]>
> > wrote:
> > >>>>
> > >>>> Hi everyone,
> > >>>> considering the upcoming release of Flink 1.13, I wanted to revive
> the
> > >>>> discussion about the Mesos support ones more. Mesos is also already
> > listed
> > >>>> as deprecated in Flink's overall roadmap [1]. Maybe, it's time to
> > align the
> > >>>> documentation accordingly to make it more explicit?
> > >>>>
> > >>>> What do you think?
> > >>>>
> > >>>> Best,
> > >>>> Matthias
> > >>>>
> > >>>> [1] https://flink.apache.org/roadmap.html#feature-radar
> > >>>>
> > >>>> On Wed, Oct 28, 2020 at 9:40 AM Till Rohrmann <[hidden email]
> >
> > wrote:
> > >>>>
> > >>>> > Hi Oleksandr,
> > >>>> >
> > >>>> > yes you are right. The biggest problem is at the moment the lack
> of
> > test
> > >>>> > coverage and thereby confidence to make changes. We have some e2e
> > tests
> > >>>> > which you can find here [1]. These tests are, however, quite
> coarse
> > grained
> > >>>> > and are missing a lot of cases. One idea would be to add a Mesos
> > e2e test
> > >>>> > based on Flink's end-to-end test framework [2]. I think what needs
> > to be
> > >>>> > done there is to add a Mesos resource and a way to submit jobs to
> a
> > Mesos
> > >>>> > cluster to write e2e tests.
> > >>>> >
> > >>>> > [1] https://github.com/apache/flink/tree/master/flink-jepsen
> > >>>> > [2]
> > >>>> >
> >
> https://github.com/apache/flink/tree/master/flink-end-to-end-tests/flink-end-to-end-tests-common
> > >>>> >
> > >>>> > Cheers,
> > >>>> > Till
> > >>>> >
> > >>>> > On Tue, Oct 27, 2020 at 12:29 PM Oleksandr Nitavskyi <
> > >>>> > [hidden email]> wrote:
> > >>>> >
> > >>>> >> Hello Xintong,
> > >>>> >>
> > >>>> >> Thanks for the insights and support.
> > >>>> >>
> > >>>> >> Browsing the Mesos backlog and didn't identify anything critical,
> > which
> > >>>> >> is left there.
> > >>>> >>
> > >>>> >> I see that there are were quite a lot of contributions to the
> > Flink Mesos
> > >>>> >> in the recent version:
> > >>>> >> https://github.com/apache/flink/commits/master/flink-mesos.
> > >>>> >> We plan to validate the current Flink master (or release 1.12
> > branch) our
> > >>>> >> Mesos setup. In case of any issues, we will try to propose
> changes.
> > >>>> >> My feeling is that our test results shouldn't affect the Flink
> 1.12
> > >>>> >> release cycle. And if any potential commits will land into the
> > 1.12.1 it
> > >>>> >> should be totally fine.
> > >>>> >>
> > >>>> >> In the future, we would be glad to help you guys with any
> > >>>> >> maintenance-related questions. One of the highest priorities
> > around this
> > >>>> >> component seems to be the development of the full e2e test.
> > >>>> >>
> > >>>> >> Kind Regards
> > >>>> >> Oleksandr Nitavskyi
> > >>>> >> ________________________________
> > >>>> >> From: Xintong Song <[hidden email]>
> > >>>> >> Sent: Tuesday, October 27, 2020 7:14 AM
> > >>>> >> To: dev <[hidden email]>; user <[hidden email]>
> > >>>> >> Cc: Piyush Narang <[hidden email]>
> > >>>> >> Subject: [BULK]Re: [SURVEY] Remove Mesos support
> > >>>> >>
> > >>>> >> Hi Piyush,
> > >>>> >>
> > >>>> >> Thanks a lot for sharing the information. It would be a great
> > relief that
> > >>>> >> you are good with Flink on Mesos as is.
> > >>>> >>
> > >>>> >> As for the jira issues, I believe the most essential ones should
> > have
> > >>>> >> already been resolved. You may find some remaining open issues
> > here [1],
> > >>>> >> but not all of them are necessary if we decide to keep Flink on
> > Mesos as is.
> > >>>> >>
> > >>>> >> At the moment and in the short future, I think helps are mostly
> > needed on
> > >>>> >> testing the upcoming release 1.12 with Mesos use cases. The
> > community is
> > >>>> >> currently actively preparing the new release, and hopefully we
> > could come
> > >>>> >> up with a release candidate early next month. It would be greatly
> > >>>> >> appreciated if you fork as experienced Flink on Mesos users can
> > help with
> > >>>> >> verifying the release candidates.
> > >>>> >>
> > >>>> >>
> > >>>> >> Thank you~
> > >>>> >>
> > >>>> >> Xintong Song
> > >>>> >>
> > >>>> >> [1]
> > >>>> >>
> >
> https://issues.apache.org/jira/browse/FLINK-17402?jql=project%20%3D%20FLINK%20AND%20component%20%3D%20%22Deployment%20%2F%20Mesos%22%20AND%20status%20%3D%20Open
> > >>>> >> <
> > >>>> >>
> >
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FFLINK-17402%3Fjql%3Dproject%2520%253D%2520FLINK%2520AND%2520component%2520%253D%2520%2522Deployment%2520%252F%2520Mesos%2522%2520AND%2520status%2520%253D%2520Open&data=04%7C01%7Co.nitavskyi%40criteo.com%7C3585e1f25bdf4e091af808d87a3f92db%7C2a35d8fd574d48e3927c8c398e225a01%7C1%7C0%7C637393760750820881%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hytJFQE0MCPzMLiQTQTdbg3GVckX5M3r1NPRGrRV8j4%3D&reserved=0
> > >>>> >> >
> > >>>> >>
> > >>>> >> On Tue, Oct 27, 2020 at 2:58 AM Piyush Narang <
> [hidden email]
> > >>>> >> <mailto:[hidden email]>> wrote:
> > >>>> >>
> > >>>> >> Hi Xintong,
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> Do you have any jiras that cover any of the items on 1 or 2? I
> can
> > reach
> > >>>> >> out to folks internally and see if I can get some folks to commit
> > to
> > >>>> >> helping out.
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> To cover the other qs:
> > >>>> >>
> > >>>> >>   *   Yes, we’ve not got a plan at the moment to get off Mesos.
> We
> > use
> > >>>> >> Yarn for some our Flink workloads when we can. Mesos is only used
> > when we
> > >>>> >> need streaming capabilities in our WW dcs (as our Yarn is
> > centralized in
> > >>>> >> one DC)
> > >>>> >>   *   We’re currently on Flink 1.9 (old planner). We have a plan
> > to bump
> > >>>> >> to 1.11 / 1.12 this quarter.
> > >>>> >>   *   We typically upgrade once every 6 months to a year (not
> every
> > >>>> >> release). We’d like to speed up the cadence but we’re not there
> > yet.
> > >>>> >>   *   We’d largely be good with keeping Flink on Mesos as-is and
> > >>>> >> functional while missing out on some of the newer features. We
> > understand
> > >>>> >> the pain on the communities side and we can take on the work if
> we
> > see some
> > >>>> >> fancy improvement in Flink on Yarn / K8s that we want in Mesos to
> > put in
> > >>>> >> the request to port it over.
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> Thanks,
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> -- Piyush
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> From: Xintong Song <[hidden email]<mailto:
> > [hidden email]>>
> > >>>> >> Date: Sunday, October 25, 2020 at 10:57 PM
> > >>>> >> To: dev <[hidden email]<mailto:[hidden email]>>,
> user
> > <
> > >>>> >> [hidden email]<mailto:[hidden email]>>
> > >>>> >> Cc: Lasse Nedergaard <[hidden email]<mailto:
> > >>>> >> [hidden email]>>, <[hidden email]<mailto:
> > >>>> >> [hidden email]>>
> > >>>> >> Subject: Re: [SURVEY] Remove Mesos support
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> Thanks for sharing the information with us, Piyush an Lasse.
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> @Piyush
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> Thanks for offering the help. IMO, there are currently several
> > problems
> > >>>> >> that make supporting Flink on Mesos challenging for us.
> > >>>> >>
> > >>>> >>   1.  Lack of Mesos experts. AFAIK, there are very few people (if
> > not
> > >>>> >> none) among the active contributors in this community that are
> > familiar
> > >>>> >> with Mesos and can help with development on this component.
> > >>>> >>   2.  Absence of tests. Mesos does not provide a testing cluster,
> > like
> > >>>> >> `MiniYARNCluster`, making it hard to test interactions between
> > Flink and
> > >>>> >> Mesos. We have only a few very simple e2e tests running on Mesos
> > deployed
> > >>>> >> in a docker, covering the most fundamental workflows. We are not
> > sure how
> > >>>> >> well those tests work, especially against some potential corner
> > cases.
> > >>>> >>   3.  Divergence from other deployment. Because of 1 and 2, the
> new
> > >>>> >> efforts (features, maintenance, refactors) tend to exclude Mesos
> if
> > >>>> >> possible. When the new efforts have to touch the Mesos related
> > components
> > >>>> >> (e.g., changes to the common resource manager interfaces), we
> have
> > to be
> > >>>> >> very careful and make as few changes as possible, to avoid
> > accidentally
> > >>>> >> breaking anything that we are not familiar with. As a result, the
> > component
> > >>>> >> diverges a lot from other deployment components (K8s/Yarn), which
> > makes it
> > >>>> >> harder to maintain.
> > >>>> >>
> > >>>> >> It would be greatly appreciated if you can help with either of
> the
> > above
> > >>>> >> issues.
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> Additionally, I have a few questions concerning your use cases at
> > Criteo.
> > >>>> >> IIUC, you are going to stay on Mesos in the foreseeable future,
> > while
> > >>>> >> keeping the Flink version up-to-date? What Flink version are you
> > currently
> > >>>> >> using? How often do you upgrade (e.g., every release)? Would you
> > be good
> > >>>> >> with keeping the Flink on Mesos component as it is (means that
> > deployment
> > >>>> >> and resource management improvements may not be ported to Mesos),
> > while
> > >>>> >> keeping other components up-to-date (e.g., improvements from
> > programming
> > >>>> >> APIs, operators, state backens, etc.)?
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> Thank you~
> > >>>> >>
> > >>>> >> Xintong Song
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> On Sat, Oct 24, 2020 at 2:48 AM Lasse Nedergaard <
> > >>>> >> [hidden email]<mailto:
> > [hidden email]>>
> > >>>> >> wrote:
> > >>>> >>
> > >>>> >> Hi
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> At Trackunit We have been using Mesos for long time but have now
> > moved to
> > >>>> >> k8s.
> > >>>> >>
> > >>>> >> Med venlig hilsen / Best regards
> > >>>> >>
> > >>>> >> Lasse Nedergaard
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> Den 23. okt. 2020 kl. 17.01 skrev Robert Metzger <
> > [hidden email]
> > >>>> >> <mailto:[hidden email]>>:
> > >>>> >>
> > >>>> >> 
> > >>>> >>
> > >>>> >> Hey Piyush,
> > >>>> >>
> > >>>> >> thanks a lot for raising this concern. I believe we should keep
> > Mesos in
> > >>>> >> Flink then in the foreseeable future.
> > >>>> >>
> > >>>> >> Your offer to help is much appreciated. We'll let you know once
> > there is
> > >>>> >> something.
> > >>>> >>
> > >>>> >>
> > >>>> >>
> > >>>> >> On Fri, Oct 23, 2020 at 4:28 PM Piyush Narang <
> [hidden email]
> > >>>> >> <mailto:[hidden email]>> wrote:
> > >>>> >>
> > >>>> >> Thanks Kostas. If there's items we can help with, I'm sure we'd
> be
> > able
> > >>>> >> to find folks who would be excited to contribute / help in any
> way.
> > >>>> >>
> > >>>> >> -- Piyush
> > >>>> >>
> > >>>> >>
> > >>>> >> On 10/23/20, 10:25 AM, "Kostas Kloudas" <[hidden email]
> > <mailto:
> > >>>> >> [hidden email]>> wrote:
> > >>>> >>
> > >>>> >>     Thanks Piyush for the message.
> > >>>> >>     After this, I revoke my +1. I agree with the previous
> opinions
> > that we
> > >>>> >>     cannot drop code that is actively used by users, especially
> if
> > it
> > >>>> >>     something that deep in the stack as support for cluster
> > management
> > >>>> >>     framework.
> > >>>> >>
> > >>>> >>     Cheers,
> > >>>> >>     Kostas
> > >>>> >>
> > >>>> >>     On Fri, Oct 23, 2020 at 4:15 PM Piyush Narang <
> > [hidden email]
> > >>>> >> <mailto:[hidden email]>> wrote:
> > >>>> >>     >
> > >>>> >>     > Hi folks,
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > We at Criteo are active users of the Flink on Mesos
> resource
> > >>>> >> management component. We are pretty heavy users of Mesos for
> > scheduling
> > >>>> >> workloads on our edge datacenters and we do want to continue to
> be
> > able to
> > >>>> >> run some of our Flink topologies (to compute machine learning
> > short term
> > >>>> >> features) on those DCs. If possible our vote would be not to drop
> > Mesos
> > >>>> >> support as that will tie us to an old release / have to maintain
> a
> > fork as
> > >>>> >> we’re not planning to migrate off Mesos anytime soon. Is the
> burden
> > >>>> >> something that can be helped with by the community? (Or are you
> > referring
> > >>>> >> to having to ensure PRs handle the Mesos piece as well when they
> > touch the
> > >>>> >> resource managers?)
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > Thanks,
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > -- Piyush
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > From: Till Rohrmann <[hidden email]<mailto:
> > >>>> >> [hidden email]>>
> > >>>> >>     > Date: Friday, October 23, 2020 at 8:19 AM
> > >>>> >>     > To: Xintong Song <[hidden email]<mailto:
> > >>>> >> [hidden email]>>
> > >>>> >>     > Cc: dev <[hidden email]<mailto:[hidden email]
> >>,
> > user <
> > >>>> >> [hidden email]<mailto:[hidden email]>>
> > >>>> >>     > Subject: Re: [SURVEY] Remove Mesos support
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > Thanks for starting this survey Robert! I second Konstantin
> > and
> > >>>> >> Xintong in the sense that our Mesos user's opinions should matter
> > most
> > >>>> >> here. If our community is no longer using the Mesos integration,
> > then I
> > >>>> >> would be +1 for removing it in order to decrease the maintenance
> > burden.
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > Cheers,
> > >>>> >>     >
> > >>>> >>     > Till
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > On Fri, Oct 23, 2020 at 2:03 PM Xintong Song <
> > [hidden email]
> > >>>> >> <mailto:[hidden email]>> wrote:
> > >>>> >>     >
> > >>>> >>     > +1 for adding a warning in 1.12 about planning to remove
> > Mesos
> > >>>> >> support.
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > With my developer hat on, removing the Mesos support would
> > >>>> >> definitely reduce the maintaining overhead for the deployment and
> > resource
> > >>>> >> management related components. On the other hand, the Flink on
> > Mesos users'
> > >>>> >> voices definitely matter a lot for this community. Either way, it
> > would be
> > >>>> >> good to draw users attention to this discussion early.
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > Thank you~
> > >>>> >>     >
> > >>>> >>     > Xintong Song
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     >
> > >>>> >>     > On Fri, Oct 23, 2020 at 7:53 PM Konstantin Knauf <
> > [hidden email]
> > >>>> >> <mailto:[hidden email]>> wrote:
> > >>>> >>     >
> > >>>> >>     > Hi Robert,
> > >>>> >>     >
> > >>>> >>     > +1 to the plan you outlined. If we were to drop support in
> > Flink
> > >>>> >> 1.13+, we
> > >>>> >>     > would still support it in Flink 1.12- with bug fixes for
> > some time
> > >>>> >> so that
> > >>>> >>     > users have time to move on.
> > >>>> >>     >
> > >>>> >>     > It would certainly be very interesting to hear from current
> > Flink
> > >>>> >> on Mesos
> > >>>> >>     > users, on how they see the evolution of this part of the
> > ecosystem.
> > >>>> >>     >
> > >>>> >>     > Best,
> > >>>> >>     >
> > >>>> >>     > Konstantin
> > >>>> >>
> > >>>> >
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> Konstantin Knauf
> > >>>
> > >>> https://twitter.com/snntrable
> > >>>
> > >>> https://github.com/knaufk
> >