Question on providing CDH packages

[Robert Metzger]

Just for the record, I was able to build a Flink version that is compatible
to CDH4 by using the official Hadoop 2.0.0-alpha release. So with the next
release (0.6.1-incubating or 0.7-incubating) we can ship an additional
"hadoop200alpha" binary package that works for users using a distro which
is based on this version. It won't include any vendor specific versions or
binaries.
See also: https://issues.apache.org/jira/browse/FLINK-1068


On Tue, Aug 19, 2014 at 8:08 PM, Alan Gates <[hidden email]> wrote:

> No objections.  That seems like a good way to help our users while avoiding
> the appearance of favoring one vendor over another.
>
> Alan.
>
>
> On Mon, Aug 18, 2014 at 10:43 AM, Stephan Ewen <[hidden email]> wrote:
>
> > I like Sean's idea very much: Creating the three packages (Hadoop 1.x,
> > Hadoop 2.x, Hadoop 2.0 with Yarn beta).
> >
> > Any objections to creating a help site that says "For that vendor with
> this
> > version pick the following binary release" ?
> >
> > Stephan
> >
> >
> >
> > > >> On Mon, Aug 18, 2014 at 5:58 PM, Henry Saputra <
> > [hidden email]>
> > > wrote:
> > > >>> As for Flink, for now the additional CDH4 packaged binary is to
> > > >>> support "non-standard" Hadoop version that some customers may
> already
> > > >>> have.
> > > >>>
> > > >>> Based on "not a question of supporting a vendor but a Hadoop
> version
> > > >>> combo.", would the approach that Flink had done to help customers
> get
> > > >>> go and running quickly seemed fair and good idea?
> > > >>>
> > > >>> There had been a lot of discussion about ASF release artifacts and
> > the
> > > >>> consistent answer is that ASF validate release of source code and
> not
> > > >>> binaries.
> > > >>> Release of binaries only used to help customers, which is the case
> > > >>> that Flink is doing with different Hadoop versions.
> > > >>>
> > > >>> - Henry
> > > >>>
> > > >>> On Mon, Aug 18, 2014 at 9:51 AM, Sean Owen <[hidden email]>
> wrote:
> > > >>>> It's probably the same thing as with Spark. Spark doesn't actually
> > > >>>> work with YARN 'beta'-era releases, but works 'stable' and
> specially
> > > >>>> supports 'alpha'. CDH 4.{2-4} or so == YARN 'beta' (not
> > non-standard,
> > > >>>> but, is probably the only distro of it you'll still run into in
> > > >>>> circulation). (And so it's kind of unhelpful that Spark has build
> > > >>>> instructions for CDH 4.2 + YARN.) Yeah, that's the thing you may
> > > >>>> handle as a corner case, or not handle and punt to the vendor. But
> > > >>>> even that -- if that's the same issue -- it's not a question of
> > > >>>> supporting a vendor but a Hadoop version combo.
> > > >>>>
> > > >>>> On Mon, Aug 18, 2014 at 5:43 PM, Stephan Ewen <[hidden email]>
> > > wrote:
> > > >>>>> I think the main problem was that CDH4 is a non standard build.
> All
> > > others
> > > >>>>> we tried worked with  hadoop-1.2 and 2.2/2.4 builds.
> > > >>>>>
> > > >>>>> But I understand your points.
> > > >>>>>
> > > >>>>> So, instead of creating those packages, we can make a guide "how
> to
> > > pick
> > > >>>>> the right distribution", which points you to the hadoop-1.2 and
> > > 2.2/2.4
> > > >>>>> builds. For some cases, the guide will ask you to
> > "compile-your-own".
> > > >>>>>
> > >
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>