Restructuring the maven projects

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Restructuring the maven projects

Stephan Ewen
Hi everyone!

I think that by now, quite a bit of the maven project structure can be
improved to get rid of some legacy artifacts. Especially the "flink-addons"
project seems to be a catch-all place for various projects.

Here is a suggestion what we could do:

1) Move "flink-yarn" to the root.

2) Move "flink-streaming" to the root (this is planned anyways for the next
release)

3) Create a project "flink-connectors", which will contain "avro", "jdbc",
and "hbase". Should we have them as separate sub-projects, or as one
project?

4) Consolidate the examples into a single project "flink-examples", where
Java, Scala, Streaming examples exist in different packages.


Greetings,
Stephan
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Aljoscha Krettek-2
And also combine flink-java and flink-scala?

P. S. Happy new year, everyone. 😀
On Jan 2, 2015 3:53 PM, "Stephan Ewen" <[hidden email]> wrote:

> Hi everyone!
>
> I think that by now, quite a bit of the maven project structure can be
> improved to get rid of some legacy artifacts. Especially the "flink-addons"
> project seems to be a catch-all place for various projects.
>
> Here is a suggestion what we could do:
>
> 1) Move "flink-yarn" to the root.
>
> 2) Move "flink-streaming" to the root (this is planned anyways for the next
> release)
>
> 3) Create a project "flink-connectors", which will contain "avro", "jdbc",
> and "hbase". Should we have them as separate sub-projects, or as one
> project?
>
> 4) Consolidate the examples into a single project "flink-examples", where
> Java, Scala, Streaming examples exist in different packages.
>
>
> Greetings,
> Stephan
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Kostas Tzoumas-2
+1

On Sat, Jan 3, 2015 at 6:46 PM, Aljoscha Krettek <[hidden email]>
wrote:

> And also combine flink-java and flink-scala?
>
> P. S. Happy new year, everyone. 😀
> On Jan 2, 2015 3:53 PM, "Stephan Ewen" <[hidden email]> wrote:
>
> > Hi everyone!
> >
> > I think that by now, quite a bit of the maven project structure can be
> > improved to get rid of some legacy artifacts. Especially the
> "flink-addons"
> > project seems to be a catch-all place for various projects.
> >
> > Here is a suggestion what we could do:
> >
> > 1) Move "flink-yarn" to the root.
> >
> > 2) Move "flink-streaming" to the root (this is planned anyways for the
> next
> > release)
> >
> > 3) Create a project "flink-connectors", which will contain "avro",
> "jdbc",
> > and "hbase". Should we have them as separate sub-projects, or as one
> > project?
> >
> > 4) Consolidate the examples into a single project "flink-examples", where
> > Java, Scala, Streaming examples exist in different packages.
> >
> >
> > Greetings,
> > Stephan
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Henry Saputra
In reply to this post by Stephan Ewen
Will this conflict with Marton's restructuring proposal which happens
in another thread (see "Project restructure" thread in the dev@ list).

Since we are doing refactoring, may I suggest that we also remove
"flink-" prefix since maven group name will indicate it is part of
Flink.

- Henry

On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]> wrote:

> Hi everyone!
>
> I think that by now, quite a bit of the maven project structure can be
> improved to get rid of some legacy artifacts. Especially the "flink-addons"
> project seems to be a catch-all place for various projects.
>
> Here is a suggestion what we could do:
>
> 1) Move "flink-yarn" to the root.
>
> 2) Move "flink-streaming" to the root (this is planned anyways for the next
> release)
>
> 3) Create a project "flink-connectors", which will contain "avro", "jdbc",
> and "hbase". Should we have them as separate sub-projects, or as one
> project?
>
> 4) Consolidate the examples into a single project "flink-examples", where
> Java, Scala, Streaming examples exist in different packages.
>
>
> Greetings,
> Stephan
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Robert Metzger
Hi,

I'm fine with consolidating some of the sub-modules. I'm currently
preparing a pull request for YARN which will move flink-yarn out of the
"flink-addons".

Regarding the "flink-" prefix: I'm voting to keep the prefix because the
resulting jars will contain "flink" in their name. Its much easier to
properly identify a file named "flink-core-0.8.jar" instead of
"core-0.8.jar".  Also, the release / nightly builds scripts would need
changes if we modify the module names.


On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <[hidden email]>
wrote:

> Will this conflict with Marton's restructuring proposal which happens
> in another thread (see "Project restructure" thread in the dev@ list).
>
> Since we are doing refactoring, may I suggest that we also remove
> "flink-" prefix since maven group name will indicate it is part of
> Flink.
>
> - Henry
>
> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]> wrote:
> > Hi everyone!
> >
> > I think that by now, quite a bit of the maven project structure can be
> > improved to get rid of some legacy artifacts. Especially the
> "flink-addons"
> > project seems to be a catch-all place for various projects.
> >
> > Here is a suggestion what we could do:
> >
> > 1) Move "flink-yarn" to the root.
> >
> > 2) Move "flink-streaming" to the root (this is planned anyways for the
> next
> > release)
> >
> > 3) Create a project "flink-connectors", which will contain "avro",
> "jdbc",
> > and "hbase". Should we have them as separate sub-projects, or as one
> > project?
> >
> > 4) Consolidate the examples into a single project "flink-examples", where
> > Java, Scala, Streaming examples exist in different packages.
> >
> >
> > Greetings,
> > Stephan
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Stephan Ewen
In reply to this post by Henry Saputra
I think this works well together with Marton's restructuring.

I would vote to keep the "flink-" prefix, because it guarantees that the
produced jars are prefixed with "flink-". Otherwise, we will have to start
configuring a lot...

Greetings,
Stephan


On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <[hidden email]>
wrote:

> Will this conflict with Marton's restructuring proposal which happens
> in another thread (see "Project restructure" thread in the dev@ list).
>
> Since we are doing refactoring, may I suggest that we also remove
> "flink-" prefix since maven group name will indicate it is part of
> Flink.
>
> - Henry
>
> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]> wrote:
> > Hi everyone!
> >
> > I think that by now, quite a bit of the maven project structure can be
> > improved to get rid of some legacy artifacts. Especially the
> "flink-addons"
> > project seems to be a catch-all place for various projects.
> >
> > Here is a suggestion what we could do:
> >
> > 1) Move "flink-yarn" to the root.
> >
> > 2) Move "flink-streaming" to the root (this is planned anyways for the
> next
> > release)
> >
> > 3) Create a project "flink-connectors", which will contain "avro",
> "jdbc",
> > and "hbase". Should we have them as separate sub-projects, or as one
> > project?
> >
> > 4) Consolidate the examples into a single project "flink-examples", where
> > Java, Scala, Streaming examples exist in different packages.
> >
> >
> > Greetings,
> > Stephan
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Henry Saputra
@Robert, and @Stephan, sure I am ok with it, thanks for the responses.

- Henry

On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]> wrote:

> I think this works well together with Marton's restructuring.
>
> I would vote to keep the "flink-" prefix, because it guarantees that the
> produced jars are prefixed with "flink-". Otherwise, we will have to start
> configuring a lot...
>
> Greetings,
> Stephan
>
>
> On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <[hidden email]>
> wrote:
>
>> Will this conflict with Marton's restructuring proposal which happens
>> in another thread (see "Project restructure" thread in the dev@ list).
>>
>> Since we are doing refactoring, may I suggest that we also remove
>> "flink-" prefix since maven group name will indicate it is part of
>> Flink.
>>
>> - Henry
>>
>> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]> wrote:
>> > Hi everyone!
>> >
>> > I think that by now, quite a bit of the maven project structure can be
>> > improved to get rid of some legacy artifacts. Especially the
>> "flink-addons"
>> > project seems to be a catch-all place for various projects.
>> >
>> > Here is a suggestion what we could do:
>> >
>> > 1) Move "flink-yarn" to the root.
>> >
>> > 2) Move "flink-streaming" to the root (this is planned anyways for the
>> next
>> > release)
>> >
>> > 3) Create a project "flink-connectors", which will contain "avro",
>> "jdbc",
>> > and "hbase". Should we have them as separate sub-projects, or as one
>> > project?
>> >
>> > 4) Consolidate the examples into a single project "flink-examples", where
>> > Java, Scala, Streaming examples exist in different packages.
>> >
>> >
>> > Greetings,
>> > Stephan
>>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Márton Balassi-3
Let us consider this thread the standard for the restructure, it is
perfectly in line with the wishes I have posted.

+1 for keeping the 'flink-' prefix.

On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <[hidden email]>
wrote:

> @Robert, and @Stephan, sure I am ok with it, thanks for the responses.
>
> - Henry
>
> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]> wrote:
> > I think this works well together with Marton's restructuring.
> >
> > I would vote to keep the "flink-" prefix, because it guarantees that the
> > produced jars are prefixed with "flink-". Otherwise, we will have to
> start
> > configuring a lot...
> >
> > Greetings,
> > Stephan
> >
> >
> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <[hidden email]>
> > wrote:
> >
> >> Will this conflict with Marton's restructuring proposal which happens
> >> in another thread (see "Project restructure" thread in the dev@ list).
> >>
> >> Since we are doing refactoring, may I suggest that we also remove
> >> "flink-" prefix since maven group name will indicate it is part of
> >> Flink.
> >>
> >> - Henry
> >>
> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]> wrote:
> >> > Hi everyone!
> >> >
> >> > I think that by now, quite a bit of the maven project structure can be
> >> > improved to get rid of some legacy artifacts. Especially the
> >> "flink-addons"
> >> > project seems to be a catch-all place for various projects.
> >> >
> >> > Here is a suggestion what we could do:
> >> >
> >> > 1) Move "flink-yarn" to the root.
> >> >
> >> > 2) Move "flink-streaming" to the root (this is planned anyways for the
> >> next
> >> > release)
> >> >
> >> > 3) Create a project "flink-connectors", which will contain "avro",
> >> "jdbc",
> >> > and "hbase". Should we have them as separate sub-projects, or as one
> >> > project?
> >> >
> >> > 4) Consolidate the examples into a single project "flink-examples",
> where
> >> > Java, Scala, Streaming examples exist in different packages.
> >> >
> >> >
> >> > Greetings,
> >> > Stephan
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Henry Saputra
Thanks Marton, having 2 threads discussing same thing can be confusing.

- Henry

On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <[hidden email]> wrote:

> Let us consider this thread the standard for the restructure, it is
> perfectly in line with the wishes I have posted.
>
> +1 for keeping the 'flink-' prefix.
>
> On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <[hidden email]>
> wrote:
>
>> @Robert, and @Stephan, sure I am ok with it, thanks for the responses.
>>
>> - Henry
>>
>> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]> wrote:
>> > I think this works well together with Marton's restructuring.
>> >
>> > I would vote to keep the "flink-" prefix, because it guarantees that the
>> > produced jars are prefixed with "flink-". Otherwise, we will have to
>> start
>> > configuring a lot...
>> >
>> > Greetings,
>> > Stephan
>> >
>> >
>> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <[hidden email]>
>> > wrote:
>> >
>> >> Will this conflict with Marton's restructuring proposal which happens
>> >> in another thread (see "Project restructure" thread in the dev@ list).
>> >>
>> >> Since we are doing refactoring, may I suggest that we also remove
>> >> "flink-" prefix since maven group name will indicate it is part of
>> >> Flink.
>> >>
>> >> - Henry
>> >>
>> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]> wrote:
>> >> > Hi everyone!
>> >> >
>> >> > I think that by now, quite a bit of the maven project structure can be
>> >> > improved to get rid of some legacy artifacts. Especially the
>> >> "flink-addons"
>> >> > project seems to be a catch-all place for various projects.
>> >> >
>> >> > Here is a suggestion what we could do:
>> >> >
>> >> > 1) Move "flink-yarn" to the root.
>> >> >
>> >> > 2) Move "flink-streaming" to the root (this is planned anyways for the
>> >> next
>> >> > release)
>> >> >
>> >> > 3) Create a project "flink-connectors", which will contain "avro",
>> >> "jdbc",
>> >> > and "hbase". Should we have them as separate sub-projects, or as one
>> >> > project?
>> >> >
>> >> > 4) Consolidate the examples into a single project "flink-examples",
>> where
>> >> > Java, Scala, Streaming examples exist in different packages.
>> >> >
>> >> >
>> >> > Greetings,
>> >> > Stephan
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Stephan Ewen
To not let this discussion die, here is a concrete JIRA and a proposed
layout to restructure to.

What remains to be discusses is whether we want to keep the Scala/Java APIs
for batch/streaming in separate projects or in one project.

Also, we need to find a good time to do this, when we are low on pull
requests...

https://issues.apache.org/jira/browse/FLINK-1712

Proposed Layout:

 - flink-hadoop (shaded fat jar)

 - Core (Core and Java and Scala)
 - Streaming (core + java + scala)
 - Runtime
 - Client (Client + Optimizer)

 - Examples (Java + Scala + Streaming Java + Streaming Scala)
 - Tests (test-utils (compile) and tests (test))

 - Quickstarts
   - Quickstart Java
   - Quickstart Scala

 - connectors / Input/Output Formats
   - Avro
   - HBase
   - HadoopCompartibility
   - HCatalogue
   - JDBC
   - kafka
   - rabbit
   - ...

 - staging
   - Gelly
   - Gilbert (ML)
   - spargel (deprecated)
   - expression API

 - contrib

 - yarn

 - dist

 - yarn tests

 - java 8

On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <[hidden email]>
wrote:

> Thanks Marton, having 2 threads discussing same thing can be confusing.
>
> - Henry
>
> On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <[hidden email]>
> wrote:
> > Let us consider this thread the standard for the restructure, it is
> > perfectly in line with the wishes I have posted.
> >
> > +1 for keeping the 'flink-' prefix.
> >
> > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <[hidden email]>
> > wrote:
> >
> >> @Robert, and @Stephan, sure I am ok with it, thanks for the responses.
> >>
> >> - Henry
> >>
> >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]> wrote:
> >> > I think this works well together with Marton's restructuring.
> >> >
> >> > I would vote to keep the "flink-" prefix, because it guarantees that
> the
> >> > produced jars are prefixed with "flink-". Otherwise, we will have to
> >> start
> >> > configuring a lot...
> >> >
> >> > Greetings,
> >> > Stephan
> >> >
> >> >
> >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> [hidden email]>
> >> > wrote:
> >> >
> >> >> Will this conflict with Marton's restructuring proposal which happens
> >> >> in another thread (see "Project restructure" thread in the dev@
> list).
> >> >>
> >> >> Since we are doing refactoring, may I suggest that we also remove
> >> >> "flink-" prefix since maven group name will indicate it is part of
> >> >> Flink.
> >> >>
> >> >> - Henry
> >> >>
> >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]>
> wrote:
> >> >> > Hi everyone!
> >> >> >
> >> >> > I think that by now, quite a bit of the maven project structure
> can be
> >> >> > improved to get rid of some legacy artifacts. Especially the
> >> >> "flink-addons"
> >> >> > project seems to be a catch-all place for various projects.
> >> >> >
> >> >> > Here is a suggestion what we could do:
> >> >> >
> >> >> > 1) Move "flink-yarn" to the root.
> >> >> >
> >> >> > 2) Move "flink-streaming" to the root (this is planned anyways for
> the
> >> >> next
> >> >> > release)
> >> >> >
> >> >> > 3) Create a project "flink-connectors", which will contain "avro",
> >> >> "jdbc",
> >> >> > and "hbase". Should we have them as separate sub-projects, or as
> one
> >> >> > project?
> >> >> >
> >> >> > 4) Consolidate the examples into a single project "flink-examples",
> >> where
> >> >> > Java, Scala, Streaming examples exist in different packages.
> >> >> >
> >> >> >
> >> >> > Greetings,
> >> >> > Stephan
> >> >>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Márton Balassi
+1 for the proposed structure.

I have no explicit preference for having batch and streaming scala together
or separated. That said streaming scala is considerably thin, it does not
really require an own maven submodule.

Marked an older JIRA for the same issue as duplicate. [1]

[1] https://issues.apache.org/jira/browse/FLINK-1340

On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <[hidden email]> wrote:

> To not let this discussion die, here is a concrete JIRA and a proposed
> layout to restructure to.
>
> What remains to be discusses is whether we want to keep the Scala/Java APIs
> for batch/streaming in separate projects or in one project.
>
> Also, we need to find a good time to do this, when we are low on pull
> requests...
>
> https://issues.apache.org/jira/browse/FLINK-1712
>
> Proposed Layout:
>
>  - flink-hadoop (shaded fat jar)
>
>  - Core (Core and Java and Scala)
>  - Streaming (core + java + scala)
>  - Runtime
>  - Client (Client + Optimizer)
>
>  - Examples (Java + Scala + Streaming Java + Streaming Scala)
>  - Tests (test-utils (compile) and tests (test))
>
>  - Quickstarts
>    - Quickstart Java
>    - Quickstart Scala
>
>  - connectors / Input/Output Formats
>    - Avro
>    - HBase
>    - HadoopCompartibility
>    - HCatalogue
>    - JDBC
>    - kafka
>    - rabbit
>    - ...
>
>  - staging
>    - Gelly
>    - Gilbert (ML)
>    - spargel (deprecated)
>    - expression API
>
>  - contrib
>
>  - yarn
>
>  - dist
>
>  - yarn tests
>
>  - java 8
>
> On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <[hidden email]>
> wrote:
>
> > Thanks Marton, having 2 threads discussing same thing can be confusing.
> >
> > - Henry
> >
> > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <[hidden email]>
> > wrote:
> > > Let us consider this thread the standard for the restructure, it is
> > > perfectly in line with the wishes I have posted.
> > >
> > > +1 for keeping the 'flink-' prefix.
> > >
> > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> [hidden email]>
> > > wrote:
> > >
> > >> @Robert, and @Stephan, sure I am ok with it, thanks for the responses.
> > >>
> > >> - Henry
> > >>
> > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]>
> wrote:
> > >> > I think this works well together with Marton's restructuring.
> > >> >
> > >> > I would vote to keep the "flink-" prefix, because it guarantees that
> > the
> > >> > produced jars are prefixed with "flink-". Otherwise, we will have to
> > >> start
> > >> > configuring a lot...
> > >> >
> > >> > Greetings,
> > >> > Stephan
> > >> >
> > >> >
> > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > [hidden email]>
> > >> > wrote:
> > >> >
> > >> >> Will this conflict with Marton's restructuring proposal which
> happens
> > >> >> in another thread (see "Project restructure" thread in the dev@
> > list).
> > >> >>
> > >> >> Since we are doing refactoring, may I suggest that we also remove
> > >> >> "flink-" prefix since maven group name will indicate it is part of
> > >> >> Flink.
> > >> >>
> > >> >> - Henry
> > >> >>
> > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]>
> > wrote:
> > >> >> > Hi everyone!
> > >> >> >
> > >> >> > I think that by now, quite a bit of the maven project structure
> > can be
> > >> >> > improved to get rid of some legacy artifacts. Especially the
> > >> >> "flink-addons"
> > >> >> > project seems to be a catch-all place for various projects.
> > >> >> >
> > >> >> > Here is a suggestion what we could do:
> > >> >> >
> > >> >> > 1) Move "flink-yarn" to the root.
> > >> >> >
> > >> >> > 2) Move "flink-streaming" to the root (this is planned anyways
> for
> > the
> > >> >> next
> > >> >> > release)
> > >> >> >
> > >> >> > 3) Create a project "flink-connectors", which will contain
> "avro",
> > >> >> "jdbc",
> > >> >> > and "hbase". Should we have them as separate sub-projects, or as
> > one
> > >> >> > project?
> > >> >> >
> > >> >> > 4) Consolidate the examples into a single project
> "flink-examples",
> > >> where
> > >> >> > Java, Scala, Streaming examples exist in different packages.
> > >> >> >
> > >> >> >
> > >> >> > Greetings,
> > >> >> > Stephan
> > >> >>
> > >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Ufuk Celebi-2
+1 I like the proposed structure.

The only thing I was wondering about is whether to name "core" => "batch"?

On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi <[hidden email]>
wrote:

> +1 for the proposed structure.
>
> I have no explicit preference for having batch and streaming scala together
> or separated. That said streaming scala is considerably thin, it does not
> really require an own maven submodule.
>
> Marked an older JIRA for the same issue as duplicate. [1]
>
> [1] https://issues.apache.org/jira/browse/FLINK-1340
>
> On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <[hidden email]> wrote:
>
> > To not let this discussion die, here is a concrete JIRA and a proposed
> > layout to restructure to.
> >
> > What remains to be discusses is whether we want to keep the Scala/Java
> APIs
> > for batch/streaming in separate projects or in one project.
> >
> > Also, we need to find a good time to do this, when we are low on pull
> > requests...
> >
> > https://issues.apache.org/jira/browse/FLINK-1712
> >
> > Proposed Layout:
> >
> >  - flink-hadoop (shaded fat jar)
> >
> >  - Core (Core and Java and Scala)
> >  - Streaming (core + java + scala)
> >  - Runtime
> >  - Client (Client + Optimizer)
> >
> >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> >  - Tests (test-utils (compile) and tests (test))
> >
> >  - Quickstarts
> >    - Quickstart Java
> >    - Quickstart Scala
> >
> >  - connectors / Input/Output Formats
> >    - Avro
> >    - HBase
> >    - HadoopCompartibility
> >    - HCatalogue
> >    - JDBC
> >    - kafka
> >    - rabbit
> >    - ...
> >
> >  - staging
> >    - Gelly
> >    - Gilbert (ML)
> >    - spargel (deprecated)
> >    - expression API
> >
> >  - contrib
> >
> >  - yarn
> >
> >  - dist
> >
> >  - yarn tests
> >
> >  - java 8
> >
> > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <[hidden email]>
> > wrote:
> >
> > > Thanks Marton, having 2 threads discussing same thing can be confusing.
> > >
> > > - Henry
> > >
> > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <[hidden email]>
> > > wrote:
> > > > Let us consider this thread the standard for the restructure, it is
> > > > perfectly in line with the wishes I have posted.
> > > >
> > > > +1 for keeping the 'flink-' prefix.
> > > >
> > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> > [hidden email]>
> > > > wrote:
> > > >
> > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the
> responses.
> > > >>
> > > >> - Henry
> > > >>
> > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]>
> > wrote:
> > > >> > I think this works well together with Marton's restructuring.
> > > >> >
> > > >> > I would vote to keep the "flink-" prefix, because it guarantees
> that
> > > the
> > > >> > produced jars are prefixed with "flink-". Otherwise, we will have
> to
> > > >> start
> > > >> > configuring a lot...
> > > >> >
> > > >> > Greetings,
> > > >> > Stephan
> > > >> >
> > > >> >
> > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > > [hidden email]>
> > > >> > wrote:
> > > >> >
> > > >> >> Will this conflict with Marton's restructuring proposal which
> > happens
> > > >> >> in another thread (see "Project restructure" thread in the dev@
> > > list).
> > > >> >>
> > > >> >> Since we are doing refactoring, may I suggest that we also remove
> > > >> >> "flink-" prefix since maven group name will indicate it is part
> of
> > > >> >> Flink.
> > > >> >>
> > > >> >> - Henry
> > > >> >>
> > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]>
> > > wrote:
> > > >> >> > Hi everyone!
> > > >> >> >
> > > >> >> > I think that by now, quite a bit of the maven project structure
> > > can be
> > > >> >> > improved to get rid of some legacy artifacts. Especially the
> > > >> >> "flink-addons"
> > > >> >> > project seems to be a catch-all place for various projects.
> > > >> >> >
> > > >> >> > Here is a suggestion what we could do:
> > > >> >> >
> > > >> >> > 1) Move "flink-yarn" to the root.
> > > >> >> >
> > > >> >> > 2) Move "flink-streaming" to the root (this is planned anyways
> > for
> > > the
> > > >> >> next
> > > >> >> > release)
> > > >> >> >
> > > >> >> > 3) Create a project "flink-connectors", which will contain
> > "avro",
> > > >> >> "jdbc",
> > > >> >> > and "hbase". Should we have them as separate sub-projects, or
> as
> > > one
> > > >> >> > project?
> > > >> >> >
> > > >> >> > 4) Consolidate the examples into a single project
> > "flink-examples",
> > > >> where
> > > >> >> > Java, Scala, Streaming examples exist in different packages.
> > > >> >> >
> > > >> >> >
> > > >> >> > Greetings,
> > > >> >> > Stephan
> > > >> >>
> > > >>
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Till Rohrmann
Putting the Scala and Java API into the same module means that we'll have
more mixed Java/Scala projects, right? I just want to check if everyone is
aware of it considering our latest experiences with these kind of modules.

On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi <[hidden email]> wrote:

> +1 I like the proposed structure.
>
> The only thing I was wondering about is whether to name "core" => "batch"?
>
> On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi <[hidden email]
> >
> wrote:
>
> > +1 for the proposed structure.
> >
> > I have no explicit preference for having batch and streaming scala
> together
> > or separated. That said streaming scala is considerably thin, it does not
> > really require an own maven submodule.
> >
> > Marked an older JIRA for the same issue as duplicate. [1]
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-1340
> >
> > On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <[hidden email]> wrote:
> >
> > > To not let this discussion die, here is a concrete JIRA and a proposed
> > > layout to restructure to.
> > >
> > > What remains to be discusses is whether we want to keep the Scala/Java
> > APIs
> > > for batch/streaming in separate projects or in one project.
> > >
> > > Also, we need to find a good time to do this, when we are low on pull
> > > requests...
> > >
> > > https://issues.apache.org/jira/browse/FLINK-1712
> > >
> > > Proposed Layout:
> > >
> > >  - flink-hadoop (shaded fat jar)
> > >
> > >  - Core (Core and Java and Scala)
> > >  - Streaming (core + java + scala)
> > >  - Runtime
> > >  - Client (Client + Optimizer)
> > >
> > >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> > >  - Tests (test-utils (compile) and tests (test))
> > >
> > >  - Quickstarts
> > >    - Quickstart Java
> > >    - Quickstart Scala
> > >
> > >  - connectors / Input/Output Formats
> > >    - Avro
> > >    - HBase
> > >    - HadoopCompartibility
> > >    - HCatalogue
> > >    - JDBC
> > >    - kafka
> > >    - rabbit
> > >    - ...
> > >
> > >  - staging
> > >    - Gelly
> > >    - Gilbert (ML)
> > >    - spargel (deprecated)
> > >    - expression API
> > >
> > >  - contrib
> > >
> > >  - yarn
> > >
> > >  - dist
> > >
> > >  - yarn tests
> > >
> > >  - java 8
> > >
> > > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <[hidden email]
> >
> > > wrote:
> > >
> > > > Thanks Marton, having 2 threads discussing same thing can be
> confusing.
> > > >
> > > > - Henry
> > > >
> > > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <[hidden email]>
> > > > wrote:
> > > > > Let us consider this thread the standard for the restructure, it is
> > > > > perfectly in line with the wishes I have posted.
> > > > >
> > > > > +1 for keeping the 'flink-' prefix.
> > > > >
> > > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the
> > responses.
> > > > >>
> > > > >> - Henry
> > > > >>
> > > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]>
> > > wrote:
> > > > >> > I think this works well together with Marton's restructuring.
> > > > >> >
> > > > >> > I would vote to keep the "flink-" prefix, because it guarantees
> > that
> > > > the
> > > > >> > produced jars are prefixed with "flink-". Otherwise, we will
> have
> > to
> > > > >> start
> > > > >> > configuring a lot...
> > > > >> >
> > > > >> > Greetings,
> > > > >> > Stephan
> > > > >> >
> > > > >> >
> > > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > > > [hidden email]>
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Will this conflict with Marton's restructuring proposal which
> > > happens
> > > > >> >> in another thread (see "Project restructure" thread in the dev@
> > > > list).
> > > > >> >>
> > > > >> >> Since we are doing refactoring, may I suggest that we also
> remove
> > > > >> >> "flink-" prefix since maven group name will indicate it is part
> > of
> > > > >> >> Flink.
> > > > >> >>
> > > > >> >> - Henry
> > > > >> >>
> > > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]
> >
> > > > wrote:
> > > > >> >> > Hi everyone!
> > > > >> >> >
> > > > >> >> > I think that by now, quite a bit of the maven project
> structure
> > > > can be
> > > > >> >> > improved to get rid of some legacy artifacts. Especially the
> > > > >> >> "flink-addons"
> > > > >> >> > project seems to be a catch-all place for various projects.
> > > > >> >> >
> > > > >> >> > Here is a suggestion what we could do:
> > > > >> >> >
> > > > >> >> > 1) Move "flink-yarn" to the root.
> > > > >> >> >
> > > > >> >> > 2) Move "flink-streaming" to the root (this is planned
> anyways
> > > for
> > > > the
> > > > >> >> next
> > > > >> >> > release)
> > > > >> >> >
> > > > >> >> > 3) Create a project "flink-connectors", which will contain
> > > "avro",
> > > > >> >> "jdbc",
> > > > >> >> > and "hbase". Should we have them as separate sub-projects, or
> > as
> > > > one
> > > > >> >> > project?
> > > > >> >> >
> > > > >> >> > 4) Consolidate the examples into a single project
> > > "flink-examples",
> > > > >> where
> > > > >> >> > Java, Scala, Streaming examples exist in different packages.
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > Greetings,
> > > > >> >> > Stephan
> > > > >> >>
> > > > >>
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Ufuk Celebi-2
Thanks for bringing this up, Till. You are right, but I think the main
issue was that tight interaction between Java and Scala was problematic. I
am not sure whether this is such a big problem for the the APIs.

On Tue, Mar 17, 2015 at 3:21 PM, Till Rohrmann <[hidden email]> wrote:

> Putting the Scala and Java API into the same module means that we'll have
> more mixed Java/Scala projects, right? I just want to check if everyone is
> aware of it considering our latest experiences with these kind of modules.
>
> On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi <[hidden email]> wrote:
>
> > +1 I like the proposed structure.
> >
> > The only thing I was wondering about is whether to name "core" =>
> "batch"?
> >
> > On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi <
> [hidden email]
> > >
> > wrote:
> >
> > > +1 for the proposed structure.
> > >
> > > I have no explicit preference for having batch and streaming scala
> > together
> > > or separated. That said streaming scala is considerably thin, it does
> not
> > > really require an own maven submodule.
> > >
> > > Marked an older JIRA for the same issue as duplicate. [1]
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-1340
> > >
> > > On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <[hidden email]>
> wrote:
> > >
> > > > To not let this discussion die, here is a concrete JIRA and a
> proposed
> > > > layout to restructure to.
> > > >
> > > > What remains to be discusses is whether we want to keep the
> Scala/Java
> > > APIs
> > > > for batch/streaming in separate projects or in one project.
> > > >
> > > > Also, we need to find a good time to do this, when we are low on pull
> > > > requests...
> > > >
> > > > https://issues.apache.org/jira/browse/FLINK-1712
> > > >
> > > > Proposed Layout:
> > > >
> > > >  - flink-hadoop (shaded fat jar)
> > > >
> > > >  - Core (Core and Java and Scala)
> > > >  - Streaming (core + java + scala)
> > > >  - Runtime
> > > >  - Client (Client + Optimizer)
> > > >
> > > >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> > > >  - Tests (test-utils (compile) and tests (test))
> > > >
> > > >  - Quickstarts
> > > >    - Quickstart Java
> > > >    - Quickstart Scala
> > > >
> > > >  - connectors / Input/Output Formats
> > > >    - Avro
> > > >    - HBase
> > > >    - HadoopCompartibility
> > > >    - HCatalogue
> > > >    - JDBC
> > > >    - kafka
> > > >    - rabbit
> > > >    - ...
> > > >
> > > >  - staging
> > > >    - Gelly
> > > >    - Gilbert (ML)
> > > >    - spargel (deprecated)
> > > >    - expression API
> > > >
> > > >  - contrib
> > > >
> > > >  - yarn
> > > >
> > > >  - dist
> > > >
> > > >  - yarn tests
> > > >
> > > >  - java 8
> > > >
> > > > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Thanks Marton, having 2 threads discussing same thing can be
> > confusing.
> > > > >
> > > > > - Henry
> > > > >
> > > > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <
> [hidden email]>
> > > > > wrote:
> > > > > > Let us consider this thread the standard for the restructure, it
> is
> > > > > > perfectly in line with the wishes I have posted.
> > > > > >
> > > > > > +1 for keeping the 'flink-' prefix.
> > > > > >
> > > > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> > > > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the
> > > responses.
> > > > > >>
> > > > > >> - Henry
> > > > > >>
> > > > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]>
> > > > wrote:
> > > > > >> > I think this works well together with Marton's restructuring.
> > > > > >> >
> > > > > >> > I would vote to keep the "flink-" prefix, because it
> guarantees
> > > that
> > > > > the
> > > > > >> > produced jars are prefixed with "flink-". Otherwise, we will
> > have
> > > to
> > > > > >> start
> > > > > >> > configuring a lot...
> > > > > >> >
> > > > > >> > Greetings,
> > > > > >> > Stephan
> > > > > >> >
> > > > > >> >
> > > > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > > > > [hidden email]>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Will this conflict with Marton's restructuring proposal which
> > > > happens
> > > > > >> >> in another thread (see "Project restructure" thread in the
> dev@
> > > > > list).
> > > > > >> >>
> > > > > >> >> Since we are doing refactoring, may I suggest that we also
> > remove
> > > > > >> >> "flink-" prefix since maven group name will indicate it is
> part
> > > of
> > > > > >> >> Flink.
> > > > > >> >>
> > > > > >> >> - Henry
> > > > > >> >>
> > > > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <
> [hidden email]
> > >
> > > > > wrote:
> > > > > >> >> > Hi everyone!
> > > > > >> >> >
> > > > > >> >> > I think that by now, quite a bit of the maven project
> > structure
> > > > > can be
> > > > > >> >> > improved to get rid of some legacy artifacts. Especially
> the
> > > > > >> >> "flink-addons"
> > > > > >> >> > project seems to be a catch-all place for various projects.
> > > > > >> >> >
> > > > > >> >> > Here is a suggestion what we could do:
> > > > > >> >> >
> > > > > >> >> > 1) Move "flink-yarn" to the root.
> > > > > >> >> >
> > > > > >> >> > 2) Move "flink-streaming" to the root (this is planned
> > anyways
> > > > for
> > > > > the
> > > > > >> >> next
> > > > > >> >> > release)
> > > > > >> >> >
> > > > > >> >> > 3) Create a project "flink-connectors", which will contain
> > > > "avro",
> > > > > >> >> "jdbc",
> > > > > >> >> > and "hbase". Should we have them as separate sub-projects,
> or
> > > as
> > > > > one
> > > > > >> >> > project?
> > > > > >> >> >
> > > > > >> >> > 4) Consolidate the examples into a single project
> > > > "flink-examples",
> > > > > >> where
> > > > > >> >> > Java, Scala, Streaming examples exist in different
> packages.
> > > > > >> >> >
> > > > > >> >> >
> > > > > >> >> > Greetings,
> > > > > >> >> > Stephan
> > > > > >> >>
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Stephan Ewen
The good thing about the API projects is that there is no dependency from
Java code to Scala code. I think that caused most of the issues.

We may still want to keep it separate. I am not fully decided on this yet...

Stephan

On Tue, Mar 17, 2015 at 3:52 PM, Ufuk Celebi <[hidden email]> wrote:

> Thanks for bringing this up, Till. You are right, but I think the main
> issue was that tight interaction between Java and Scala was problematic. I
> am not sure whether this is such a big problem for the the APIs.
>
> On Tue, Mar 17, 2015 at 3:21 PM, Till Rohrmann <[hidden email]>
> wrote:
>
> > Putting the Scala and Java API into the same module means that we'll have
> > more mixed Java/Scala projects, right? I just want to check if everyone
> is
> > aware of it considering our latest experiences with these kind of
> modules.
> >
> > On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi <[hidden email]> wrote:
> >
> > > +1 I like the proposed structure.
> > >
> > > The only thing I was wondering about is whether to name "core" =>
> > "batch"?
> > >
> > > On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > +1 for the proposed structure.
> > > >
> > > > I have no explicit preference for having batch and streaming scala
> > > together
> > > > or separated. That said streaming scala is considerably thin, it does
> > not
> > > > really require an own maven submodule.
> > > >
> > > > Marked an older JIRA for the same issue as duplicate. [1]
> > > >
> > > > [1] https://issues.apache.org/jira/browse/FLINK-1340
> > > >
> > > > On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <[hidden email]>
> > wrote:
> > > >
> > > > > To not let this discussion die, here is a concrete JIRA and a
> > proposed
> > > > > layout to restructure to.
> > > > >
> > > > > What remains to be discusses is whether we want to keep the
> > Scala/Java
> > > > APIs
> > > > > for batch/streaming in separate projects or in one project.
> > > > >
> > > > > Also, we need to find a good time to do this, when we are low on
> pull
> > > > > requests...
> > > > >
> > > > > https://issues.apache.org/jira/browse/FLINK-1712
> > > > >
> > > > > Proposed Layout:
> > > > >
> > > > >  - flink-hadoop (shaded fat jar)
> > > > >
> > > > >  - Core (Core and Java and Scala)
> > > > >  - Streaming (core + java + scala)
> > > > >  - Runtime
> > > > >  - Client (Client + Optimizer)
> > > > >
> > > > >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> > > > >  - Tests (test-utils (compile) and tests (test))
> > > > >
> > > > >  - Quickstarts
> > > > >    - Quickstart Java
> > > > >    - Quickstart Scala
> > > > >
> > > > >  - connectors / Input/Output Formats
> > > > >    - Avro
> > > > >    - HBase
> > > > >    - HadoopCompartibility
> > > > >    - HCatalogue
> > > > >    - JDBC
> > > > >    - kafka
> > > > >    - rabbit
> > > > >    - ...
> > > > >
> > > > >  - staging
> > > > >    - Gelly
> > > > >    - Gilbert (ML)
> > > > >    - spargel (deprecated)
> > > > >    - expression API
> > > > >
> > > > >  - contrib
> > > > >
> > > > >  - yarn
> > > > >
> > > > >  - dist
> > > > >
> > > > >  - yarn tests
> > > > >
> > > > >  - java 8
> > > > >
> > > > > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > >
> > > > > > Thanks Marton, having 2 threads discussing same thing can be
> > > confusing.
> > > > > >
> > > > > > - Henry
> > > > > >
> > > > > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <
> > [hidden email]>
> > > > > > wrote:
> > > > > > > Let us consider this thread the standard for the restructure,
> it
> > is
> > > > > > > perfectly in line with the wishes I have posted.
> > > > > > >
> > > > > > > +1 for keeping the 'flink-' prefix.
> > > > > > >
> > > > > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> > > > > [hidden email]>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the
> > > > responses.
> > > > > > >>
> > > > > > >> - Henry
> > > > > > >>
> > > > > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <
> [hidden email]>
> > > > > wrote:
> > > > > > >> > I think this works well together with Marton's
> restructuring.
> > > > > > >> >
> > > > > > >> > I would vote to keep the "flink-" prefix, because it
> > guarantees
> > > > that
> > > > > > the
> > > > > > >> > produced jars are prefixed with "flink-". Otherwise, we will
> > > have
> > > > to
> > > > > > >> start
> > > > > > >> > configuring a lot...
> > > > > > >> >
> > > > > > >> > Greetings,
> > > > > > >> > Stephan
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > > > > > [hidden email]>
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> >> Will this conflict with Marton's restructuring proposal
> which
> > > > > happens
> > > > > > >> >> in another thread (see "Project restructure" thread in the
> > dev@
> > > > > > list).
> > > > > > >> >>
> > > > > > >> >> Since we are doing refactoring, may I suggest that we also
> > > remove
> > > > > > >> >> "flink-" prefix since maven group name will indicate it is
> > part
> > > > of
> > > > > > >> >> Flink.
> > > > > > >> >>
> > > > > > >> >> - Henry
> > > > > > >> >>
> > > > > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <
> > [hidden email]
> > > >
> > > > > > wrote:
> > > > > > >> >> > Hi everyone!
> > > > > > >> >> >
> > > > > > >> >> > I think that by now, quite a bit of the maven project
> > > structure
> > > > > > can be
> > > > > > >> >> > improved to get rid of some legacy artifacts. Especially
> > the
> > > > > > >> >> "flink-addons"
> > > > > > >> >> > project seems to be a catch-all place for various
> projects.
> > > > > > >> >> >
> > > > > > >> >> > Here is a suggestion what we could do:
> > > > > > >> >> >
> > > > > > >> >> > 1) Move "flink-yarn" to the root.
> > > > > > >> >> >
> > > > > > >> >> > 2) Move "flink-streaming" to the root (this is planned
> > > anyways
> > > > > for
> > > > > > the
> > > > > > >> >> next
> > > > > > >> >> > release)
> > > > > > >> >> >
> > > > > > >> >> > 3) Create a project "flink-connectors", which will
> contain
> > > > > "avro",
> > > > > > >> >> "jdbc",
> > > > > > >> >> > and "hbase". Should we have them as separate
> sub-projects,
> > or
> > > > as
> > > > > > one
> > > > > > >> >> > project?
> > > > > > >> >> >
> > > > > > >> >> > 4) Consolidate the examples into a single project
> > > > > "flink-examples",
> > > > > > >> where
> > > > > > >> >> > Java, Scala, Streaming examples exist in different
> > packages.
> > > > > > >> >> >
> > > > > > >> >> >
> > > > > > >> >> > Greetings,
> > > > > > >> >> > Stephan
> > > > > > >> >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Fabian Hueske-2
I agree that it's a good idea to move the APIs into one module.

But why should we merge client and compiler (optimizer) and the examples
into one module?
I think modules with clearly separated responsibilities can also help new
contributors to navigate the code.

2015-03-17 16:16 GMT+01:00 Stephan Ewen <[hidden email]>:

> The good thing about the API projects is that there is no dependency from
> Java code to Scala code. I think that caused most of the issues.
>
> We may still want to keep it separate. I am not fully decided on this
> yet...
>
> Stephan
>
> On Tue, Mar 17, 2015 at 3:52 PM, Ufuk Celebi <[hidden email]> wrote:
>
> > Thanks for bringing this up, Till. You are right, but I think the main
> > issue was that tight interaction between Java and Scala was problematic.
> I
> > am not sure whether this is such a big problem for the the APIs.
> >
> > On Tue, Mar 17, 2015 at 3:21 PM, Till Rohrmann <[hidden email]>
> > wrote:
> >
> > > Putting the Scala and Java API into the same module means that we'll
> have
> > > more mixed Java/Scala projects, right? I just want to check if everyone
> > is
> > > aware of it considering our latest experiences with these kind of
> > modules.
> > >
> > > On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi <[hidden email]> wrote:
> > >
> > > > +1 I like the proposed structure.
> > > >
> > > > The only thing I was wondering about is whether to name "core" =>
> > > "batch"?
> > > >
> > > > On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi <
> > > [hidden email]
> > > > >
> > > > wrote:
> > > >
> > > > > +1 for the proposed structure.
> > > > >
> > > > > I have no explicit preference for having batch and streaming scala
> > > > together
> > > > > or separated. That said streaming scala is considerably thin, it
> does
> > > not
> > > > > really require an own maven submodule.
> > > > >
> > > > > Marked an older JIRA for the same issue as duplicate. [1]
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/FLINK-1340
> > > > >
> > > > > On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <[hidden email]>
> > > wrote:
> > > > >
> > > > > > To not let this discussion die, here is a concrete JIRA and a
> > > proposed
> > > > > > layout to restructure to.
> > > > > >
> > > > > > What remains to be discusses is whether we want to keep the
> > > Scala/Java
> > > > > APIs
> > > > > > for batch/streaming in separate projects or in one project.
> > > > > >
> > > > > > Also, we need to find a good time to do this, when we are low on
> > pull
> > > > > > requests...
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/FLINK-1712
> > > > > >
> > > > > > Proposed Layout:
> > > > > >
> > > > > >  - flink-hadoop (shaded fat jar)
> > > > > >
> > > > > >  - Core (Core and Java and Scala)
> > > > > >  - Streaming (core + java + scala)
> > > > > >  - Runtime
> > > > > >  - Client (Client + Optimizer)
> > > > > >
> > > > > >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> > > > > >  - Tests (test-utils (compile) and tests (test))
> > > > > >
> > > > > >  - Quickstarts
> > > > > >    - Quickstart Java
> > > > > >    - Quickstart Scala
> > > > > >
> > > > > >  - connectors / Input/Output Formats
> > > > > >    - Avro
> > > > > >    - HBase
> > > > > >    - HadoopCompartibility
> > > > > >    - HCatalogue
> > > > > >    - JDBC
> > > > > >    - kafka
> > > > > >    - rabbit
> > > > > >    - ...
> > > > > >
> > > > > >  - staging
> > > > > >    - Gelly
> > > > > >    - Gilbert (ML)
> > > > > >    - spargel (deprecated)
> > > > > >    - expression API
> > > > > >
> > > > > >  - contrib
> > > > > >
> > > > > >  - yarn
> > > > > >
> > > > > >  - dist
> > > > > >
> > > > > >  - yarn tests
> > > > > >
> > > > > >  - java 8
> > > > > >
> > > > > > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <
> > > [hidden email]
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Thanks Marton, having 2 threads discussing same thing can be
> > > > confusing.
> > > > > > >
> > > > > > > - Henry
> > > > > > >
> > > > > > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <
> > > [hidden email]>
> > > > > > > wrote:
> > > > > > > > Let us consider this thread the standard for the restructure,
> > it
> > > is
> > > > > > > > perfectly in line with the wishes I have posted.
> > > > > > > >
> > > > > > > > +1 for keeping the 'flink-' prefix.
> > > > > > > >
> > > > > > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> > > > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> @Robert, and @Stephan, sure I am ok with it, thanks for the
> > > > > responses.
> > > > > > > >>
> > > > > > > >> - Henry
> > > > > > > >>
> > > > > > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <
> > [hidden email]>
> > > > > > wrote:
> > > > > > > >> > I think this works well together with Marton's
> > restructuring.
> > > > > > > >> >
> > > > > > > >> > I would vote to keep the "flink-" prefix, because it
> > > guarantees
> > > > > that
> > > > > > > the
> > > > > > > >> > produced jars are prefixed with "flink-". Otherwise, we
> will
> > > > have
> > > > > to
> > > > > > > >> start
> > > > > > > >> > configuring a lot...
> > > > > > > >> >
> > > > > > > >> > Greetings,
> > > > > > > >> > Stephan
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > > > > > > [hidden email]>
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> >> Will this conflict with Marton's restructuring proposal
> > which
> > > > > > happens
> > > > > > > >> >> in another thread (see "Project restructure" thread in
> the
> > > dev@
> > > > > > > list).
> > > > > > > >> >>
> > > > > > > >> >> Since we are doing refactoring, may I suggest that we
> also
> > > > remove
> > > > > > > >> >> "flink-" prefix since maven group name will indicate it
> is
> > > part
> > > > > of
> > > > > > > >> >> Flink.
> > > > > > > >> >>
> > > > > > > >> >> - Henry
> > > > > > > >> >>
> > > > > > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <
> > > [hidden email]
> > > > >
> > > > > > > wrote:
> > > > > > > >> >> > Hi everyone!
> > > > > > > >> >> >
> > > > > > > >> >> > I think that by now, quite a bit of the maven project
> > > > structure
> > > > > > > can be
> > > > > > > >> >> > improved to get rid of some legacy artifacts.
> Especially
> > > the
> > > > > > > >> >> "flink-addons"
> > > > > > > >> >> > project seems to be a catch-all place for various
> > projects.
> > > > > > > >> >> >
> > > > > > > >> >> > Here is a suggestion what we could do:
> > > > > > > >> >> >
> > > > > > > >> >> > 1) Move "flink-yarn" to the root.
> > > > > > > >> >> >
> > > > > > > >> >> > 2) Move "flink-streaming" to the root (this is planned
> > > > anyways
> > > > > > for
> > > > > > > the
> > > > > > > >> >> next
> > > > > > > >> >> > release)
> > > > > > > >> >> >
> > > > > > > >> >> > 3) Create a project "flink-connectors", which will
> > contain
> > > > > > "avro",
> > > > > > > >> >> "jdbc",
> > > > > > > >> >> > and "hbase". Should we have them as separate
> > sub-projects,
> > > or
> > > > > as
> > > > > > > one
> > > > > > > >> >> > project?
> > > > > > > >> >> >
> > > > > > > >> >> > 4) Consolidate the examples into a single project
> > > > > > "flink-examples",
> > > > > > > >> where
> > > > > > > >> >> > Java, Scala, Streaming examples exist in different
> > > packages.
> > > > > > > >> >> >
> > > > > > > >> >> >
> > > > > > > >> >> > Greetings,
> > > > > > > >> >> > Stephan
> > > > > > > >> >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Stephan Ewen
Optimizer and Compiler can stay separate, of course.

I was just thinking that we have a lot of projects and this would be a
simple way to get rid of one. But it actually is intuitive to me as well to
keep them separate.

On Wed, Mar 18, 2015 at 1:23 AM, Fabian Hueske <[hidden email]> wrote:

> I agree that it's a good idea to move the APIs into one module.
>
> But why should we merge client and compiler (optimizer) and the examples
> into one module?
> I think modules with clearly separated responsibilities can also help new
> contributors to navigate the code.
>
> 2015-03-17 16:16 GMT+01:00 Stephan Ewen <[hidden email]>:
>
> > The good thing about the API projects is that there is no dependency from
> > Java code to Scala code. I think that caused most of the issues.
> >
> > We may still want to keep it separate. I am not fully decided on this
> > yet...
> >
> > Stephan
> >
> > On Tue, Mar 17, 2015 at 3:52 PM, Ufuk Celebi <[hidden email]> wrote:
> >
> > > Thanks for bringing this up, Till. You are right, but I think the main
> > > issue was that tight interaction between Java and Scala was
> problematic.
> > I
> > > am not sure whether this is such a big problem for the the APIs.
> > >
> > > On Tue, Mar 17, 2015 at 3:21 PM, Till Rohrmann <[hidden email]>
> > > wrote:
> > >
> > > > Putting the Scala and Java API into the same module means that we'll
> > have
> > > > more mixed Java/Scala projects, right? I just want to check if
> everyone
> > > is
> > > > aware of it considering our latest experiences with these kind of
> > > modules.
> > > >
> > > > On Tue, Mar 17, 2015 at 2:21 PM, Ufuk Celebi <[hidden email]> wrote:
> > > >
> > > > > +1 I like the proposed structure.
> > > > >
> > > > > The only thing I was wondering about is whether to name "core" =>
> > > > "batch"?
> > > > >
> > > > > On Tue, Mar 17, 2015 at 11:37 AM, Márton Balassi <
> > > > [hidden email]
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > +1 for the proposed structure.
> > > > > >
> > > > > > I have no explicit preference for having batch and streaming
> scala
> > > > > together
> > > > > > or separated. That said streaming scala is considerably thin, it
> > does
> > > > not
> > > > > > really require an own maven submodule.
> > > > > >
> > > > > > Marked an older JIRA for the same issue as duplicate. [1]
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/FLINK-1340
> > > > > >
> > > > > > On Tue, Mar 17, 2015 at 10:46 AM, Stephan Ewen <[hidden email]
> >
> > > > wrote:
> > > > > >
> > > > > > > To not let this discussion die, here is a concrete JIRA and a
> > > > proposed
> > > > > > > layout to restructure to.
> > > > > > >
> > > > > > > What remains to be discusses is whether we want to keep the
> > > > Scala/Java
> > > > > > APIs
> > > > > > > for batch/streaming in separate projects or in one project.
> > > > > > >
> > > > > > > Also, we need to find a good time to do this, when we are low
> on
> > > pull
> > > > > > > requests...
> > > > > > >
> > > > > > > https://issues.apache.org/jira/browse/FLINK-1712
> > > > > > >
> > > > > > > Proposed Layout:
> > > > > > >
> > > > > > >  - flink-hadoop (shaded fat jar)
> > > > > > >
> > > > > > >  - Core (Core and Java and Scala)
> > > > > > >  - Streaming (core + java + scala)
> > > > > > >  - Runtime
> > > > > > >  - Client (Client + Optimizer)
> > > > > > >
> > > > > > >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> > > > > > >  - Tests (test-utils (compile) and tests (test))
> > > > > > >
> > > > > > >  - Quickstarts
> > > > > > >    - Quickstart Java
> > > > > > >    - Quickstart Scala
> > > > > > >
> > > > > > >  - connectors / Input/Output Formats
> > > > > > >    - Avro
> > > > > > >    - HBase
> > > > > > >    - HadoopCompartibility
> > > > > > >    - HCatalogue
> > > > > > >    - JDBC
> > > > > > >    - kafka
> > > > > > >    - rabbit
> > > > > > >    - ...
> > > > > > >
> > > > > > >  - staging
> > > > > > >    - Gelly
> > > > > > >    - Gilbert (ML)
> > > > > > >    - spargel (deprecated)
> > > > > > >    - expression API
> > > > > > >
> > > > > > >  - contrib
> > > > > > >
> > > > > > >  - yarn
> > > > > > >
> > > > > > >  - dist
> > > > > > >
> > > > > > >  - yarn tests
> > > > > > >
> > > > > > >  - java 8
> > > > > > >
> > > > > > > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <
> > > > [hidden email]
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Thanks Marton, having 2 threads discussing same thing can be
> > > > > confusing.
> > > > > > > >
> > > > > > > > - Henry
> > > > > > > >
> > > > > > > > On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <
> > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > > > Let us consider this thread the standard for the
> restructure,
> > > it
> > > > is
> > > > > > > > > perfectly in line with the wishes I have posted.
> > > > > > > > >
> > > > > > > > > +1 for keeping the 'flink-' prefix.
> > > > > > > > >
> > > > > > > > > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> > > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> @Robert, and @Stephan, sure I am ok with it, thanks for
> the
> > > > > > responses.
> > > > > > > > >>
> > > > > > > > >> - Henry
> > > > > > > > >>
> > > > > > > > >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <
> > > [hidden email]>
> > > > > > > wrote:
> > > > > > > > >> > I think this works well together with Marton's
> > > restructuring.
> > > > > > > > >> >
> > > > > > > > >> > I would vote to keep the "flink-" prefix, because it
> > > > guarantees
> > > > > > that
> > > > > > > > the
> > > > > > > > >> > produced jars are prefixed with "flink-". Otherwise, we
> > will
> > > > > have
> > > > > > to
> > > > > > > > >> start
> > > > > > > > >> > configuring a lot...
> > > > > > > > >> >
> > > > > > > > >> > Greetings,
> > > > > > > > >> > Stephan
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> > > > > > > > [hidden email]>
> > > > > > > > >> > wrote:
> > > > > > > > >> >
> > > > > > > > >> >> Will this conflict with Marton's restructuring proposal
> > > which
> > > > > > > happens
> > > > > > > > >> >> in another thread (see "Project restructure" thread in
> > the
> > > > dev@
> > > > > > > > list).
> > > > > > > > >> >>
> > > > > > > > >> >> Since we are doing refactoring, may I suggest that we
> > also
> > > > > remove
> > > > > > > > >> >> "flink-" prefix since maven group name will indicate it
> > is
> > > > part
> > > > > > of
> > > > > > > > >> >> Flink.
> > > > > > > > >> >>
> > > > > > > > >> >> - Henry
> > > > > > > > >> >>
> > > > > > > > >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <
> > > > [hidden email]
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >> >> > Hi everyone!
> > > > > > > > >> >> >
> > > > > > > > >> >> > I think that by now, quite a bit of the maven project
> > > > > structure
> > > > > > > > can be
> > > > > > > > >> >> > improved to get rid of some legacy artifacts.
> > Especially
> > > > the
> > > > > > > > >> >> "flink-addons"
> > > > > > > > >> >> > project seems to be a catch-all place for various
> > > projects.
> > > > > > > > >> >> >
> > > > > > > > >> >> > Here is a suggestion what we could do:
> > > > > > > > >> >> >
> > > > > > > > >> >> > 1) Move "flink-yarn" to the root.
> > > > > > > > >> >> >
> > > > > > > > >> >> > 2) Move "flink-streaming" to the root (this is
> planned
> > > > > anyways
> > > > > > > for
> > > > > > > > the
> > > > > > > > >> >> next
> > > > > > > > >> >> > release)
> > > > > > > > >> >> >
> > > > > > > > >> >> > 3) Create a project "flink-connectors", which will
> > > contain
> > > > > > > "avro",
> > > > > > > > >> >> "jdbc",
> > > > > > > > >> >> > and "hbase". Should we have them as separate
> > > sub-projects,
> > > > or
> > > > > > as
> > > > > > > > one
> > > > > > > > >> >> > project?
> > > > > > > > >> >> >
> > > > > > > > >> >> > 4) Consolidate the examples into a single project
> > > > > > > "flink-examples",
> > > > > > > > >> where
> > > > > > > > >> >> > Java, Scala, Streaming examples exist in different
> > > > packages.
> > > > > > > > >> >> >
> > > > > > > > >> >> >
> > > > > > > > >> >> > Greetings,
> > > > > > > > >> >> > Stephan
> > > > > > > > >> >>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Henry Saputra
In reply to this post by Stephan Ewen
Also looks like runtime getting too big.

Thoughts about moving the web info frontend to separate maven module?

- Henry


On Tue, Mar 17, 2015 at 2:46 AM, Stephan Ewen <[hidden email]> wrote:

> To not let this discussion die, here is a concrete JIRA and a proposed
> layout to restructure to.
>
> What remains to be discusses is whether we want to keep the Scala/Java APIs
> for batch/streaming in separate projects or in one project.
>
> Also, we need to find a good time to do this, when we are low on pull
> requests...
>
> https://issues.apache.org/jira/browse/FLINK-1712
>
> Proposed Layout:
>
>  - flink-hadoop (shaded fat jar)
>
>  - Core (Core and Java and Scala)
>  - Streaming (core + java + scala)
>  - Runtime
>  - Client (Client + Optimizer)
>
>  - Examples (Java + Scala + Streaming Java + Streaming Scala)
>  - Tests (test-utils (compile) and tests (test))
>
>  - Quickstarts
>    - Quickstart Java
>    - Quickstart Scala
>
>  - connectors / Input/Output Formats
>    - Avro
>    - HBase
>    - HadoopCompartibility
>    - HCatalogue
>    - JDBC
>    - kafka
>    - rabbit
>    - ...
>
>  - staging
>    - Gelly
>    - Gilbert (ML)
>    - spargel (deprecated)
>    - expression API
>
>  - contrib
>
>  - yarn
>
>  - dist
>
>  - yarn tests
>
>  - java 8
>
> On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <[hidden email]>
> wrote:
>
>> Thanks Marton, having 2 threads discussing same thing can be confusing.
>>
>> - Henry
>>
>> On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <[hidden email]>
>> wrote:
>> > Let us consider this thread the standard for the restructure, it is
>> > perfectly in line with the wishes I have posted.
>> >
>> > +1 for keeping the 'flink-' prefix.
>> >
>> > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <[hidden email]>
>> > wrote:
>> >
>> >> @Robert, and @Stephan, sure I am ok with it, thanks for the responses.
>> >>
>> >> - Henry
>> >>
>> >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]> wrote:
>> >> > I think this works well together with Marton's restructuring.
>> >> >
>> >> > I would vote to keep the "flink-" prefix, because it guarantees that
>> the
>> >> > produced jars are prefixed with "flink-". Otherwise, we will have to
>> >> start
>> >> > configuring a lot...
>> >> >
>> >> > Greetings,
>> >> > Stephan
>> >> >
>> >> >
>> >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
>> [hidden email]>
>> >> > wrote:
>> >> >
>> >> >> Will this conflict with Marton's restructuring proposal which happens
>> >> >> in another thread (see "Project restructure" thread in the dev@
>> list).
>> >> >>
>> >> >> Since we are doing refactoring, may I suggest that we also remove
>> >> >> "flink-" prefix since maven group name will indicate it is part of
>> >> >> Flink.
>> >> >>
>> >> >> - Henry
>> >> >>
>> >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]>
>> wrote:
>> >> >> > Hi everyone!
>> >> >> >
>> >> >> > I think that by now, quite a bit of the maven project structure
>> can be
>> >> >> > improved to get rid of some legacy artifacts. Especially the
>> >> >> "flink-addons"
>> >> >> > project seems to be a catch-all place for various projects.
>> >> >> >
>> >> >> > Here is a suggestion what we could do:
>> >> >> >
>> >> >> > 1) Move "flink-yarn" to the root.
>> >> >> >
>> >> >> > 2) Move "flink-streaming" to the root (this is planned anyways for
>> the
>> >> >> next
>> >> >> > release)
>> >> >> >
>> >> >> > 3) Create a project "flink-connectors", which will contain "avro",
>> >> >> "jdbc",
>> >> >> > and "hbase". Should we have them as separate sub-projects, or as
>> one
>> >> >> > project?
>> >> >> >
>> >> >> > 4) Consolidate the examples into a single project "flink-examples",
>> >> where
>> >> >> > Java, Scala, Streaming examples exist in different packages.
>> >> >> >
>> >> >> >
>> >> >> > Greetings,
>> >> >> > Stephan
>> >> >>
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Restructuring the maven projects

Stephan Ewen
I am reworking the web frontend a bit (PR may come in a bit) and was
thinking the same thing.

Would also allow to vastly reduce the runtime dependencies, since all the
webserver stuff would be out...

On Sun, Mar 29, 2015 at 9:11 PM, Henry Saputra <[hidden email]>
wrote:

> Also looks like runtime getting too big.
>
> Thoughts about moving the web info frontend to separate maven module?
>
> - Henry
>
>
> On Tue, Mar 17, 2015 at 2:46 AM, Stephan Ewen <[hidden email]> wrote:
> > To not let this discussion die, here is a concrete JIRA and a proposed
> > layout to restructure to.
> >
> > What remains to be discusses is whether we want to keep the Scala/Java
> APIs
> > for batch/streaming in separate projects or in one project.
> >
> > Also, we need to find a good time to do this, when we are low on pull
> > requests...
> >
> > https://issues.apache.org/jira/browse/FLINK-1712
> >
> > Proposed Layout:
> >
> >  - flink-hadoop (shaded fat jar)
> >
> >  - Core (Core and Java and Scala)
> >  - Streaming (core + java + scala)
> >  - Runtime
> >  - Client (Client + Optimizer)
> >
> >  - Examples (Java + Scala + Streaming Java + Streaming Scala)
> >  - Tests (test-utils (compile) and tests (test))
> >
> >  - Quickstarts
> >    - Quickstart Java
> >    - Quickstart Scala
> >
> >  - connectors / Input/Output Formats
> >    - Avro
> >    - HBase
> >    - HadoopCompartibility
> >    - HCatalogue
> >    - JDBC
> >    - kafka
> >    - rabbit
> >    - ...
> >
> >  - staging
> >    - Gelly
> >    - Gilbert (ML)
> >    - spargel (deprecated)
> >    - expression API
> >
> >  - contrib
> >
> >  - yarn
> >
> >  - dist
> >
> >  - yarn tests
> >
> >  - java 8
> >
> > On Mon, Jan 5, 2015 at 7:45 PM, Henry Saputra <[hidden email]>
> > wrote:
> >
> >> Thanks Marton, having 2 threads discussing same thing can be confusing.
> >>
> >> - Henry
> >>
> >> On Mon, Jan 5, 2015 at 3:52 AM, Márton Balassi <[hidden email]>
> >> wrote:
> >> > Let us consider this thread the standard for the restructure, it is
> >> > perfectly in line with the wishes I have posted.
> >> >
> >> > +1 for keeping the 'flink-' prefix.
> >> >
> >> > On Mon, Jan 5, 2015 at 10:24 AM, Henry Saputra <
> [hidden email]>
> >> > wrote:
> >> >
> >> >> @Robert, and @Stephan, sure I am ok with it, thanks for the
> responses.
> >> >>
> >> >> - Henry
> >> >>
> >> >> On Mon, Jan 5, 2015 at 1:18 AM, Stephan Ewen <[hidden email]>
> wrote:
> >> >> > I think this works well together with Marton's restructuring.
> >> >> >
> >> >> > I would vote to keep the "flink-" prefix, because it guarantees
> that
> >> the
> >> >> > produced jars are prefixed with "flink-". Otherwise, we will have
> to
> >> >> start
> >> >> > configuring a lot...
> >> >> >
> >> >> > Greetings,
> >> >> > Stephan
> >> >> >
> >> >> >
> >> >> > On Mon, Jan 5, 2015 at 9:32 AM, Henry Saputra <
> >> [hidden email]>
> >> >> > wrote:
> >> >> >
> >> >> >> Will this conflict with Marton's restructuring proposal which
> happens
> >> >> >> in another thread (see "Project restructure" thread in the dev@
> >> list).
> >> >> >>
> >> >> >> Since we are doing refactoring, may I suggest that we also remove
> >> >> >> "flink-" prefix since maven group name will indicate it is part of
> >> >> >> Flink.
> >> >> >>
> >> >> >> - Henry
> >> >> >>
> >> >> >> On Fri, Jan 2, 2015 at 6:52 AM, Stephan Ewen <[hidden email]>
> >> wrote:
> >> >> >> > Hi everyone!
> >> >> >> >
> >> >> >> > I think that by now, quite a bit of the maven project structure
> >> can be
> >> >> >> > improved to get rid of some legacy artifacts. Especially the
> >> >> >> "flink-addons"
> >> >> >> > project seems to be a catch-all place for various projects.
> >> >> >> >
> >> >> >> > Here is a suggestion what we could do:
> >> >> >> >
> >> >> >> > 1) Move "flink-yarn" to the root.
> >> >> >> >
> >> >> >> > 2) Move "flink-streaming" to the root (this is planned anyways
> for
> >> the
> >> >> >> next
> >> >> >> > release)
> >> >> >> >
> >> >> >> > 3) Create a project "flink-connectors", which will contain
> "avro",
> >> >> >> "jdbc",
> >> >> >> > and "hbase". Should we have them as separate sub-projects, or as
> >> one
> >> >> >> > project?
> >> >> >> >
> >> >> >> > 4) Consolidate the examples into a single project
> "flink-examples",
> >> >> where
> >> >> >> > Java, Scala, Streaming examples exist in different packages.
> >> >> >> >
> >> >> >> >
> >> >> >> > Greetings,
> >> >> >> > Stephan
> >> >> >>
> >> >>
> >>
>