errors after updating master branch

[Daniel Blazevski]

Hello,

I recently made a pull request for an exact knn algorithm, and have been
considering to start on the approximate knn algorithm and had an issue with
updating the master branch of Flink.

I am curious to know what best practices are in terms of keeping up to date
with the master branch to avoid the issue I had (to be explained below)

Some background:
-- I forked another Flink commiter's branch of Flink since he had done
other work on knn (namely https://github.com/chiwanpark/flink)
-- I imported that master branch into IntelliJ
-- I wrote code on a separate branch FLINK-1745, and that built fine.
-- I made the mistake in not using Travis CI from the start, but that was
not a huge issue since only a single import needed to be changed and my
Travis CI build goes through now.

I then checked out the master branch, and updated master via
`git pull apache master`

The master branch no longer builds, I get the following error in IngelliJ:

Error:scalac: Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/classes
is shared between: Module 'flink-storm-compatibility-core' production,
Module 'flink-storm-compatibility-core (1)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/classes
is shared between: Module 'flink-connector-kafka' production, Module
'flink-connector-kafka (1)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/classes
is shared between: Module 'flink-spargel' production, Module 'flink-spargel
(2)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/classes
is shared between: Module 'flink-streaming (2)' production, Module
'flink-streaming-parent' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/test-classes
is shared between: Module 'flink-storm-compatibility-examples' tests,
Module 'flink-storm-compatibility-examples (1)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/test-classes
is shared between: Module 'flink-connector-rabbitmq' tests, Module
'flink-connector-rabbitmq (2)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/classes
is shared between: Module 'flink-connector-elasticsearch' production,
Module 'flink-connector-elasticsearch (2)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/classes
is shared between: Module 'flink-streaming-connectors (1)' production,
Module 'flink-streaming-connectors-parent' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/test-classes
is shared between: Module 'flink-connector-elasticsearch' tests, Module
'flink-connector-elasticsearch (2)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/test-classes
is shared between: Module 'flink-storm-compatibility-core' tests, Module
'flink-storm-compatibility-core (1)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/test-classes
is shared between: Module 'flink-streaming-scala' tests, Module
'flink-streaming-scala (1)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/test-classes
is shared between: Module 'flink-storm-compatibility (1)' tests, Module
'flink-storm-compatibility-parent' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/test-classes
is shared between: Module 'flink-gelly' tests, Module 'flink-gelly (4)'
tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/classes
is shared between: Module 'flink-connector-rabbitmq' production, Module
'flink-connector-rabbitmq (2)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/test-classes
is shared between: Module 'flink-spargel' tests, Module 'flink-spargel (2)'
tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/test-classes
is shared between: Module 'flink-connector-twitter' tests, Module
'flink-connector-twitter (2)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/classes
is shared between: Module 'flink-connector-twitter' production, Module
'flink-connector-twitter (2)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/classes
is shared between: Module 'flink-gelly' production, Module 'flink-gelly
(4)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/test-classes
is shared between: Module 'flink-connector-kafka' tests, Module
'flink-connector-kafka (1)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/test-classes
is shared between: Module 'flink-streaming-connectors (1)' tests, Module
'flink-streaming-connectors-parent' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/test-classes
is shared between: Module 'flink-connector-flume' tests, Module
'flink-connector-flume (1)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/classes
is shared between: Module 'flink-streaming-examples' production, Module
'flink-streaming-examples (1)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/test-classes
is shared between: Module 'flink-streaming-core' tests, Module
'flink-streaming-core (1)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/test-classes
is shared between: Module 'flink-streaming-examples' tests, Module
'flink-streaming-examples (1)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/classes
is shared between: Module 'flink-storm-compatibility (1)' production,
Module 'flink-storm-compatibility-parent' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/classes
is shared between: Module 'flink-streaming-core' production, Module
'flink-streaming-core (1)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/test-classes
is shared between: Module 'flink-streaming (2)' tests, Module
'flink-streaming-parent' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/classes
is shared between: Module 'flink-gelly-scala' production, Module
'flink-gelly-scala (4)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/classes
is shared between: Module 'flink-connector-flume' production, Module
'flink-connector-flume (1)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/test-classes
is shared between: Module 'flink-gelly-scala' tests, Module
'flink-gelly-scala (4)' tests
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/classes
is shared between: Module 'flink-streaming-scala' production, Module
'flink-streaming-scala (1)' production
Output path
/Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/classes
is shared between: Module 'flink-storm-compatibility-examples' production,
Module 'flink-storm-compatibility-examples (1)' production
Please configure separate output paths to proceed with the compilation.
TIP: you can use Project Artifacts to combine compiled classes if needed.


What is the most seamless way to
(1) pull master branch + build --- >  (2) create new branch ---> (3) make
pull request ---> (4) update master ---> (5) create new branch ----> repeat

I was able to do steps (1)-(3), but got an error on step (4).  The only
solution I can think of is a hack:  import a whole new project into
IntelliJ using the most up-to-date version of master, but that sounds silly
to do that each time I want to repeat step (4)

Thanks!
Dan

[Daniel Blazevski]

P.S. minor typo, a slightly better chart than

(1) pull master branch + build --- >  (2) create new branch ---> (3) make
pull request ---> (4) update master ---> (5) create new branch ----> repeat

is

(1) clone master branch + build --- >  (2) create new branch ---> (3) make
pull request ---> (4) update master ---> (5) create new branch ----> (6)
make pull request ---> repeat (4)-(6)



On Mon, Nov 2, 2015 at 5:21 PM, Daniel Blazevski <[hidden email]
> wrote:

> Hello,
>
> I recently made a pull request for an exact knn algorithm, and have been
> considering to start on the approximate knn algorithm and had an issue with
> updating the master branch of Flink.
>
> I am curious to know what best practices are in terms of keeping up to
> date with the master branch to avoid the issue I had (to be explained below)
>
> Some background:
> -- I forked another Flink commiter's branch of Flink since he had done
> other work on knn (namely https://github.com/chiwanpark/flink)
> -- I imported that master branch into IntelliJ
> -- I wrote code on a separate branch FLINK-1745, and that built fine.
> -- I made the mistake in not using Travis CI from the start, but that was
> not a huge issue since only a single import needed to be changed and my
> Travis CI build goes through now.
>
> I then checked out the master branch, and updated master via
> `git pull apache master`
>
> The master branch no longer builds, I get the following error in IngelliJ:
>
> Error:scalac: Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/classes
> is shared between: Module 'flink-storm-compatibility-core' production,
> Module 'flink-storm-compatibility-core (1)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/classes
> is shared between: Module 'flink-connector-kafka' production, Module
> 'flink-connector-kafka (1)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/classes
> is shared between: Module 'flink-spargel' production, Module 'flink-spargel
> (2)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/classes
> is shared between: Module 'flink-streaming (2)' production, Module
> 'flink-streaming-parent' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/test-classes
> is shared between: Module 'flink-storm-compatibility-examples' tests,
> Module 'flink-storm-compatibility-examples (1)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/test-classes
> is shared between: Module 'flink-connector-rabbitmq' tests, Module
> 'flink-connector-rabbitmq (2)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/classes
> is shared between: Module 'flink-connector-elasticsearch' production,
> Module 'flink-connector-elasticsearch (2)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/classes
> is shared between: Module 'flink-streaming-connectors (1)' production,
> Module 'flink-streaming-connectors-parent' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/test-classes
> is shared between: Module 'flink-connector-elasticsearch' tests, Module
> 'flink-connector-elasticsearch (2)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/test-classes
> is shared between: Module 'flink-storm-compatibility-core' tests, Module
> 'flink-storm-compatibility-core (1)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/test-classes
> is shared between: Module 'flink-streaming-scala' tests, Module
> 'flink-streaming-scala (1)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/test-classes
> is shared between: Module 'flink-storm-compatibility (1)' tests, Module
> 'flink-storm-compatibility-parent' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/test-classes
> is shared between: Module 'flink-gelly' tests, Module 'flink-gelly (4)'
> tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/classes
> is shared between: Module 'flink-connector-rabbitmq' production, Module
> 'flink-connector-rabbitmq (2)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/test-classes
> is shared between: Module 'flink-spargel' tests, Module 'flink-spargel (2)'
> tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/test-classes
> is shared between: Module 'flink-connector-twitter' tests, Module
> 'flink-connector-twitter (2)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/classes
> is shared between: Module 'flink-connector-twitter' production, Module
> 'flink-connector-twitter (2)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/classes
> is shared between: Module 'flink-gelly' production, Module 'flink-gelly
> (4)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/test-classes
> is shared between: Module 'flink-connector-kafka' tests, Module
> 'flink-connector-kafka (1)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/test-classes
> is shared between: Module 'flink-streaming-connectors (1)' tests, Module
> 'flink-streaming-connectors-parent' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/test-classes
> is shared between: Module 'flink-connector-flume' tests, Module
> 'flink-connector-flume (1)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/classes
> is shared between: Module 'flink-streaming-examples' production, Module
> 'flink-streaming-examples (1)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/test-classes
> is shared between: Module 'flink-streaming-core' tests, Module
> 'flink-streaming-core (1)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/test-classes
> is shared between: Module 'flink-streaming-examples' tests, Module
> 'flink-streaming-examples (1)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/classes
> is shared between: Module 'flink-storm-compatibility (1)' production,
> Module 'flink-storm-compatibility-parent' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/classes
> is shared between: Module 'flink-streaming-core' production, Module
> 'flink-streaming-core (1)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/test-classes
> is shared between: Module 'flink-streaming (2)' tests, Module
> 'flink-streaming-parent' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/classes
> is shared between: Module 'flink-gelly-scala' production, Module
> 'flink-gelly-scala (4)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/classes
> is shared between: Module 'flink-connector-flume' production, Module
> 'flink-connector-flume (1)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/test-classes
> is shared between: Module 'flink-gelly-scala' tests, Module
> 'flink-gelly-scala (4)' tests
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/classes
> is shared between: Module 'flink-streaming-scala' production, Module
> 'flink-streaming-scala (1)' production
> Output path
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/classes
> is shared between: Module 'flink-storm-compatibility-examples' production,
> Module 'flink-storm-compatibility-examples (1)' production
> Please configure separate output paths to proceed with the compilation.
> TIP: you can use Project Artifacts to combine compiled classes if needed.
>
>
> What is the most seamless way to
> (1) pull master branch + build --- >  (2) create new branch ---> (3) make
> pull request ---> (4) update master ---> (5) create new branch ----> repeat
>
> I was able to do steps (1)-(3), but got an error on step (4).  The only
> solution I can think of is a hack:  import a whole new project into
> IntelliJ using the most up-to-date version of master, but that sounds silly
> to do that each time I want to repeat step (4)
>
> Thanks!
> Dan
>
>
>
>
>
>

[Stephan Ewen]

I have seen something like that before in IntelliJ as well. I think under
some circumstances the IntelliJ project settings and caches can be come
corrupt. I tried the following:

  - Use the option "Restart and clear caches"
  - Use maven->reimport project

That did the trick for me. Hope it works for you as well.

On Mon, Nov 2, 2015 at 2:26 PM, Daniel Blazevski <[hidden email]
> wrote:

> P.S. minor typo, a slightly better chart than
>
> (1) pull master branch + build --- >  (2) create new branch ---> (3) make
> pull request ---> (4) update master ---> (5) create new branch ----> repeat
>
> is
>
> (1) clone master branch + build --- >  (2) create new branch ---> (3) make
> pull request ---> (4) update master ---> (5) create new branch ----> (6)
> make pull request ---> repeat (4)-(6)
>
>
>
> On Mon, Nov 2, 2015 at 5:21 PM, Daniel Blazevski <
> [hidden email]
> > wrote:
>
> > Hello,
> >
> > I recently made a pull request for an exact knn algorithm, and have been
> > considering to start on the approximate knn algorithm and had an issue
> with
> > updating the master branch of Flink.
> >
> > I am curious to know what best practices are in terms of keeping up to
> > date with the master branch to avoid the issue I had (to be explained
> below)
> >
> > Some background:
> > -- I forked another Flink commiter's branch of Flink since he had done
> > other work on knn (namely https://github.com/chiwanpark/flink)
> > -- I imported that master branch into IntelliJ
> > -- I wrote code on a separate branch FLINK-1745, and that built fine.
> > -- I made the mistake in not using Travis CI from the start, but that was
> > not a huge issue since only a single import needed to be changed and my
> > Travis CI build goes through now.
> >
> > I then checked out the master branch, and updated master via
> > `git pull apache master`
> >
> > The master branch no longer builds, I get the following error in
> IngelliJ:
> >
> > Error:scalac: Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/classes
> > is shared between: Module 'flink-storm-compatibility-core' production,
> > Module 'flink-storm-compatibility-core (1)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/classes
> > is shared between: Module 'flink-connector-kafka' production, Module
> > 'flink-connector-kafka (1)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/classes
> > is shared between: Module 'flink-spargel' production, Module
> 'flink-spargel
> > (2)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/classes
> > is shared between: Module 'flink-streaming (2)' production, Module
> > 'flink-streaming-parent' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/test-classes
> > is shared between: Module 'flink-storm-compatibility-examples' tests,
> > Module 'flink-storm-compatibility-examples (1)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/test-classes
> > is shared between: Module 'flink-connector-rabbitmq' tests, Module
> > 'flink-connector-rabbitmq (2)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/classes
> > is shared between: Module 'flink-connector-elasticsearch' production,
> > Module 'flink-connector-elasticsearch (2)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/classes
> > is shared between: Module 'flink-streaming-connectors (1)' production,
> > Module 'flink-streaming-connectors-parent' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/test-classes
> > is shared between: Module 'flink-connector-elasticsearch' tests, Module
> > 'flink-connector-elasticsearch (2)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/test-classes
> > is shared between: Module 'flink-storm-compatibility-core' tests, Module
> > 'flink-storm-compatibility-core (1)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/test-classes
> > is shared between: Module 'flink-streaming-scala' tests, Module
> > 'flink-streaming-scala (1)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/test-classes
> > is shared between: Module 'flink-storm-compatibility (1)' tests, Module
> > 'flink-storm-compatibility-parent' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/test-classes
> > is shared between: Module 'flink-gelly' tests, Module 'flink-gelly (4)'
> > tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/classes
> > is shared between: Module 'flink-connector-rabbitmq' production, Module
> > 'flink-connector-rabbitmq (2)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/test-classes
> > is shared between: Module 'flink-spargel' tests, Module 'flink-spargel
> (2)'
> > tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/test-classes
> > is shared between: Module 'flink-connector-twitter' tests, Module
> > 'flink-connector-twitter (2)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/classes
> > is shared between: Module 'flink-connector-twitter' production, Module
> > 'flink-connector-twitter (2)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/classes
> > is shared between: Module 'flink-gelly' production, Module 'flink-gelly
> > (4)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/test-classes
> > is shared between: Module 'flink-connector-kafka' tests, Module
> > 'flink-connector-kafka (1)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/test-classes
> > is shared between: Module 'flink-streaming-connectors (1)' tests, Module
> > 'flink-streaming-connectors-parent' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/test-classes
> > is shared between: Module 'flink-connector-flume' tests, Module
> > 'flink-connector-flume (1)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/classes
> > is shared between: Module 'flink-streaming-examples' production, Module
> > 'flink-streaming-examples (1)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/test-classes
> > is shared between: Module 'flink-streaming-core' tests, Module
> > 'flink-streaming-core (1)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/test-classes
> > is shared between: Module 'flink-streaming-examples' tests, Module
> > 'flink-streaming-examples (1)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/classes
> > is shared between: Module 'flink-storm-compatibility (1)' production,
> > Module 'flink-storm-compatibility-parent' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/classes
> > is shared between: Module 'flink-streaming-core' production, Module
> > 'flink-streaming-core (1)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/test-classes
> > is shared between: Module 'flink-streaming (2)' tests, Module
> > 'flink-streaming-parent' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/classes
> > is shared between: Module 'flink-gelly-scala' production, Module
> > 'flink-gelly-scala (4)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/classes
> > is shared between: Module 'flink-connector-flume' production, Module
> > 'flink-connector-flume (1)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/test-classes
> > is shared between: Module 'flink-gelly-scala' tests, Module
> > 'flink-gelly-scala (4)' tests
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/classes
> > is shared between: Module 'flink-streaming-scala' production, Module
> > 'flink-streaming-scala (1)' production
> > Output path
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/classes
> > is shared between: Module 'flink-storm-compatibility-examples'
> production,
> > Module 'flink-storm-compatibility-examples (1)' production
> > Please configure separate output paths to proceed with the compilation.
> > TIP: you can use Project Artifacts to combine compiled classes if needed.
> >
> >
> > What is the most seamless way to
> > (1) pull master branch + build --- >  (2) create new branch ---> (3) make
> > pull request ---> (4) update master ---> (5) create new branch ---->
> repeat
> >
> > I was able to do steps (1)-(3), but got an error on step (4).  The only
> > solution I can think of is a hack:  import a whole new project into
> > IntelliJ using the most up-to-date version of master, but that sounds
> silly
> > to do that each time I want to repeat step (4)
> >
> > Thanks!
> > Dan
> >
> >
> >
> >
> >
> >
>

[Sachin Goel]

I've observed this sometimes too. Command line build succeeds, however
IntelliJ build fails.
The method suggested by Stephan doesn't work for me usually. However,
recreating the project from scratch takes at most 30-40 seconds, so I do
just that. :)

-- Sachin Goel
Computer Science, IIT Delhi
m. [phone]

On Tue, Nov 3, 2015 at 4:20 AM, Stephan Ewen <[hidden email]> wrote:

> I have seen something like that before in IntelliJ as well. I think under
> some circumstances the IntelliJ project settings and caches can be come
> corrupt. I tried the following:
>
>   - Use the option "Restart and clear caches"
>   - Use maven->reimport project
>
> That did the trick for me. Hope it works for you as well.
>
> On Mon, Nov 2, 2015 at 2:26 PM, Daniel Blazevski <
> [hidden email]
> > wrote:
>
> > P.S. minor typo, a slightly better chart than
> >
> > (1) pull master branch + build --- >  (2) create new branch ---> (3) make
> > pull request ---> (4) update master ---> (5) create new branch ---->
> repeat
> >
> > is
> >
> > (1) clone master branch + build --- >  (2) create new branch ---> (3)
> make
> > pull request ---> (4) update master ---> (5) create new branch ----> (6)
> > make pull request ---> repeat (4)-(6)
> >
> >
> >
> > On Mon, Nov 2, 2015 at 5:21 PM, Daniel Blazevski <
> > [hidden email]
> > > wrote:
> >
> > > Hello,
> > >
> > > I recently made a pull request for an exact knn algorithm, and have
> been
> > > considering to start on the approximate knn algorithm and had an issue
> > with
> > > updating the master branch of Flink.
> > >
> > > I am curious to know what best practices are in terms of keeping up to
> > > date with the master branch to avoid the issue I had (to be explained
> > below)
> > >
> > > Some background:
> > > -- I forked another Flink commiter's branch of Flink since he had done
> > > other work on knn (namely https://github.com/chiwanpark/flink)
> > > -- I imported that master branch into IntelliJ
> > > -- I wrote code on a separate branch FLINK-1745, and that built fine.
> > > -- I made the mistake in not using Travis CI from the start, but that
> was
> > > not a huge issue since only a single import needed to be changed and my
> > > Travis CI build goes through now.
> > >
> > > I then checked out the master branch, and updated master via
> > > `git pull apache master`
> > >
> > > The master branch no longer builds, I get the following error in
> > IngelliJ:
> > >
> > > Error:scalac: Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/classes
> > > is shared between: Module 'flink-storm-compatibility-core' production,
> > > Module 'flink-storm-compatibility-core (1)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/classes
> > > is shared between: Module 'flink-connector-kafka' production, Module
> > > 'flink-connector-kafka (1)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/classes
> > > is shared between: Module 'flink-spargel' production, Module
> > 'flink-spargel
> > > (2)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/classes
> > > is shared between: Module 'flink-streaming (2)' production, Module
> > > 'flink-streaming-parent' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/test-classes
> > > is shared between: Module 'flink-storm-compatibility-examples' tests,
> > > Module 'flink-storm-compatibility-examples (1)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/test-classes
> > > is shared between: Module 'flink-connector-rabbitmq' tests, Module
> > > 'flink-connector-rabbitmq (2)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/classes
> > > is shared between: Module 'flink-connector-elasticsearch' production,
> > > Module 'flink-connector-elasticsearch (2)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/classes
> > > is shared between: Module 'flink-streaming-connectors (1)' production,
> > > Module 'flink-streaming-connectors-parent' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/test-classes
> > > is shared between: Module 'flink-connector-elasticsearch' tests, Module
> > > 'flink-connector-elasticsearch (2)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/test-classes
> > > is shared between: Module 'flink-storm-compatibility-core' tests,
> Module
> > > 'flink-storm-compatibility-core (1)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/test-classes
> > > is shared between: Module 'flink-streaming-scala' tests, Module
> > > 'flink-streaming-scala (1)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/test-classes
> > > is shared between: Module 'flink-storm-compatibility (1)' tests, Module
> > > 'flink-storm-compatibility-parent' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/test-classes
> > > is shared between: Module 'flink-gelly' tests, Module 'flink-gelly (4)'
> > > tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/classes
> > > is shared between: Module 'flink-connector-rabbitmq' production, Module
> > > 'flink-connector-rabbitmq (2)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/test-classes
> > > is shared between: Module 'flink-spargel' tests, Module 'flink-spargel
> > (2)'
> > > tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/test-classes
> > > is shared between: Module 'flink-connector-twitter' tests, Module
> > > 'flink-connector-twitter (2)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/classes
> > > is shared between: Module 'flink-connector-twitter' production, Module
> > > 'flink-connector-twitter (2)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/classes
> > > is shared between: Module 'flink-gelly' production, Module 'flink-gelly
> > > (4)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/test-classes
> > > is shared between: Module 'flink-connector-kafka' tests, Module
> > > 'flink-connector-kafka (1)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/test-classes
> > > is shared between: Module 'flink-streaming-connectors (1)' tests,
> Module
> > > 'flink-streaming-connectors-parent' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/test-classes
> > > is shared between: Module 'flink-connector-flume' tests, Module
> > > 'flink-connector-flume (1)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/classes
> > > is shared between: Module 'flink-streaming-examples' production, Module
> > > 'flink-streaming-examples (1)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/test-classes
> > > is shared between: Module 'flink-streaming-core' tests, Module
> > > 'flink-streaming-core (1)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/test-classes
> > > is shared between: Module 'flink-streaming-examples' tests, Module
> > > 'flink-streaming-examples (1)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/classes
> > > is shared between: Module 'flink-storm-compatibility (1)' production,
> > > Module 'flink-storm-compatibility-parent' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/classes
> > > is shared between: Module 'flink-streaming-core' production, Module
> > > 'flink-streaming-core (1)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/test-classes
> > > is shared between: Module 'flink-streaming (2)' tests, Module
> > > 'flink-streaming-parent' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/classes
> > > is shared between: Module 'flink-gelly-scala' production, Module
> > > 'flink-gelly-scala (4)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/classes
> > > is shared between: Module 'flink-connector-flume' production, Module
> > > 'flink-connector-flume (1)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/test-classes
> > > is shared between: Module 'flink-gelly-scala' tests, Module
> > > 'flink-gelly-scala (4)' tests
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/classes
> > > is shared between: Module 'flink-streaming-scala' production, Module
> > > 'flink-streaming-scala (1)' production
> > > Output path
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/classes
> > > is shared between: Module 'flink-storm-compatibility-examples'
> > production,
> > > Module 'flink-storm-compatibility-examples (1)' production
> > > Please configure separate output paths to proceed with the compilation.
> > > TIP: you can use Project Artifacts to combine compiled classes if
> needed.
> > >
> > >
> > > What is the most seamless way to
> > > (1) pull master branch + build --- >  (2) create new branch ---> (3)
> make
> > > pull request ---> (4) update master ---> (5) create new branch ---->
> > repeat
> > >
> > > I was able to do steps (1)-(3), but got an error on step (4).  The only
> > > solution I can think of is a hack:  import a whole new project into
> > > IntelliJ using the most up-to-date version of master, but that sounds
> > silly
> > > to do that each time I want to repeat step (4)
> > >
> > > Thanks!
> > > Dan
> > >
> > >
> > >
> > >
> > >
> > >
> >
>

[Daniel Blazevski]

Thanks for the responses.

In IntelliJ, I tried:
File-->Invalidate Cahces / Restart --> Just Restart
and was able build the updated master branch, and the old FLINK-1745 branch.

Cheers,
Dan


On Mon, Nov 2, 2015 at 6:04 PM, Sachin Goel <[hidden email]>
wrote:

> I've observed this sometimes too. Command line build succeeds, however
> IntelliJ build fails.
> The method suggested by Stephan doesn't work for me usually. However,
> recreating the project from scratch takes at most 30-40 seconds, so I do
> just that. :)
>
> -- Sachin Goel
> Computer Science, IIT Delhi
> m. [phone]
>
> On Tue, Nov 3, 2015 at 4:20 AM, Stephan Ewen <[hidden email]> wrote:
>
> > I have seen something like that before in IntelliJ as well. I think under
> > some circumstances the IntelliJ project settings and caches can be come
> > corrupt. I tried the following:
> >
> >   - Use the option "Restart and clear caches"
> >   - Use maven->reimport project
> >
> > That did the trick for me. Hope it works for you as well.
> >
> > On Mon, Nov 2, 2015 at 2:26 PM, Daniel Blazevski <
> > [hidden email]
> > > wrote:
> >
> > > P.S. minor typo, a slightly better chart than
> > >
> > > (1) pull master branch + build --- >  (2) create new branch ---> (3)
> make
> > > pull request ---> (4) update master ---> (5) create new branch ---->
> > repeat
> > >
> > > is
> > >
> > > (1) clone master branch + build --- >  (2) create new branch ---> (3)
> > make
> > > pull request ---> (4) update master ---> (5) create new branch ---->
> (6)
> > > make pull request ---> repeat (4)-(6)
> > >
> > >
> > >
> > > On Mon, Nov 2, 2015 at 5:21 PM, Daniel Blazevski <
> > > [hidden email]
> > > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I recently made a pull request for an exact knn algorithm, and have
> > been
> > > > considering to start on the approximate knn algorithm and had an
> issue
> > > with
> > > > updating the master branch of Flink.
> > > >
> > > > I am curious to know what best practices are in terms of keeping up
> to
> > > > date with the master branch to avoid the issue I had (to be explained
> > > below)
> > > >
> > > > Some background:
> > > > -- I forked another Flink commiter's branch of Flink since he had
> done
> > > > other work on knn (namely https://github.com/chiwanpark/flink)
> > > > -- I imported that master branch into IntelliJ
> > > > -- I wrote code on a separate branch FLINK-1745, and that built fine.
> > > > -- I made the mistake in not using Travis CI from the start, but that
> > was
> > > > not a huge issue since only a single import needed to be changed and
> my
> > > > Travis CI build goes through now.
> > > >
> > > > I then checked out the master branch, and updated master via
> > > > `git pull apache master`
> > > >
> > > > The master branch no longer builds, I get the following error in
> > > IngelliJ:
> > > >
> > > > Error:scalac: Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/classes
> > > > is shared between: Module 'flink-storm-compatibility-core'
> production,
> > > > Module 'flink-storm-compatibility-core (1)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/classes
> > > > is shared between: Module 'flink-connector-kafka' production, Module
> > > > 'flink-connector-kafka (1)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/classes
> > > > is shared between: Module 'flink-spargel' production, Module
> > > 'flink-spargel
> > > > (2)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/classes
> > > > is shared between: Module 'flink-streaming (2)' production, Module
> > > > 'flink-streaming-parent' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/test-classes
> > > > is shared between: Module 'flink-storm-compatibility-examples' tests,
> > > > Module 'flink-storm-compatibility-examples (1)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/test-classes
> > > > is shared between: Module 'flink-connector-rabbitmq' tests, Module
> > > > 'flink-connector-rabbitmq (2)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/classes
> > > > is shared between: Module 'flink-connector-elasticsearch' production,
> > > > Module 'flink-connector-elasticsearch (2)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/classes
> > > > is shared between: Module 'flink-streaming-connectors (1)'
> production,
> > > > Module 'flink-streaming-connectors-parent' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/test-classes
> > > > is shared between: Module 'flink-connector-elasticsearch' tests,
> Module
> > > > 'flink-connector-elasticsearch (2)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/test-classes
> > > > is shared between: Module 'flink-storm-compatibility-core' tests,
> > Module
> > > > 'flink-storm-compatibility-core (1)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/test-classes
> > > > is shared between: Module 'flink-streaming-scala' tests, Module
> > > > 'flink-streaming-scala (1)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/test-classes
> > > > is shared between: Module 'flink-storm-compatibility (1)' tests,
> Module
> > > > 'flink-storm-compatibility-parent' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/test-classes
> > > > is shared between: Module 'flink-gelly' tests, Module 'flink-gelly
> (4)'
> > > > tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/classes
> > > > is shared between: Module 'flink-connector-rabbitmq' production,
> Module
> > > > 'flink-connector-rabbitmq (2)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/test-classes
> > > > is shared between: Module 'flink-spargel' tests, Module
> 'flink-spargel
> > > (2)'
> > > > tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/test-classes
> > > > is shared between: Module 'flink-connector-twitter' tests, Module
> > > > 'flink-connector-twitter (2)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/classes
> > > > is shared between: Module 'flink-connector-twitter' production,
> Module
> > > > 'flink-connector-twitter (2)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/classes
> > > > is shared between: Module 'flink-gelly' production, Module
> 'flink-gelly
> > > > (4)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/test-classes
> > > > is shared between: Module 'flink-connector-kafka' tests, Module
> > > > 'flink-connector-kafka (1)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/test-classes
> > > > is shared between: Module 'flink-streaming-connectors (1)' tests,
> > Module
> > > > 'flink-streaming-connectors-parent' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/test-classes
> > > > is shared between: Module 'flink-connector-flume' tests, Module
> > > > 'flink-connector-flume (1)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/classes
> > > > is shared between: Module 'flink-streaming-examples' production,
> Module
> > > > 'flink-streaming-examples (1)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/test-classes
> > > > is shared between: Module 'flink-streaming-core' tests, Module
> > > > 'flink-streaming-core (1)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/test-classes
> > > > is shared between: Module 'flink-streaming-examples' tests, Module
> > > > 'flink-streaming-examples (1)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/classes
> > > > is shared between: Module 'flink-storm-compatibility (1)' production,
> > > > Module 'flink-storm-compatibility-parent' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/classes
> > > > is shared between: Module 'flink-streaming-core' production, Module
> > > > 'flink-streaming-core (1)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/test-classes
> > > > is shared between: Module 'flink-streaming (2)' tests, Module
> > > > 'flink-streaming-parent' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/classes
> > > > is shared between: Module 'flink-gelly-scala' production, Module
> > > > 'flink-gelly-scala (4)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/classes
> > > > is shared between: Module 'flink-connector-flume' production, Module
> > > > 'flink-connector-flume (1)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/test-classes
> > > > is shared between: Module 'flink-gelly-scala' tests, Module
> > > > 'flink-gelly-scala (4)' tests
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/classes
> > > > is shared between: Module 'flink-streaming-scala' production, Module
> > > > 'flink-streaming-scala (1)' production
> > > > Output path
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/classes
> > > > is shared between: Module 'flink-storm-compatibility-examples'
> > > production,
> > > > Module 'flink-storm-compatibility-examples (1)' production
> > > > Please configure separate output paths to proceed with the
> compilation.
> > > > TIP: you can use Project Artifacts to combine compiled classes if
> > needed.
> > > >
> > > >
> > > > What is the most seamless way to
> > > > (1) pull master branch + build --- >  (2) create new branch ---> (3)
> > make
> > > > pull request ---> (4) update master ---> (5) create new branch ---->
> > > repeat
> > > >
> > > > I was able to do steps (1)-(3), but got an error on step (4).  The
> only
> > > > solution I can think of is a hack:  import a whole new project into
> > > > IntelliJ using the most up-to-date version of master, but that sounds
> > > silly
> > > > to do that each time I want to repeat step (4)
> > > >
> > > > Thanks!
> > > > Dan
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
>

[Stephan Ewen]

Good to hear!

On Mon, Nov 2, 2015 at 3:38 PM, Daniel Blazevski <[hidden email]
> wrote:

> Thanks for the responses.
>
> In IntelliJ, I tried:
> File-->Invalidate Cahces / Restart --> Just Restart
> and was able build the updated master branch, and the old FLINK-1745
> branch.
>
> Cheers,
> Dan
>
>
> On Mon, Nov 2, 2015 at 6:04 PM, Sachin Goel <[hidden email]>
> wrote:
>
> > I've observed this sometimes too. Command line build succeeds, however
> > IntelliJ build fails.
> > The method suggested by Stephan doesn't work for me usually. However,
> > recreating the project from scratch takes at most 30-40 seconds, so I do
> > just that. :)
> >
> > -- Sachin Goel
> > Computer Science, IIT Delhi
> > m. [phone]
> >
> > On Tue, Nov 3, 2015 at 4:20 AM, Stephan Ewen <[hidden email]> wrote:
> >
> > > I have seen something like that before in IntelliJ as well. I think
> under
> > > some circumstances the IntelliJ project settings and caches can be come
> > > corrupt. I tried the following:
> > >
> > >   - Use the option "Restart and clear caches"
> > >   - Use maven->reimport project
> > >
> > > That did the trick for me. Hope it works for you as well.
> > >
> > > On Mon, Nov 2, 2015 at 2:26 PM, Daniel Blazevski <
> > > [hidden email]
> > > > wrote:
> > >
> > > > P.S. minor typo, a slightly better chart than
> > > >
> > > > (1) pull master branch + build --- >  (2) create new branch ---> (3)
> > make
> > > > pull request ---> (4) update master ---> (5) create new branch ---->
> > > repeat
> > > >
> > > > is
> > > >
> > > > (1) clone master branch + build --- >  (2) create new branch ---> (3)
> > > make
> > > > pull request ---> (4) update master ---> (5) create new branch ---->
> > (6)
> > > > make pull request ---> repeat (4)-(6)
> > > >
> > > >
> > > >
> > > > On Mon, Nov 2, 2015 at 5:21 PM, Daniel Blazevski <
> > > > [hidden email]
> > > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I recently made a pull request for an exact knn algorithm, and have
> > > been
> > > > > considering to start on the approximate knn algorithm and had an
> > issue
> > > > with
> > > > > updating the master branch of Flink.
> > > > >
> > > > > I am curious to know what best practices are in terms of keeping up
> > to
> > > > > date with the master branch to avoid the issue I had (to be
> explained
> > > > below)
> > > > >
> > > > > Some background:
> > > > > -- I forked another Flink commiter's branch of Flink since he had
> > done
> > > > > other work on knn (namely https://github.com/chiwanpark/flink)
> > > > > -- I imported that master branch into IntelliJ
> > > > > -- I wrote code on a separate branch FLINK-1745, and that built
> fine.
> > > > > -- I made the mistake in not using Travis CI from the start, but
> that
> > > was
> > > > > not a huge issue since only a single import needed to be changed
> and
> > my
> > > > > Travis CI build goes through now.
> > > > >
> > > > > I then checked out the master branch, and updated master via
> > > > > `git pull apache master`
> > > > >
> > > > > The master branch no longer builds, I get the following error in
> > > > IngelliJ:
> > > > >
> > > > > Error:scalac: Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/classes
> > > > > is shared between: Module 'flink-storm-compatibility-core'
> > production,
> > > > > Module 'flink-storm-compatibility-core (1)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/classes
> > > > > is shared between: Module 'flink-connector-kafka' production,
> Module
> > > > > 'flink-connector-kafka (1)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/classes
> > > > > is shared between: Module 'flink-spargel' production, Module
> > > > 'flink-spargel
> > > > > (2)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/classes
> > > > > is shared between: Module 'flink-streaming (2)' production, Module
> > > > > 'flink-streaming-parent' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/test-classes
> > > > > is shared between: Module 'flink-storm-compatibility-examples'
> tests,
> > > > > Module 'flink-storm-compatibility-examples (1)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/test-classes
> > > > > is shared between: Module 'flink-connector-rabbitmq' tests, Module
> > > > > 'flink-connector-rabbitmq (2)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/classes
> > > > > is shared between: Module 'flink-connector-elasticsearch'
> production,
> > > > > Module 'flink-connector-elasticsearch (2)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/classes
> > > > > is shared between: Module 'flink-streaming-connectors (1)'
> > production,
> > > > > Module 'flink-streaming-connectors-parent' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-elasticsearch/target/test-classes
> > > > > is shared between: Module 'flink-connector-elasticsearch' tests,
> > Module
> > > > > 'flink-connector-elasticsearch (2)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-core/target/test-classes
> > > > > is shared between: Module 'flink-storm-compatibility-core' tests,
> > > Module
> > > > > 'flink-storm-compatibility-core (1)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/test-classes
> > > > > is shared between: Module 'flink-streaming-scala' tests, Module
> > > > > 'flink-streaming-scala (1)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/test-classes
> > > > > is shared between: Module 'flink-storm-compatibility (1)' tests,
> > Module
> > > > > 'flink-storm-compatibility-parent' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/test-classes
> > > > > is shared between: Module 'flink-gelly' tests, Module 'flink-gelly
> > (4)'
> > > > > tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-rabbitmq/target/classes
> > > > > is shared between: Module 'flink-connector-rabbitmq' production,
> > Module
> > > > > 'flink-connector-rabbitmq (2)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-spargel/target/test-classes
> > > > > is shared between: Module 'flink-spargel' tests, Module
> > 'flink-spargel
> > > > (2)'
> > > > > tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/test-classes
> > > > > is shared between: Module 'flink-connector-twitter' tests, Module
> > > > > 'flink-connector-twitter (2)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-twitter/target/classes
> > > > > is shared between: Module 'flink-connector-twitter' production,
> > Module
> > > > > 'flink-connector-twitter (2)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly/target/classes
> > > > > is shared between: Module 'flink-gelly' production, Module
> > 'flink-gelly
> > > > > (4)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-kafka/target/test-classes
> > > > > is shared between: Module 'flink-connector-kafka' tests, Module
> > > > > 'flink-connector-kafka (1)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/target/test-classes
> > > > > is shared between: Module 'flink-streaming-connectors (1)' tests,
> > > Module
> > > > > 'flink-streaming-connectors-parent' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/test-classes
> > > > > is shared between: Module 'flink-connector-flume' tests, Module
> > > > > 'flink-connector-flume (1)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/classes
> > > > > is shared between: Module 'flink-streaming-examples' production,
> > Module
> > > > > 'flink-streaming-examples (1)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/test-classes
> > > > > is shared between: Module 'flink-streaming-core' tests, Module
> > > > > 'flink-streaming-core (1)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-examples/target/test-classes
> > > > > is shared between: Module 'flink-streaming-examples' tests, Module
> > > > > 'flink-streaming-examples (1)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/target/classes
> > > > > is shared between: Module 'flink-storm-compatibility (1)'
> production,
> > > > > Module 'flink-storm-compatibility-parent' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-core/target/classes
> > > > > is shared between: Module 'flink-streaming-core' production, Module
> > > > > 'flink-streaming-core (1)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/target/test-classes
> > > > > is shared between: Module 'flink-streaming (2)' tests, Module
> > > > > 'flink-streaming-parent' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/classes
> > > > > is shared between: Module 'flink-gelly-scala' production, Module
> > > > > 'flink-gelly-scala (4)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-connectors/flink-connector-flume/target/classes
> > > > > is shared between: Module 'flink-connector-flume' production,
> Module
> > > > > 'flink-connector-flume (1)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-gelly-scala/target/test-classes
> > > > > is shared between: Module 'flink-gelly-scala' tests, Module
> > > > > 'flink-gelly-scala (4)' tests
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-staging/flink-streaming/flink-streaming-scala/target/classes
> > > > > is shared between: Module 'flink-streaming-scala' production,
> Module
> > > > > 'flink-streaming-scala (1)' production
> > > > > Output path
> > > > >
> > > >
> > >
> >
> /Users/danielblazevski/Dropbox/code/flink-chiwan/flink/flink-contrib/flink-storm-compatibility/flink-storm-compatibility-examples/target/classes
> > > > > is shared between: Module 'flink-storm-compatibility-examples'
> > > > production,
> > > > > Module 'flink-storm-compatibility-examples (1)' production
> > > > > Please configure separate output paths to proceed with the
> > compilation.
> > > > > TIP: you can use Project Artifacts to combine compiled classes if
> > > needed.
> > > > >
> > > > >
> > > > > What is the most seamless way to
> > > > > (1) pull master branch + build --- >  (2) create new branch --->
> (3)
> > > make
> > > > > pull request ---> (4) update master ---> (5) create new branch
> ---->
> > > > repeat
> > > > >
> > > > > I was able to do steps (1)-(3), but got an error on step (4).  The
> > only
> > > > > solution I can think of is a hack:  import a whole new project into
> > > > > IntelliJ using the most up-to-date version of master, but that
> sounds
> > > > silly
> > > > > to do that each time I want to repeat step (4)
> > > > >
> > > > > Thanks!
> > > > > Dan
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

[André Petermann]

Hi all,

We are working on an implementation of Frequent Subgraph Mining using
Flink. At that, the "too few memory segments error" prevents the most
promising solution. The problem is not specific for graphs, but all
iterative problems where

- working and solution sets contain data of different types
- the working set may grow, shrink or is replaced for each iteration
- the solution set grows for each iteration
- the termination criterion is based on data set metrics,
   e.g. while working set not empty

An illustration of our problem workflow, generalized to graph unspecific
frequent pattern mining, can be found here:
https://github.com/dbs-leipzig/gradoop/blob/master/dev-support/loopWithIntermediateResultMemory.pdf

Page 1 shows the most promising solution. We started implementing it
using a for loop. However, the "too few memory segments error" makes it
untestable. As the iteration body itself is a complex workflow and the
number of iterations is arbitrary, unrolling it while reserving operator
memory will be a permanent limitation. Increasing limits or physical
memory would only delay the problem. The resulting question is:

Would it be possible to implement a while-not-empty or at least a for
loop, that isn't unrolled and can be executed more memory efficient?

Page 2 shows an alternative solution to our problem using the concept of
delta iteration. However, Flink delta iteration does neither support
broadcasting nor working-set independent intermediate results. Page 3
shows our working solution using two workarounds for those restrictions.
However, these workarounds lead to unnecessary memory consumption and
redundant expensive computations. So, in the case the answer to the
first question is no, a second question:

Would it be possible to extend the delta iteration by the support of
rich map functions with broadcast sets and the memory of intermediate
results?

We think, that a while-not-empty loop might be useful for other
algorithms too, e.g. variable length path search in graphs. Did we miss
Flink features meeting our requirements? Do you think it's worth to
create an improvement issue? At that, we'd of course be willing to
contribute in the form of development.

Best Regards
Andre