Commits for next release

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Commits for next release

Stephan Ewen
Hi all!

Please find below a list of commits that I would propose to include in the
bugfix release.

It does not contain any streaming commits. Marton, Gyula, can you add that?

Thing is: it contains 90% of all commits. I put some in parenthesis that
are not bug fixes, but minor additions that are useful also in a minor
release.

We might as well just take all commits and make it 0.8.0 ;-)

Greetings,
Stephan


------------------------------------------------------------------

770ce23  [FLINK-1301] Added Apache license headers to Markdown, HTML, SVG,
Pyt

979837a  [scala] Fix TupleSerializerTest for corrected CaseClassSerializer

3470b37  [scala] Fix non-reuse copy in CaseClassSerializer

a677c77  [FLINK-1157] Document TaskManager slots and minor fixes.

858d1bc  [Java API] [docs] Rephrase Javadoc of minBy/maxBy and link from
min/max

bc41197  [FLINK-1292] Allow for longer normalized keys when using composite
keys

33724f0  [Docs] Fix rendering issue and typo

e3eaac2  [FLINK-984] Compiler tests for distinct().

84d6da8  [FLINK-1291] Remove default GC options from the shell scripts …

4e9c32f  [Docs] Just a typo in docs/index.md

a9b7cbe  [scala] [runtime] Add extra tests for sorting of case classes

45fb6d8  [FLINK-1290] Fix Optimizer to create plans when encountering
incompat

d554faa  [FLINK-1273] [runtime] Add Void type to basic types

17bc479  [FLINK-1095] [clients] Remove '-d' option in info from CliFrontend

bbe54ac  [FLINK-1270] [APIs] FS.get() supports relative paths

3d242fd  Removed TypeSerializerFactory from ScalaCsvInputFormat. The
TypeSeria… …

1a7a50f  [scala] Add equals() to TraversableTypeInfo

1dd8b8d  Rewrite TPC-H Q3 to use Long instead of Integer …

1a91132  [scala] Fix slow building of adjacency list in Scala PageRank

f66892d  [runtime] CaseClassSerializer correctly treated as stateful

2ae366b  [docs] Fix paths to work with local builds

(  e8e7abb  [scala] Add ClosureCleaner for Scala Lambdas  )

74bc7dd  [FLINK-1282] [docs] Make Flink logo link back to frontpage

8a8155e  [FLINK-1282] [docs] Added {{ site.baseurl }} to css and image paths

112b3a9  [Scala API] Case Class serializer can work with classes that
cannot b… …

48b6d01  [FLINK-1278] [runtime] (part 1) Remove special code paths for the
Rec… …

d858930  [FLINK-1286] [APIs] [runtime] Fix serialization in
CollectionInputFor… …

82f5154  [FLINK-1282] [docs] Update layout of docs …

87497b7  [runtime] Quick fix for error with unsupported ship strategy

392683f  [FLINK-658] [APIs] Add group sorting to CoGroup

606a6d4  [Java API] Fix various warnings

8724bf7  Small changes and tests for the newly introduced EnumType

(   65d85f1  [FLINK-1221] Use StackTraceElement methods instead of String
parsing   )

c03237a   [FLINK-1221] Use only method name and location as default
operator names

(   bcdd167  [FLINK-1249] [APIs] [compiler] Add custom partitioner for
CoGroup   )

4838efe  Fixes FLINK-1276

42fe874  Minor code cleanups - left from previous patches

cf54a1c  [FLINK-1263] [optimizer] Implement compatibility checks for binary
operators

d7853fd  [APIs] Enhance test coverage for CollectionInputFormat and add
tests...

02e2857  Log memory usage stats at INFO level instead of DEBUG

a69d60a  Removed object reusage in GenericArraySerializer's deserialize
method.

98ff76b  [FLINK-820] [compiler] Support for disconnected data flows

d0f2db0  [FLINK-1264] [compiler] Properly forward custom partitioners to
the runtime

b3e5ed0  [Flink-1262] inconsistency between CsvReader.java and
TupleGenerator.

4a74f32  [FLINK-1265] Fix user classloader bug for registerInputOutput()
method

a77d752  Added binary input and output format which uses the objects'
TypeSeri… …

8af0473  [FLINK-1252] address pull request comments

8081ddc  [FLINK-1251] Enums are now handled properly by the collection
input format

591f16d  [FLINK-1252] Add support for serializing Date and Enums in the
Java API

4203bf9  Minor code clean up …

5c3dceb  [FLINK-1142] Log information about I/O manager temp dirs …

(   3ac4df8  [FLINK-993] Primitive input format fails for invalid input.   )

d640b6c  [FLINK-933] Add primitive input format to read a sequence of
primitives …

ae505ad  [FLINK-1253] [tests] Make sure tests do not die with garbage
collecti… …

ce822bf  [FLINK-1254] [compiler] Fix compiler bug for pipeline breaker
placement

54aa41b  Added getExecutionEnvironment to the DataSet of the Scala API.

da60630  Add byte array serialization to InstantiationUtil

f84e4ec  Minor code clean up to get rid of IntelliJ IDEA warnings …

1ecacf0  [FLINK-1247] [docs] Fix broken links in documentation

9f6a0b8  [FLINK-1250] [config] [docs] Correct and document config keys for
hea… …

b264221  [FLINK-1209] [compiler] Improve error messages when forgetting to
clo… …

6989cec  [bin] Remove MaxPermSize parameter for JVM 1.8 …

(   2000b45  [FLINK-1237] Add support for custom partitioners   )

83d0256  [FLINK-1207] Context environments are realized through factories …

ae07abe  [build] Manage version of joda-time to prevent conflicts between
depe… …

06c259f  [tests] Various stability fixes to tests

f76eb15  [FLINK-1246] Add additional debug output to flaky recovery test

2d1532f  Fix flaky test SumMinMaxITCase

9c1585e  [FLINK-1172] [docs] Fix broken links in documentation …

a296f40  [FLINK-1244] setCombinable() returns operator …

7f8296e  [FLINK-890] [docs] Adjust examples to show reader methods and
correct… …

42828f2  [FLINK-1008] Fix createProgramPlan() throws exception …

e23874c  [FLINK-1242] Fix streaming-examples and scala-examples POMs to
work p… …

922f478  [docs] Add description of Quasiquotes compiler plugin in Eclipse
to FAQ

8a955e5  [FLINK-1233] Fix flaky AggregateITCase

d7d7238  [FLINK-1239] [streaming] IterateExample fix and parallelism
setting i… …

c339c26  [FLINK-1218] Replace new Integer with Integer.valueOf in tests …

9ff2e5b  [scala] Self-contained build for scala examples …

8f9d8ea  Fix caching of receivers

c6dd9b1  [FLINK-1204] [streaming] Individual, self-contained packaging for
str… …

818ebda  [FLINK-1221] Use the source line as the default operator name …

b253cb2  [FLINK-1230] Add documentation and an example for collection-based
ex… …

b3c290f  [FLINK-1226] Add API support for passing Configurations to
InputFormats …

21b1b97  [FLINK-1235] Compiler accepts iterations referenced from the
static p… …

3772d30  Improved input type inference, bug fixing, code simplification …

57ecbcc  [FLINK-1233] Improve error message when not enough slots are
availabl… …

e855ef4  [FLINK-1223] Allow value escaping in CSV files …

2f1176a  [FLINK-983] Fix WebClient truncated function names …

(   c9cfe3b [FLINK-1323] Refactor I/O Manager Readers and Writers to
interfaces   )

8e4c772  [FLINK-1186] Fix wrong flat key position when using expression key
fo… …

65d0359  Make sure S3 directory paths are suffixed with '/' on creation

81c5b2a  [FLINK-1215] Increase robustness to spurious failures when
creating o… …

4b75d83  [FLINK-1213] Added .gitattributes to normalize EOL …

90730fe  [FLINK-1212] Fixed shell script interpreter directive …

1e979e9  [Flink-1113] clean up JQuery dependencies

e580497  [FLINK-1215] Fix spurious failures when creating output
directories

ef9a373  [runtime] In local mode, make sure taskmanagers have completed
regist

ef40691  [FLINK-1222] Tasks send close acknowledgements early.

a959dd5  [FLINK-1220] Promote various log statements to INFO level. Add log
st… …

17e0186  Add test case for TaskManager connection loss, leading to task
failure.

6ecd0f8  [FLINK-1214] Prevent partitionings on subsets of fields from being
pushed down

a586614  [FLINK-1005] By default, new objects are created for each element
in group reduce

c3835cd  [scala] Add union operation to Scala ExecutionEnvironment

e21d78d  [scala] Make getType public in Scala DataSet

bd66a08  [FLINK-1191] Add support for Scala Collections and Special Types

1e1df6d  [scala] Add macroparadise to enable Quasiquotes

3510bf9  Change TypeExtractor to support interfaces within classes

233161b  [FLINK-1210] Improve error message in delta iterations when the
next

9b6561d  [doc] Fix programming guide link in iterations.md

9b25942  Temporarily disable fork reuse in flink-tests Unit tests
(workaround

ca9e6ab  Make sure that operator names are properly escaped for display in
the web frontend

(   7d2db95  [FLINK-1198] Fix nullpointer exception in broadcast variable
material   )

(   e1fd689  [FLINK-1198] Broadcast variables can share initialization   )

(   d8052d4  [FLINK-1198] Make Broadcast variables shared per taskmanager,
rather   )

(   cc76653  [FLINK-1198] Broadcast variables are shared between tasks in
the same   )

bbd8209  [FLINK-1203] Use forked execution in all tests, except in the
end-to-end tests

a6152c3  [FLINK-1205] Fix library cache manager to track references to
tasks a… …

5e48fc9  Make sure TaskManager does not accept task deployments when
shutting … …

8b39ba9  Improve error messages in case of invalid file paths or URIs …

(   f0fd882  Introduce a delay before restarts to make sure that
taskmanager failurs   )

(   dd687bc  Implement coarse-grained fault tolerance   )

2557832  Remove obsolete collection execution example.

a747b61  [FLINK-1202] Remove incomplete file outputs on failure …

f42dcc3 [FLINK-1203] Deactivate fork reuse in tests

f5898a0  Another round to remove final modifier to private methods because
it

0ee60d3  [FLINK-1181] Fix IOReadableWritable checks in RPC service …

b0c1af5  [FLINK-1182] Improve error messages for POJO type mismatches

(   eebe510  Add DeltaPageRank example   )

3f33278  [FLINK-1194] Fix Java quickstart archetype POM file

73ee385  Remove unnecessary final modifier in
PlanExecutor.loadExecutorClass m… …

f2b8650  [FLINK-1178] Changed type preference to ValueType, WritableType,
Case… …
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Robert Metzger
I'm voting to release the current master as Flink 0.8.0.

Once we have merged these two major changes (
https://github.com/apache/incubator-flink/pull/149,
https://github.com/apache/incubator-flink/pull/254) in the queue we will
need some time until we reach a well tested, stable state again.

On Mon, Dec 8, 2014 at 10:30 AM, Stephan Ewen <[hidden email]> wrote:

> Hi all!
>
> Please find below a list of commits that I would propose to include in the
> bugfix release.
>
> It does not contain any streaming commits. Marton, Gyula, can you add that?
>
> Thing is: it contains 90% of all commits. I put some in parenthesis that
> are not bug fixes, but minor additions that are useful also in a minor
> release.
>
> We might as well just take all commits and make it 0.8.0 ;-)
>
> Greetings,
> Stephan
>
>
> ------------------------------------------------------------------
>
> 770ce23  [FLINK-1301] Added Apache license headers to Markdown, HTML, SVG,
> Pyt
>
> 979837a  [scala] Fix TupleSerializerTest for corrected CaseClassSerializer
>
> 3470b37  [scala] Fix non-reuse copy in CaseClassSerializer
>
> a677c77  [FLINK-1157] Document TaskManager slots and minor fixes.
>
> 858d1bc  [Java API] [docs] Rephrase Javadoc of minBy/maxBy and link from
> min/max
>
> bc41197  [FLINK-1292] Allow for longer normalized keys when using composite
> keys
>
> 33724f0  [Docs] Fix rendering issue and typo
>
> e3eaac2  [FLINK-984] Compiler tests for distinct().
>
> 84d6da8  [FLINK-1291] Remove default GC options from the shell scripts …
>
> 4e9c32f  [Docs] Just a typo in docs/index.md
>
> a9b7cbe  [scala] [runtime] Add extra tests for sorting of case classes
>
> 45fb6d8  [FLINK-1290] Fix Optimizer to create plans when encountering
> incompat
>
> d554faa  [FLINK-1273] [runtime] Add Void type to basic types
>
> 17bc479  [FLINK-1095] [clients] Remove '-d' option in info from CliFrontend
>
> bbe54ac  [FLINK-1270] [APIs] FS.get() supports relative paths
>
> 3d242fd  Removed TypeSerializerFactory from ScalaCsvInputFormat. The
> TypeSeria… …
>
> 1a7a50f  [scala] Add equals() to TraversableTypeInfo
>
> 1dd8b8d  Rewrite TPC-H Q3 to use Long instead of Integer …
>
> 1a91132  [scala] Fix slow building of adjacency list in Scala PageRank
>
> f66892d  [runtime] CaseClassSerializer correctly treated as stateful
>
> 2ae366b  [docs] Fix paths to work with local builds
>
> (  e8e7abb  [scala] Add ClosureCleaner for Scala Lambdas  )
>
> 74bc7dd  [FLINK-1282] [docs] Make Flink logo link back to frontpage
>
> 8a8155e  [FLINK-1282] [docs] Added {{ site.baseurl }} to css and image
> paths
>
> 112b3a9  [Scala API] Case Class serializer can work with classes that
> cannot b… …
>
> 48b6d01  [FLINK-1278] [runtime] (part 1) Remove special code paths for the
> Rec… …
>
> d858930  [FLINK-1286] [APIs] [runtime] Fix serialization in
> CollectionInputFor… …
>
> 82f5154  [FLINK-1282] [docs] Update layout of docs …
>
> 87497b7  [runtime] Quick fix for error with unsupported ship strategy
>
> 392683f  [FLINK-658] [APIs] Add group sorting to CoGroup
>
> 606a6d4  [Java API] Fix various warnings
>
> 8724bf7  Small changes and tests for the newly introduced EnumType
>
> (   65d85f1  [FLINK-1221] Use StackTraceElement methods instead of String
> parsing   )
>
> c03237a   [FLINK-1221] Use only method name and location as default
> operator names
>
> (   bcdd167  [FLINK-1249] [APIs] [compiler] Add custom partitioner for
> CoGroup   )
>
> 4838efe  Fixes FLINK-1276
>
> 42fe874  Minor code cleanups - left from previous patches
>
> cf54a1c  [FLINK-1263] [optimizer] Implement compatibility checks for binary
> operators
>
> d7853fd  [APIs] Enhance test coverage for CollectionInputFormat and add
> tests...
>
> 02e2857  Log memory usage stats at INFO level instead of DEBUG
>
> a69d60a  Removed object reusage in GenericArraySerializer's deserialize
> method.
>
> 98ff76b  [FLINK-820] [compiler] Support for disconnected data flows
>
> d0f2db0  [FLINK-1264] [compiler] Properly forward custom partitioners to
> the runtime
>
> b3e5ed0  [Flink-1262] inconsistency between CsvReader.java and
> TupleGenerator.
>
> 4a74f32  [FLINK-1265] Fix user classloader bug for registerInputOutput()
> method
>
> a77d752  Added binary input and output format which uses the objects'
> TypeSeri… …
>
> 8af0473  [FLINK-1252] address pull request comments
>
> 8081ddc  [FLINK-1251] Enums are now handled properly by the collection
> input format
>
> 591f16d  [FLINK-1252] Add support for serializing Date and Enums in the
> Java API
>
> 4203bf9  Minor code clean up …
>
> 5c3dceb  [FLINK-1142] Log information about I/O manager temp dirs …
>
> (   3ac4df8  [FLINK-993] Primitive input format fails for invalid input.
>  )
>
> d640b6c  [FLINK-933] Add primitive input format to read a sequence of
> primitives …
>
> ae505ad  [FLINK-1253] [tests] Make sure tests do not die with garbage
> collecti… …
>
> ce822bf  [FLINK-1254] [compiler] Fix compiler bug for pipeline breaker
> placement
>
> 54aa41b  Added getExecutionEnvironment to the DataSet of the Scala API.
>
> da60630  Add byte array serialization to InstantiationUtil
>
> f84e4ec  Minor code clean up to get rid of IntelliJ IDEA warnings …
>
> 1ecacf0  [FLINK-1247] [docs] Fix broken links in documentation
>
> 9f6a0b8  [FLINK-1250] [config] [docs] Correct and document config keys for
> hea… …
>
> b264221  [FLINK-1209] [compiler] Improve error messages when forgetting to
> clo… …
>
> 6989cec  [bin] Remove MaxPermSize parameter for JVM 1.8 …
>
> (   2000b45  [FLINK-1237] Add support for custom partitioners   )
>
> 83d0256  [FLINK-1207] Context environments are realized through factories …
>
> ae07abe  [build] Manage version of joda-time to prevent conflicts between
> depe… …
>
> 06c259f  [tests] Various stability fixes to tests
>
> f76eb15  [FLINK-1246] Add additional debug output to flaky recovery test
>
> 2d1532f  Fix flaky test SumMinMaxITCase
>
> 9c1585e  [FLINK-1172] [docs] Fix broken links in documentation …
>
> a296f40  [FLINK-1244] setCombinable() returns operator …
>
> 7f8296e  [FLINK-890] [docs] Adjust examples to show reader methods and
> correct… …
>
> 42828f2  [FLINK-1008] Fix createProgramPlan() throws exception …
>
> e23874c  [FLINK-1242] Fix streaming-examples and scala-examples POMs to
> work p… …
>
> 922f478  [docs] Add description of Quasiquotes compiler plugin in Eclipse
> to FAQ
>
> 8a955e5  [FLINK-1233] Fix flaky AggregateITCase
>
> d7d7238  [FLINK-1239] [streaming] IterateExample fix and parallelism
> setting i… …
>
> c339c26  [FLINK-1218] Replace new Integer with Integer.valueOf in tests …
>
> 9ff2e5b  [scala] Self-contained build for scala examples …
>
> 8f9d8ea  Fix caching of receivers
>
> c6dd9b1  [FLINK-1204] [streaming] Individual, self-contained packaging for
> str… …
>
> 818ebda  [FLINK-1221] Use the source line as the default operator name …
>
> b253cb2  [FLINK-1230] Add documentation and an example for collection-based
> ex… …
>
> b3c290f  [FLINK-1226] Add API support for passing Configurations to
> InputFormats …
>
> 21b1b97  [FLINK-1235] Compiler accepts iterations referenced from the
> static p… …
>
> 3772d30  Improved input type inference, bug fixing, code simplification …
>
> 57ecbcc  [FLINK-1233] Improve error message when not enough slots are
> availabl… …
>
> e855ef4  [FLINK-1223] Allow value escaping in CSV files …
>
> 2f1176a  [FLINK-983] Fix WebClient truncated function names …
>
> (   c9cfe3b [FLINK-1323] Refactor I/O Manager Readers and Writers to
> interfaces   )
>
> 8e4c772  [FLINK-1186] Fix wrong flat key position when using expression key
> fo… …
>
> 65d0359  Make sure S3 directory paths are suffixed with '/' on creation
>
> 81c5b2a  [FLINK-1215] Increase robustness to spurious failures when
> creating o… …
>
> 4b75d83  [FLINK-1213] Added .gitattributes to normalize EOL …
>
> 90730fe  [FLINK-1212] Fixed shell script interpreter directive …
>
> 1e979e9  [Flink-1113] clean up JQuery dependencies
>
> e580497  [FLINK-1215] Fix spurious failures when creating output
> directories
>
> ef9a373  [runtime] In local mode, make sure taskmanagers have completed
> regist
>
> ef40691  [FLINK-1222] Tasks send close acknowledgements early.
>
> a959dd5  [FLINK-1220] Promote various log statements to INFO level. Add log
> st… …
>
> 17e0186  Add test case for TaskManager connection loss, leading to task
> failure.
>
> 6ecd0f8  [FLINK-1214] Prevent partitionings on subsets of fields from being
> pushed down
>
> a586614  [FLINK-1005] By default, new objects are created for each element
> in group reduce
>
> c3835cd  [scala] Add union operation to Scala ExecutionEnvironment
>
> e21d78d  [scala] Make getType public in Scala DataSet
>
> bd66a08  [FLINK-1191] Add support for Scala Collections and Special Types
>
> 1e1df6d  [scala] Add macroparadise to enable Quasiquotes
>
> 3510bf9  Change TypeExtractor to support interfaces within classes
>
> 233161b  [FLINK-1210] Improve error message in delta iterations when the
> next
>
> 9b6561d  [doc] Fix programming guide link in iterations.md
>
> 9b25942  Temporarily disable fork reuse in flink-tests Unit tests
> (workaround
>
> ca9e6ab  Make sure that operator names are properly escaped for display in
> the web frontend
>
> (   7d2db95  [FLINK-1198] Fix nullpointer exception in broadcast variable
> material   )
>
> (   e1fd689  [FLINK-1198] Broadcast variables can share initialization   )
>
> (   d8052d4  [FLINK-1198] Make Broadcast variables shared per taskmanager,
> rather   )
>
> (   cc76653  [FLINK-1198] Broadcast variables are shared between tasks in
> the same   )
>
> bbd8209  [FLINK-1203] Use forked execution in all tests, except in the
> end-to-end tests
>
> a6152c3  [FLINK-1205] Fix library cache manager to track references to
> tasks a… …
>
> 5e48fc9  Make sure TaskManager does not accept task deployments when
> shutting … …
>
> 8b39ba9  Improve error messages in case of invalid file paths or URIs …
>
> (   f0fd882  Introduce a delay before restarts to make sure that
> taskmanager failurs   )
>
> (   dd687bc  Implement coarse-grained fault tolerance   )
>
> 2557832  Remove obsolete collection execution example.
>
> a747b61  [FLINK-1202] Remove incomplete file outputs on failure …
>
> f42dcc3 [FLINK-1203] Deactivate fork reuse in tests
>
> f5898a0  Another round to remove final modifier to private methods because
> it
>
> 0ee60d3  [FLINK-1181] Fix IOReadableWritable checks in RPC service …
>
> b0c1af5  [FLINK-1182] Improve error messages for POJO type mismatches
>
> (   eebe510  Add DeltaPageRank example   )
>
> 3f33278  [FLINK-1194] Fix Java quickstart archetype POM file
>
> 73ee385  Remove unnecessary final modifier in
> PlanExecutor.loadExecutorClass m… …
>
> f2b8650  [FLINK-1178] Changed type preference to ValueType, WritableType,
> Case… …
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Stephan Ewen
Agree, also a good reason.

Any other opinions?

On Mon, Dec 8, 2014 at 10:34 AM, Robert Metzger <[hidden email]> wrote:

> I'm voting to release the current master as Flink 0.8.0.
>
> Once we have merged these two major changes (
> https://github.com/apache/incubator-flink/pull/149,
> https://github.com/apache/incubator-flink/pull/254) in the queue we will
> need some time until we reach a well tested, stable state again.
>
> On Mon, Dec 8, 2014 at 10:30 AM, Stephan Ewen <[hidden email]> wrote:
>
> > Hi all!
> >
> > Please find below a list of commits that I would propose to include in
> the
> > bugfix release.
> >
> > It does not contain any streaming commits. Marton, Gyula, can you add
> that?
> >
> > Thing is: it contains 90% of all commits. I put some in parenthesis that
> > are not bug fixes, but minor additions that are useful also in a minor
> > release.
> >
> > We might as well just take all commits and make it 0.8.0 ;-)
> >
> > Greetings,
> > Stephan
> >
> >
> > ------------------------------------------------------------------
> >
> > 770ce23  [FLINK-1301] Added Apache license headers to Markdown, HTML,
> SVG,
> > Pyt
> >
> > 979837a  [scala] Fix TupleSerializerTest for corrected
> CaseClassSerializer
> >
> > 3470b37  [scala] Fix non-reuse copy in CaseClassSerializer
> >
> > a677c77  [FLINK-1157] Document TaskManager slots and minor fixes.
> >
> > 858d1bc  [Java API] [docs] Rephrase Javadoc of minBy/maxBy and link from
> > min/max
> >
> > bc41197  [FLINK-1292] Allow for longer normalized keys when using
> composite
> > keys
> >
> > 33724f0  [Docs] Fix rendering issue and typo
> >
> > e3eaac2  [FLINK-984] Compiler tests for distinct().
> >
> > 84d6da8  [FLINK-1291] Remove default GC options from the shell scripts …
> >
> > 4e9c32f  [Docs] Just a typo in docs/index.md
> >
> > a9b7cbe  [scala] [runtime] Add extra tests for sorting of case classes
> >
> > 45fb6d8  [FLINK-1290] Fix Optimizer to create plans when encountering
> > incompat
> >
> > d554faa  [FLINK-1273] [runtime] Add Void type to basic types
> >
> > 17bc479  [FLINK-1095] [clients] Remove '-d' option in info from
> CliFrontend
> >
> > bbe54ac  [FLINK-1270] [APIs] FS.get() supports relative paths
> >
> > 3d242fd  Removed TypeSerializerFactory from ScalaCsvInputFormat. The
> > TypeSeria… …
> >
> > 1a7a50f  [scala] Add equals() to TraversableTypeInfo
> >
> > 1dd8b8d  Rewrite TPC-H Q3 to use Long instead of Integer …
> >
> > 1a91132  [scala] Fix slow building of adjacency list in Scala PageRank
> >
> > f66892d  [runtime] CaseClassSerializer correctly treated as stateful
> >
> > 2ae366b  [docs] Fix paths to work with local builds
> >
> > (  e8e7abb  [scala] Add ClosureCleaner for Scala Lambdas  )
> >
> > 74bc7dd  [FLINK-1282] [docs] Make Flink logo link back to frontpage
> >
> > 8a8155e  [FLINK-1282] [docs] Added {{ site.baseurl }} to css and image
> > paths
> >
> > 112b3a9  [Scala API] Case Class serializer can work with classes that
> > cannot b… …
> >
> > 48b6d01  [FLINK-1278] [runtime] (part 1) Remove special code paths for
> the
> > Rec… …
> >
> > d858930  [FLINK-1286] [APIs] [runtime] Fix serialization in
> > CollectionInputFor… …
> >
> > 82f5154  [FLINK-1282] [docs] Update layout of docs …
> >
> > 87497b7  [runtime] Quick fix for error with unsupported ship strategy
> >
> > 392683f  [FLINK-658] [APIs] Add group sorting to CoGroup
> >
> > 606a6d4  [Java API] Fix various warnings
> >
> > 8724bf7  Small changes and tests for the newly introduced EnumType
> >
> > (   65d85f1  [FLINK-1221] Use StackTraceElement methods instead of String
> > parsing   )
> >
> > c03237a   [FLINK-1221] Use only method name and location as default
> > operator names
> >
> > (   bcdd167  [FLINK-1249] [APIs] [compiler] Add custom partitioner for
> > CoGroup   )
> >
> > 4838efe  Fixes FLINK-1276
> >
> > 42fe874  Minor code cleanups - left from previous patches
> >
> > cf54a1c  [FLINK-1263] [optimizer] Implement compatibility checks for
> binary
> > operators
> >
> > d7853fd  [APIs] Enhance test coverage for CollectionInputFormat and add
> > tests...
> >
> > 02e2857  Log memory usage stats at INFO level instead of DEBUG
> >
> > a69d60a  Removed object reusage in GenericArraySerializer's deserialize
> > method.
> >
> > 98ff76b  [FLINK-820] [compiler] Support for disconnected data flows
> >
> > d0f2db0  [FLINK-1264] [compiler] Properly forward custom partitioners to
> > the runtime
> >
> > b3e5ed0  [Flink-1262] inconsistency between CsvReader.java and
> > TupleGenerator.
> >
> > 4a74f32  [FLINK-1265] Fix user classloader bug for registerInputOutput()
> > method
> >
> > a77d752  Added binary input and output format which uses the objects'
> > TypeSeri… …
> >
> > 8af0473  [FLINK-1252] address pull request comments
> >
> > 8081ddc  [FLINK-1251] Enums are now handled properly by the collection
> > input format
> >
> > 591f16d  [FLINK-1252] Add support for serializing Date and Enums in the
> > Java API
> >
> > 4203bf9  Minor code clean up …
> >
> > 5c3dceb  [FLINK-1142] Log information about I/O manager temp dirs …
> >
> > (   3ac4df8  [FLINK-993] Primitive input format fails for invalid input.
> >  )
> >
> > d640b6c  [FLINK-933] Add primitive input format to read a sequence of
> > primitives …
> >
> > ae505ad  [FLINK-1253] [tests] Make sure tests do not die with garbage
> > collecti… …
> >
> > ce822bf  [FLINK-1254] [compiler] Fix compiler bug for pipeline breaker
> > placement
> >
> > 54aa41b  Added getExecutionEnvironment to the DataSet of the Scala API.
> >
> > da60630  Add byte array serialization to InstantiationUtil
> >
> > f84e4ec  Minor code clean up to get rid of IntelliJ IDEA warnings …
> >
> > 1ecacf0  [FLINK-1247] [docs] Fix broken links in documentation
> >
> > 9f6a0b8  [FLINK-1250] [config] [docs] Correct and document config keys
> for
> > hea… …
> >
> > b264221  [FLINK-1209] [compiler] Improve error messages when forgetting
> to
> > clo… …
> >
> > 6989cec  [bin] Remove MaxPermSize parameter for JVM 1.8 …
> >
> > (   2000b45  [FLINK-1237] Add support for custom partitioners   )
> >
> > 83d0256  [FLINK-1207] Context environments are realized through
> factories …
> >
> > ae07abe  [build] Manage version of joda-time to prevent conflicts between
> > depe… …
> >
> > 06c259f  [tests] Various stability fixes to tests
> >
> > f76eb15  [FLINK-1246] Add additional debug output to flaky recovery test
> >
> > 2d1532f  Fix flaky test SumMinMaxITCase
> >
> > 9c1585e  [FLINK-1172] [docs] Fix broken links in documentation …
> >
> > a296f40  [FLINK-1244] setCombinable() returns operator …
> >
> > 7f8296e  [FLINK-890] [docs] Adjust examples to show reader methods and
> > correct… …
> >
> > 42828f2  [FLINK-1008] Fix createProgramPlan() throws exception …
> >
> > e23874c  [FLINK-1242] Fix streaming-examples and scala-examples POMs to
> > work p… …
> >
> > 922f478  [docs] Add description of Quasiquotes compiler plugin in Eclipse
> > to FAQ
> >
> > 8a955e5  [FLINK-1233] Fix flaky AggregateITCase
> >
> > d7d7238  [FLINK-1239] [streaming] IterateExample fix and parallelism
> > setting i… …
> >
> > c339c26  [FLINK-1218] Replace new Integer with Integer.valueOf in tests …
> >
> > 9ff2e5b  [scala] Self-contained build for scala examples …
> >
> > 8f9d8ea  Fix caching of receivers
> >
> > c6dd9b1  [FLINK-1204] [streaming] Individual, self-contained packaging
> for
> > str… …
> >
> > 818ebda  [FLINK-1221] Use the source line as the default operator name …
> >
> > b253cb2  [FLINK-1230] Add documentation and an example for
> collection-based
> > ex… …
> >
> > b3c290f  [FLINK-1226] Add API support for passing Configurations to
> > InputFormats …
> >
> > 21b1b97  [FLINK-1235] Compiler accepts iterations referenced from the
> > static p… …
> >
> > 3772d30  Improved input type inference, bug fixing, code simplification …
> >
> > 57ecbcc  [FLINK-1233] Improve error message when not enough slots are
> > availabl… …
> >
> > e855ef4  [FLINK-1223] Allow value escaping in CSV files …
> >
> > 2f1176a  [FLINK-983] Fix WebClient truncated function names …
> >
> > (   c9cfe3b [FLINK-1323] Refactor I/O Manager Readers and Writers to
> > interfaces   )
> >
> > 8e4c772  [FLINK-1186] Fix wrong flat key position when using expression
> key
> > fo… …
> >
> > 65d0359  Make sure S3 directory paths are suffixed with '/' on creation
> >
> > 81c5b2a  [FLINK-1215] Increase robustness to spurious failures when
> > creating o… …
> >
> > 4b75d83  [FLINK-1213] Added .gitattributes to normalize EOL …
> >
> > 90730fe  [FLINK-1212] Fixed shell script interpreter directive …
> >
> > 1e979e9  [Flink-1113] clean up JQuery dependencies
> >
> > e580497  [FLINK-1215] Fix spurious failures when creating output
> > directories
> >
> > ef9a373  [runtime] In local mode, make sure taskmanagers have completed
> > regist
> >
> > ef40691  [FLINK-1222] Tasks send close acknowledgements early.
> >
> > a959dd5  [FLINK-1220] Promote various log statements to INFO level. Add
> log
> > st… …
> >
> > 17e0186  Add test case for TaskManager connection loss, leading to task
> > failure.
> >
> > 6ecd0f8  [FLINK-1214] Prevent partitionings on subsets of fields from
> being
> > pushed down
> >
> > a586614  [FLINK-1005] By default, new objects are created for each
> element
> > in group reduce
> >
> > c3835cd  [scala] Add union operation to Scala ExecutionEnvironment
> >
> > e21d78d  [scala] Make getType public in Scala DataSet
> >
> > bd66a08  [FLINK-1191] Add support for Scala Collections and Special Types
> >
> > 1e1df6d  [scala] Add macroparadise to enable Quasiquotes
> >
> > 3510bf9  Change TypeExtractor to support interfaces within classes
> >
> > 233161b  [FLINK-1210] Improve error message in delta iterations when the
> > next
> >
> > 9b6561d  [doc] Fix programming guide link in iterations.md
> >
> > 9b25942  Temporarily disable fork reuse in flink-tests Unit tests
> > (workaround
> >
> > ca9e6ab  Make sure that operator names are properly escaped for display
> in
> > the web frontend
> >
> > (   7d2db95  [FLINK-1198] Fix nullpointer exception in broadcast variable
> > material   )
> >
> > (   e1fd689  [FLINK-1198] Broadcast variables can share initialization
>  )
> >
> > (   d8052d4  [FLINK-1198] Make Broadcast variables shared per
> taskmanager,
> > rather   )
> >
> > (   cc76653  [FLINK-1198] Broadcast variables are shared between tasks in
> > the same   )
> >
> > bbd8209  [FLINK-1203] Use forked execution in all tests, except in the
> > end-to-end tests
> >
> > a6152c3  [FLINK-1205] Fix library cache manager to track references to
> > tasks a… …
> >
> > 5e48fc9  Make sure TaskManager does not accept task deployments when
> > shutting … …
> >
> > 8b39ba9  Improve error messages in case of invalid file paths or URIs …
> >
> > (   f0fd882  Introduce a delay before restarts to make sure that
> > taskmanager failurs   )
> >
> > (   dd687bc  Implement coarse-grained fault tolerance   )
> >
> > 2557832  Remove obsolete collection execution example.
> >
> > a747b61  [FLINK-1202] Remove incomplete file outputs on failure …
> >
> > f42dcc3 [FLINK-1203] Deactivate fork reuse in tests
> >
> > f5898a0  Another round to remove final modifier to private methods
> because
> > it
> >
> > 0ee60d3  [FLINK-1181] Fix IOReadableWritable checks in RPC service …
> >
> > b0c1af5  [FLINK-1182] Improve error messages for POJO type mismatches
> >
> > (   eebe510  Add DeltaPageRank example   )
> >
> > 3f33278  [FLINK-1194] Fix Java quickstart archetype POM file
> >
> > 73ee385  Remove unnecessary final modifier in
> > PlanExecutor.loadExecutorClass m… …
> >
> > f2b8650  [FLINK-1178] Changed type preference to ValueType, WritableType,
> > Case… …
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Ufuk Celebi-2
In reply to this post by Robert Metzger
On Mon, Dec 8, 2014 at 10:34 AM, Robert Metzger <[hidden email]> wrote:

> I'm voting to release the current master as Flink 0.8.0.



I agree +1. The big changes you pointed out are a very good point for
releasing 0.8.0 now. :-) It also makes it much easier (no cherry picking
etc.) ;-)

– Ufuk
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Márton Balassi
Ok guys, then I also agree to skip 0.7.1 and go straight for 0.8.0. As for
the streaming side we would like to finish a couple of features (lambda 8
support, type handling rework, filesystems i/o support, at least once fault
tolerance prototype). I'm confident that we can get most of the things that
we really want in there done by the end of this week.

Cheers,

Marton

On Mon, Dec 8, 2014 at 10:41 AM, Ufuk Celebi <[hidden email]> wrote:

> On Mon, Dec 8, 2014 at 10:34 AM, Robert Metzger <[hidden email]>
> wrote:
>
> > I'm voting to release the current master as Flink 0.8.0.
>
>
>
> I agree +1. The big changes you pointed out are a very good point for
> releasing 0.8.0 now. :-) It also makes it much easier (no cherry picking
> etc.) ;-)
>
> – Ufuk
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Ufuk Celebi-2
On Mon, Dec 8, 2014 at 11:05 AM, Márton Balassi <[hidden email]>
wrote:

> Ok guys, then I also agree to skip 0.7.1 and go straight for 0.8.0. As for
> the streaming side we would like to finish a couple of features (lambda 8
> support, type handling rework, filesystems i/o support, at least once fault
> tolerance prototype). I'm confident that we can get most of the things that
> we really want in there done by the end of this week.
>


This sounds like a lot. I can understand that you want as much in a release
as possible though. ;) Would it make sense to reduce this list to the most
important changes and keep the rest for 0.8.1?
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Robert Metzger
I agree with Ufuk.
I'm uncertain if its a good idea to delay the 0.8.0 for these "last minute"
streaming features.
I have the impression that the current master is really stable and very
well tested. If we now bring in some prototypes and last minute features,
we probably end up discussing a 0.8.1 bugfix release in two weeks.
I would prefer a 0.8.0 release asap because it has already been way to long
with the issues unfixed in 0.7.0. Also, if the graduation is going to
happen soon, we'll be unable to release for two weeks or so because INFRA
has to transfer us to new namespace (mailing lists, website domain, release
spaces, ... )


As a general lesson from this discussion, we still have to improve our
communication to get a better mutual understanding of what we are working
on. I'll create a wiki page today that contains a roadmap of the features
we are working on.



On Mon, Dec 8, 2014 at 1:28 PM, Ufuk Celebi <[hidden email]> wrote:

> On Mon, Dec 8, 2014 at 11:05 AM, Márton Balassi <[hidden email]>
> wrote:
>
> > Ok guys, then I also agree to skip 0.7.1 and go straight for 0.8.0. As
> for
> > the streaming side we would like to finish a couple of features (lambda 8
> > support, type handling rework, filesystems i/o support, at least once
> fault
> > tolerance prototype). I'm confident that we can get most of the things
> that
> > we really want in there done by the end of this week.
> >
>
>
> This sounds like a lot. I can understand that you want as much in a release
> as possible though. ;) Would it make sense to reduce this list to the most
> important changes and keep the rest for 0.8.1?
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Stephan Ewen
In reply to this post by Ufuk Celebi-2
Hey!

I sort of agree with Ufuk. Type system rework easily leads to followup
issues. I suspect that if we wait for that, there is not going to be a
release before Christmas.

How essential is that functionality? Is the streaming API fundamentally
broken without that, or is it rather a set of features that define the next
big step? For issues that fall into the first category, let's prioritize
and include them. But for issues in the second category, I think it is fine
to postpone them to the next release (which we may want to do rather soon
after).

Stephan


On Mon, Dec 8, 2014 at 1:28 PM, Ufuk Celebi <[hidden email]> wrote:

> On Mon, Dec 8, 2014 at 11:05 AM, Márton Balassi <[hidden email]>
> wrote:
>
> > Ok guys, then I also agree to skip 0.7.1 and go straight for 0.8.0. As
> for
> > the streaming side we would like to finish a couple of features (lambda 8
> > support, type handling rework, filesystems i/o support, at least once
> fault
> > tolerance prototype). I'm confident that we can get most of the things
> that
> > we really want in there done by the end of this week.
> >
>
>
> This sounds like a lot. I can understand that you want as much in a release
> as possible though. ;) Would it make sense to reduce this list to the most
> important changes and keep the rest for 0.8.1?
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Stephan Ewen
So what do we do now? Can we narrow down what features we want to merge
still?

In any case, it would be good to branch off the 0.8 release branch soon, as
we have major changes waiting to be merged that should only go into the
next major version (0.9).

Stephan


On Mon, Dec 8, 2014 at 1:36 PM, Stephan Ewen <[hidden email]> wrote:

> Hey!
>
> I sort of agree with Ufuk. Type system rework easily leads to followup
> issues. I suspect that if we wait for that, there is not going to be a
> release before Christmas.
>
> How essential is that functionality? Is the streaming API fundamentally
> broken without that, or is it rather a set of features that define the next
> big step? For issues that fall into the first category, let's prioritize
> and include them. But for issues in the second category, I think it is fine
> to postpone them to the next release (which we may want to do rather soon
> after).
>
> Stephan
>
>
> On Mon, Dec 8, 2014 at 1:28 PM, Ufuk Celebi <[hidden email]> wrote:
>
>> On Mon, Dec 8, 2014 at 11:05 AM, Márton Balassi <[hidden email]
>> >
>> wrote:
>>
>> > Ok guys, then I also agree to skip 0.7.1 and go straight for 0.8.0. As
>> for
>> > the streaming side we would like to finish a couple of features (lambda
>> 8
>> > support, type handling rework, filesystems i/o support, at least once
>> fault
>> > tolerance prototype). I'm confident that we can get most of the things
>> that
>> > we really want in there done by the end of this week.
>> >
>>
>>
>> This sounds like a lot. I can understand that you want as much in a
>> release
>> as possible though. ;) Would it make sense to reduce this list to the most
>> important changes and keep the rest for 0.8.1?
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Márton Balassi
In reply to this post by Stephan Ewen
Hey,

Sorry for the late answer. We delayed the fault tolerance issue for 0.9,
the lambda expression support is here [1] and also contains the refactor on
serialization (which I wrongly mentioned as type handling). The commit on
filesystems is also pretty much ready, but I have not tested it on the
cluster yet. Can push it tomorrow morning. That is all from the streaming
side that was in question for 0.8.

Would you like to have the major changes Robert mentioned in 0.8?
(https://github.com/apache/incubator-flink/pull/149,
https://github.com/apache/incubator-flink/pull/254)

[1] https://github.com/apache/incubator-flink/pull/256

On Mon, Dec 8, 2014 at 1:36 PM, Stephan Ewen <[hidden email]> wrote:

> Hey!
>
> I sort of agree with Ufuk. Type system rework easily leads to followup
> issues. I suspect that if we wait for that, there is not going to be a
> release before Christmas.
>
> How essential is that functionality? Is the streaming API fundamentally
> broken without that, or is it rather a set of features that define the next
> big step? For issues that fall into the first category, let's prioritize
> and include them. But for issues in the second category, I think it is fine
> to postpone them to the next release (which we may want to do rather soon
> after).
>
> Stephan
>
>
> On Mon, Dec 8, 2014 at 1:28 PM, Ufuk Celebi <[hidden email]> wrote:
>
> > On Mon, Dec 8, 2014 at 11:05 AM, Márton Balassi <
> [hidden email]>
> > wrote:
> >
> > > Ok guys, then I also agree to skip 0.7.1 and go straight for 0.8.0. As
> > for
> > > the streaming side we would like to finish a couple of features
> (lambda 8
> > > support, type handling rework, filesystems i/o support, at least once
> > fault
> > > tolerance prototype). I'm confident that we can get most of the things
> > that
> > > we really want in there done by the end of this week.
> > >
> >
> >
> > This sounds like a lot. I can understand that you want as much in a
> release
> > as possible though. ;) Would it make sense to reduce this list to the
> most
> > important changes and keep the rest for 0.8.1?
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Stephan Ewen
Hey!

Good to hear, so we could feature-freeze the 0.8 branch tomorrow or
Wednesday and focus on testing and remaining bug fixes?

The two big pull requests Robert mentioned should not go into 0.8 in my
opinion (and I think Till and Ufuk who created them are on the same page),
because they will probably need a few additional
weeks of continuous testing until we are confident that they are in a
release ready state.

Stephan


On Tue, Dec 9, 2014 at 6:01 PM, Márton Balassi <[hidden email]>
wrote:

> Hey,
>
> Sorry for the late answer. We delayed the fault tolerance issue for 0.9,
> the lambda expression support is here [1] and also contains the refactor on
> serialization (which I wrongly mentioned as type handling). The commit on
> filesystems is also pretty much ready, but I have not tested it on the
> cluster yet. Can push it tomorrow morning. That is all from the streaming
> side that was in question for 0.8.
>
> Would you like to have the major changes Robert mentioned in 0.8?
> (https://github.com/apache/incubator-flink/pull/149,
> https://github.com/apache/incubator-flink/pull/254)
>
> [1] https://github.com/apache/incubator-flink/pull/256
>
> On Mon, Dec 8, 2014 at 1:36 PM, Stephan Ewen <[hidden email]> wrote:
>
> > Hey!
> >
> > I sort of agree with Ufuk. Type system rework easily leads to followup
> > issues. I suspect that if we wait for that, there is not going to be a
> > release before Christmas.
> >
> > How essential is that functionality? Is the streaming API fundamentally
> > broken without that, or is it rather a set of features that define the
> next
> > big step? For issues that fall into the first category, let's prioritize
> > and include them. But for issues in the second category, I think it is
> fine
> > to postpone them to the next release (which we may want to do rather soon
> > after).
> >
> > Stephan
> >
> >
> > On Mon, Dec 8, 2014 at 1:28 PM, Ufuk Celebi <[hidden email]> wrote:
> >
> > > On Mon, Dec 8, 2014 at 11:05 AM, Márton Balassi <
> > [hidden email]>
> > > wrote:
> > >
> > > > Ok guys, then I also agree to skip 0.7.1 and go straight for 0.8.0.
> As
> > > for
> > > > the streaming side we would like to finish a couple of features
> > (lambda 8
> > > > support, type handling rework, filesystems i/o support, at least once
> > > fault
> > > > tolerance prototype). I'm confident that we can get most of the
> things
> > > that
> > > > we really want in there done by the end of this week.
> > > >
> > >
> > >
> > > This sounds like a lot. I can understand that you want as much in a
> > release
> > > as possible though. ;) Would it make sense to reduce this list to the
> > most
> > > important changes and keep the rest for 0.8.1?
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Ufuk Celebi-2
On Tue, Dec 9, 2014 at 6:07 PM, Stephan Ewen <[hidden email]> wrote:

> The two big pull requests Robert mentioned should not go into 0.8 in my
> opinion (and I think Till and Ufuk who created them are on the same page),
> because they will probably need a few additional
> weeks of continuous testing until we are confident that they are in a
> release ready state.
>


I agree. It would be nice if
https://github.com/apache/incubator-flink/pull/255 could make it though ;)
Reply | Threaded
Open this post in threaded view
|

Re: Commits for next release

Till Rohrmann
I agree as well.

On Tue, Dec 9, 2014 at 6:13 PM, Ufuk Celebi <[hidden email]> wrote:

> On Tue, Dec 9, 2014 at 6:07 PM, Stephan Ewen <[hidden email]> wrote:
>
> > The two big pull requests Robert mentioned should not go into 0.8 in my
> > opinion (and I think Till and Ufuk who created them are on the same
> page),
> > because they will probably need a few additional
> > weeks of continuous testing until we are confident that they are in a
> > release ready state.
> >
>
>
> I agree. It would be nice if
> https://github.com/apache/incubator-flink/pull/255 could make it though ;)
>