I agree with Dawid that we should aim for 1.9. Compared to other
> Hi all,
>
> I'm investigating using Table-API as an abstraction for Batch-API in
> our GRADOOP project [1]. As a user of Flink I like to give you some
> feedback from the user perspective.
>
> I really like Timo's ideas - since we built another framework which is
> based on Apache Flink we are just providing another API for our users.
> Using java string literals (or building boolean expressions via string
> builder, for instance) in our code isn't convenient. Therefore I built
> little helpers for our special use cases:
>
> - A builder [2] to build a single Expression, e.g.
>
> ExpressionBuilder builder = new ExpressionBuilder();
> Expression e = builder
> .field("foo")
> .as("bar")
> .toExpression();
>
> - A builder [3] to build a Seq<Expression>, e.g.
>
> ExpressionSeqBuilder builder = new ExpressionSeqBuilder();
> Seq<Expression> seq = builder
> .field("foo")
> .as("a")
> .field("bar")
> .as("b")
> .buildSeq()
>
> Using those builders we make use of those "select(Expression)" API
> methods. Of course, we would appreciate a DSL provided by Flink!
>
> Best,
> Elias
>
> [1]
https://dbs.uni-leipzig.de/en/research/projects/gradoop>
> [2]
>
https://github.com/lordon/gradoop/blob/table-api/gradoop-flink/src/main/java/org/gradoop/flink/model/impl/layouts/table/util/ExpressionBuilder.java>
> [3]
>
https://github.com/lordon/gradoop/blob/table-api/gradoop-flink/src/main/java/org/gradoop/flink/model/impl/layouts/table/util/ExpressionSeqBuilder.java>
> Am 26.03.19 um 10:22 schrieb jincheng sun:
>> Thanks for bringing up this DISCUSS Timo!
>>
>> Java Expression DSL is pretty useful for java user. When we have the
>> Java
>> Expression DSL, Java API will become very rich and easy to use!
>>
>> +1 from my side.
>>
>> Best,
>> Jincheng
>>
>>
>> Dawid Wysakowicz <
[hidden email]> 于2019年3月26日周二
>> 下午5:08写道：
>>
>>> Hi,
>>>
>>> I really like the idea of introducing Java Expression DSL. I think this
>>> will solve many problems e.g. right now it's quite tricky how string
>>> literals work in scala (sometimes it might go through the
>>> ExpressionParser
>>> and it will end up as an UnresolvedFieldReference), another important
>>> problem we could solve with this is the need for unique column names in
>>> tables right now. We could at some point introduce sth like:
>>>
>>> Table table = ...
>>>
>>> table.field("fieldName")
>>>
>>> and etc. A common "entry point" to expressions should simplify a lot.
>>>
>>> Therefore I am strongly +1 for introducing this feature.
>>>
>>> @Jark I think we could aim to introduce the new Java DSL API in 1.9 and
>>> once we do that we could deprecate the string approach.
>>>
>>> Best,
>>>
>>> Dawid
>>> On 22/03/2019 03:36, Jark Wu wrote:
>>>
>>> Hi Timo,
>>>
>>> Sounds good to me.
>>>
>>> Do you want to deprecate the string-based API in 1.9 or make the
>>> decision
>>> in 1.10 after some feedbacks ?
>>>
>>>
>>> On Thu, 21 Mar 2019 at 21:32, Timo Walther <
[hidden email]> wrote:
>>>
>>>> Thanks for your feedback Rong and Jark.
>>>>
>>>> @Jark: Yes, you are right that the string-based API is used quite a
>>>> lot.
>>>> On the other side, the potential user base in the future is still
>>>> bigger
>>>> than our current user base. Because the Table API will become equally
>>>> important as the DataStream API, we really need to fix some crucial
>>>> design
>>>> decisions before it is too late. I would suggest to introduce the
>>>> new DSL
>>>> in 1.9 and remove the Expression parser either in 1.10 or 1.11. From a
>>>> developement point of view, I think we can handle the overhead to
>>>> maintain
>>>> 3 APIs until then because 2 APIs will share the same code base +
>>>> expression
>>>> parser.
>>>>
>>>> Regards,
>>>> Timo
>>>>
>>>> Am 21.03.19 um 05:21 schrieb Jark Wu:
>>>>
>>>> Hi Timo,
>>>>
>>>> I'm +1 on the proposal. I like the idea to provide a Java DSL which is
>>>> more friendly than string-based approach in programming.
>>>>
>>>> My concern is if/when we can drop the string-based expression
>>>> parser. If
>>>> it takes a very long time, we have to paid more development
>>>> cost on the three Table APIs. As far as I know, the string-based
>>>> API is
>>>> used in many companies.
>>>> We should also get some feedbacks from users. So I'm CCing this
>>>> email to
>>>> user mailing list.
>>>>
>>>> Best,
>>>> Jark
>>>>
>>>>
>>>>
>>>> On Wed, 20 Mar 2019 at 08:51, Rong Rong <
[hidden email]> wrote:
>>>>
>>>>> Thanks for sharing the initiative of improving Java side Table
>>>>> expression
>>>>> DSL.
>>>>>
>>>>> I agree as in the doc stated that Java DSL was always a "3rd class
>>>>> citizen"
>>>>> and we've run into many hand holding scenarios with our Flink
>>>>> developers
>>>>> trying to get the Stringify syntax working.
>>>>> Overall I am a +1 on this, it also help reduce the development
>>>>> cost of
>>>>> the
>>>>> Table API so that we no longer need to maintain different DSL and
>>>>> documentations.
>>>>>
>>>>> I left a few comments in the doc. and also some features that I think
>>>>> will
>>>>> be beneficial to the final outcome. Please kindly take a look @Timo.
>>>>>
>>>>> Many thanks,
>>>>> Rong
>>>>>
>>>>> On Mon, Mar 18, 2019 at 7:15 AM Timo Walther <
[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>> some of you might have already noticed the JIRA issue that I opened
>>>>>> recently [1] about introducing a proper Java expression DSL for the
>>>>>> Table API. Instead of using string-based expressions, we should
>>>>>> aim for
>>>>>> a unified, maintainable, programmatic Java DSL.
>>>>>>
>>>>>> Some background: The Blink merging efforts and the big
>>>>>> refactorings as
>>>>>> part of FLIP-32 have revealed many shortcomings in the current
>>>>>> Table &
>>>>>> SQL API design. Most of these legacy issues cause problems
>>>>>> nowadays in
>>>>>> making the Table API a first-class API next to the DataStream
>>>>>> API. An
>>>>>> example is the ExpressionParser class[2]. It was implemented in the
>>>>>> early days of the Table API using Scala parser combinators.
>>>>>> During the
>>>>>> last years, this parser caused many JIRA issues and user
>>>>>> confusion on
>>>>>> the mailing list. Because the exceptions and syntax might not be
>>>>>> straight forward.
>>>>>>
>>>>>> For FLINK-11908, we added a temporary bridge instead of
>>>>>> reimplementing
>>>>>> the parser in Java for FLIP-32. However, this is only a intermediate
>>>>>> solution until we made a final decision.
>>>>>>
>>>>>> I would like to propose a new, parser-free version of the Java Table
>>>>> API:
>>>>>>
>>>>>>
>>>>>
https://docs.google.com/document/d/1r3bfR9R6q5Km0wXKcnhfig2XQ4aMiLG5h2MTx960Fg8/edit?usp=sharing
>>>>>
>>>>>> I already implemented an early protoype that shows that such a
>>>>>> DSL is
>>>>>> not much implementation effort and integrates nicely with all
>>>>>> existing
>>>>>> API methods.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> Thanks for your feedback,
>>>>>>
>>>>>> Timo
>>>>>>
>>>>>> [1]
https://issues.apache.org/jira/browse/FLINK-11890>>>>>>
>>>>>> [2]
>>>>>>
>>>>>>
>>>>>
https://github.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/scala/org/apache/flink/table/expressions/PlannerExpressionParserImpl.scala
>>>>>
>>>>>>
>>>>
Locked
signature.asc (849 bytes)