[Discuss] Migrate walkthroughs to flink-playgrounds for 1.11

[Seth Wiesman-4]

Hi Everyone

I am currently going through the documentation for 1.11 and noticed an
issue with the walkthroughs.

Flink currently contains two walkthroughs for new users, one for data
stream and one for table[1, 2]. They both have corresponding maven
archetypes for users to create template projects with the proper
dependencies and follow along[3, 4].

The table walkthrough was added several releases ago and now actively
exposes deprecated and even internal apis. These are meant to serve as
users first exposure to Flink and is not a good impression.

To that end, I want to propose dropping the maven archetypes from 1.11 and
adding new dockerized playgrounds to go along with the docs in the
flink-playgrounds repo.

While we are past the feature freeze I think this is important and would
not affect release testing or 1.11's stability. I am volunteering to ensure
the work is completed before the release.

Seth

[1]
https://ci.apache.org/projects/flink/flink-docs-master/getting-started/walkthroughs/table_api.html
[2]
https://github.com/apache/flink/blob/master/flink-walkthroughs/flink-walkthrough-table-java/src/main/resources/archetype-resources/src/main/java/SpendReport.java

[dwysakowicz]

I am very much in favour of this effort.

Best,

Dawid

On 09/06/2020 16:06, Seth Wiesman wrote:

> Hi Everyone
>
> I am currently going through the documentation for 1.11 and noticed an
> issue with the walkthroughs.
>
> Flink currently contains two walkthroughs for new users, one for data
> stream and one for table[1, 2]. They both have corresponding maven
> archetypes for users to create template projects with the proper
> dependencies and follow along[3, 4].
>
> The table walkthrough was added several releases ago and now actively
> exposes deprecated and even internal apis. These are meant to serve as
> users first exposure to Flink and is not a good impression.
>
> To that end, I want to propose dropping the maven archetypes from 1.11 and
> adding new dockerized playgrounds to go along with the docs in the
> flink-playgrounds repo.
>
> While we are past the feature freeze I think this is important and would
> not affect release testing or 1.11's stability. I am volunteering to ensure
> the work is completed before the release.
>
> Seth
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-master/getting-started/walkthroughs/table_api.html
> [2]
> https://github.com/apache/flink/blob/master/flink-walkthroughs/flink-walkthrough-table-java/src/main/resources/archetype-resources/src/main/java/SpendReport.java
>

[Konstantin Knauf-4]

Hi Seth,

good catch! I agree this is important to fix.

I don't quite understand the resolution you are proposing. The current
walkthrough for the Table API is a *code *walkthrough. How do you plan to
replace this by a dockerized playground? Drop the code walkthrough and add
a pure SQL playground?
Best,

Kosntantin


On Tue, Jun 9, 2020 at 4:27 PM Dawid Wysakowicz <[hidden email]>
wrote:

> I am very much in favour of this effort.
>
> Best,
>
> Dawid
>
> On 09/06/2020 16:06, Seth Wiesman wrote:
> > Hi Everyone
> >
> > I am currently going through the documentation for 1.11 and noticed an
> > issue with the walkthroughs.
> >
> > Flink currently contains two walkthroughs for new users, one for data
> > stream and one for table[1, 2]. They both have corresponding maven
> > archetypes for users to create template projects with the proper
> > dependencies and follow along[3, 4].
> >
> > The table walkthrough was added several releases ago and now actively
> > exposes deprecated and even internal apis. These are meant to serve as
> > users first exposure to Flink and is not a good impression.
> >
> > To that end, I want to propose dropping the maven archetypes from 1.11
> and
> > adding new dockerized playgrounds to go along with the docs in the
> > flink-playgrounds repo.
> >
> > While we are past the feature freeze I think this is important and would
> > not affect release testing or 1.11's stability. I am volunteering to
> ensure
> > the work is completed before the release.
> >
> > Seth
> >
> > [1]
> >
> https://ci.apache.org/projects/flink/flink-docs-master/getting-started/walkthroughs/table_api.html
> > [2]
> >
> https://github.com/apache/flink/blob/master/flink-walkthroughs/flink-walkthrough-table-java/src/main/resources/archetype-resources/src/main/java/SpendReport.java
> >
>
>

--

Konstantin Knauf

https://twitter.com/snntrable

https://github.com/knaufk

[Seth Wiesman-4]

Hi Konstantin,

I am linking my working branch for what the playground might look like. It
contains a Java application you can import into your IDE and then fill out
the same way you do currently. It is still a code walkthrough. Docker
allows users to actually run the completed app locally and not just in
their IDE.

[1] https://github.com/sjwiesman/flink-playgrounds/tree/walkthroughs

On Tue, Jun 9, 2020 at 2:39 PM Konstantin Knauf <[hidden email]> wrote:

> Hi Seth,
>
> good catch! I agree this is important to fix.
>
> I don't quite understand the resolution you are proposing. The current
> walkthrough for the Table API is a *code *walkthrough. How do you plan to
> replace this by a dockerized playground? Drop the code walkthrough and add
> a pure SQL playground?
> Best,
>
> Kosntantin
>
>
> On Tue, Jun 9, 2020 at 4:27 PM Dawid Wysakowicz <[hidden email]>
> wrote:
>
>> I am very much in favour of this effort.
>>
>> Best,
>>
>> Dawid
>>
>> On 09/06/2020 16:06, Seth Wiesman wrote:
>> > Hi Everyone
>> >
>> > I am currently going through the documentation for 1.11 and noticed an
>> > issue with the walkthroughs.
>> >
>> > Flink currently contains two walkthroughs for new users, one for data
>> > stream and one for table[1, 2]. They both have corresponding maven
>> > archetypes for users to create template projects with the proper
>> > dependencies and follow along[3, 4].
>> >
>> > The table walkthrough was added several releases ago and now actively
>> > exposes deprecated and even internal apis. These are meant to serve as
>> > users first exposure to Flink and is not a good impression.
>> >
>> > To that end, I want to propose dropping the maven archetypes from 1.11
>> and
>> > adding new dockerized playgrounds to go along with the docs in the
>> > flink-playgrounds repo.
>> >
>> > While we are past the feature freeze I think this is important and would
>> > not affect release testing or 1.11's stability. I am volunteering to
>> ensure
>> > the work is completed before the release.
>> >
>> > Seth
>> >
>> > [1]
>> >
>> https://ci.apache.org/projects/flink/flink-docs-master/getting-started/walkthroughs/table_api.html
>> > [2]
>> >
>> https://github.com/apache/flink/blob/master/flink-walkthroughs/flink-walkthrough-table-java/src/main/resources/archetype-resources/src/main/java/SpendReport.java
>> >
>>
>>
>
> --
>
> Konstantin Knauf
>
> https://twitter.com/snntrable
>
> https://github.com/knaufk
>

[Seth Wiesman-4]

IMO, having the walkthroughs as maven archetypes is not maintainable.

On Tue, Jun 9, 2020 at 2:47 PM Seth Wiesman <[hidden email]> wrote:

> Hi Konstantin,
>
> I am linking my working branch for what the playground might look like. It
> contains a Java application you can import into your IDE and then fill out
> the same way you do currently. It is still a code walkthrough. Docker
> allows users to actually run the completed app locally and not just in
> their IDE.
>
> [1] https://github.com/sjwiesman/flink-playgrounds/tree/walkthroughs
>
> On Tue, Jun 9, 2020 at 2:39 PM Konstantin Knauf <[hidden email]> wrote:
>
>> Hi Seth,
>>
>> good catch! I agree this is important to fix.
>>
>> I don't quite understand the resolution you are proposing. The current
>> walkthrough for the Table API is a *code *walkthrough. How do you plan
>> to replace this by a dockerized playground? Drop the code walkthrough and
>> add a pure SQL playground?
>> Best,
>>
>> Kosntantin
>>
>>
>> On Tue, Jun 9, 2020 at 4:27 PM Dawid Wysakowicz <[hidden email]>
>> wrote:
>>
>>> I am very much in favour of this effort.
>>>
>>> Best,
>>>
>>> Dawid
>>>
>>> On 09/06/2020 16:06, Seth Wiesman wrote:
>>> > Hi Everyone
>>> >
>>> > I am currently going through the documentation for 1.11 and noticed an
>>> > issue with the walkthroughs.
>>> >
>>> > Flink currently contains two walkthroughs for new users, one for data
>>> > stream and one for table[1, 2]. They both have corresponding maven
>>> > archetypes for users to create template projects with the proper
>>> > dependencies and follow along[3, 4].
>>> >
>>> > The table walkthrough was added several releases ago and now actively
>>> > exposes deprecated and even internal apis. These are meant to serve as
>>> > users first exposure to Flink and is not a good impression.
>>> >
>>> > To that end, I want to propose dropping the maven archetypes from 1.11
>>> and
>>> > adding new dockerized playgrounds to go along with the docs in the
>>> > flink-playgrounds repo.
>>> >
>>> > While we are past the feature freeze I think this is important and
>>> would
>>> > not affect release testing or 1.11's stability. I am volunteering to
>>> ensure
>>> > the work is completed before the release.
>>> >
>>> > Seth
>>> >
>>> > [1]
>>> >
>>> https://ci.apache.org/projects/flink/flink-docs-master/getting-started/walkthroughs/table_api.html
>>> > [2]
>>> >
>>> https://github.com/apache/flink/blob/master/flink-walkthroughs/flink-walkthrough-table-java/src/main/resources/archetype-resources/src/main/java/SpendReport.java
>>> >
>>>
>>>
>>
>> --
>>
>> Konstantin Knauf
>>
>> https://twitter.com/snntrable
>>
>> https://github.com/knaufk
>>
>

[Konstantin Knauf-3]

Hi Seth,

I see, that's neat. I think it is a good idea moving the archetypes out of
the main repository into flink-playgrounds.

Best,

Konstantin



On Tue, Jun 9, 2020 at 9:47 PM Seth Wiesman <[hidden email]> wrote:

> Hi Konstantin,
>
> I am linking my working branch for what the playground might look like. It
> contains a Java application you can import into your IDE and then fill out
> the same way you do currently. It is still a code walkthrough. Docker
> allows users to actually run the completed app locally and not just in
> their IDE.
>
> [1] https://github.com/sjwiesman/flink-playgrounds/tree/walkthroughs
>
> On Tue, Jun 9, 2020 at 2:39 PM Konstantin Knauf <[hidden email]> wrote:
>
> > Hi Seth,
> >
> > good catch! I agree this is important to fix.
> >
> > I don't quite understand the resolution you are proposing. The current
> > walkthrough for the Table API is a *code *walkthrough. How do you plan to
> > replace this by a dockerized playground? Drop the code walkthrough and
> add
> > a pure SQL playground?
> > Best,
> >
> > Kosntantin
> >
> >
> > On Tue, Jun 9, 2020 at 4:27 PM Dawid Wysakowicz <[hidden email]>
> > wrote:
> >
> >> I am very much in favour of this effort.
> >>
> >> Best,
> >>
> >> Dawid
> >>
> >> On 09/06/2020 16:06, Seth Wiesman wrote:
> >> > Hi Everyone
> >> >
> >> > I am currently going through the documentation for 1.11 and noticed an
> >> > issue with the walkthroughs.
> >> >
> >> > Flink currently contains two walkthroughs for new users, one for data
> >> > stream and one for table[1, 2]. They both have corresponding maven
> >> > archetypes for users to create template projects with the proper
> >> > dependencies and follow along[3, 4].
> >> >
> >> > The table walkthrough was added several releases ago and now actively
> >> > exposes deprecated and even internal apis. These are meant to serve as
> >> > users first exposure to Flink and is not a good impression.
> >> >
> >> > To that end, I want to propose dropping the maven archetypes from 1.11
> >> and
> >> > adding new dockerized playgrounds to go along with the docs in the
> >> > flink-playgrounds repo.
> >> >
> >> > While we are past the feature freeze I think this is important and
> would
> >> > not affect release testing or 1.11's stability. I am volunteering to
> >> ensure
> >> > the work is completed before the release.
> >> >
> >> > Seth
> >> >
> >> > [1]
> >> >
> >>
> https://ci.apache.org/projects/flink/flink-docs-master/getting-started/walkthroughs/table_api.html
> >> > [2]
> >> >
> >>
> https://github.com/apache/flink/blob/master/flink-walkthroughs/flink-walkthrough-table-java/src/main/resources/archetype-resources/src/main/java/SpendReport.java
> >> >
> >>
> >>
> >
> > --
> >
> > Konstantin Knauf
> >
> > https://twitter.com/snntrable
> >
> > https://github.com/knaufk
> >
>

--

Konstantin Knauf | Head of Product

[phone]


Follow us @VervericaData Ververica <https://www.ververica.com/>


--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Tony) Cheng

[Seth Wiesman-4]

It seems like there is general consensus. I have opened FLINK-18194 and
corresponding PRs against the docs and flink-playgrounds

Seth

[1] https://github.com/apache/flink/pull/12592
[2] https://github.com/apache/flink-playgrounds/pull/13

On Tue, Jun 9, 2020 at 2:50 PM Konstantin Knauf <[hidden email]>
wrote:

> Hi Seth,
>
> I see, that's neat. I think it is a good idea moving the archetypes out of
> the main repository into flink-playgrounds.
>
> Best,
>
> Konstantin
>
>
>
> On Tue, Jun 9, 2020 at 9:47 PM Seth Wiesman <[hidden email]> wrote:
>
> > Hi Konstantin,
> >
> > I am linking my working branch for what the playground might look like.
> It
> > contains a Java application you can import into your IDE and then fill
> out
> > the same way you do currently. It is still a code walkthrough. Docker
> > allows users to actually run the completed app locally and not just in
> > their IDE.
> >
> > [1] https://github.com/sjwiesman/flink-playgrounds/tree/walkthroughs
> >
> > On Tue, Jun 9, 2020 at 2:39 PM Konstantin Knauf <[hidden email]>
> wrote:
> >
> > > Hi Seth,
> > >
> > > good catch! I agree this is important to fix.
> > >
> > > I don't quite understand the resolution you are proposing. The current
> > > walkthrough for the Table API is a *code *walkthrough. How do you plan
> to
> > > replace this by a dockerized playground? Drop the code walkthrough and
> > add
> > > a pure SQL playground?
> > > Best,
> > >
> > > Kosntantin
> > >
> > >
> > > On Tue, Jun 9, 2020 at 4:27 PM Dawid Wysakowicz <
> [hidden email]>
> > > wrote:
> > >
> > >> I am very much in favour of this effort.
> > >>
> > >> Best,
> > >>
> > >> Dawid
> > >>
> > >> On 09/06/2020 16:06, Seth Wiesman wrote:
> > >> > Hi Everyone
> > >> >
> > >> > I am currently going through the documentation for 1.11 and noticed
> an
> > >> > issue with the walkthroughs.
> > >> >
> > >> > Flink currently contains two walkthroughs for new users, one for
> data
> > >> > stream and one for table[1, 2]. They both have corresponding maven
> > >> > archetypes for users to create template projects with the proper
> > >> > dependencies and follow along[3, 4].
> > >> >
> > >> > The table walkthrough was added several releases ago and now
> actively
> > >> > exposes deprecated and even internal apis. These are meant to serve
> as
> > >> > users first exposure to Flink and is not a good impression.
> > >> >
> > >> > To that end, I want to propose dropping the maven archetypes from
> 1.11
> > >> and
> > >> > adding new dockerized playgrounds to go along with the docs in the
> > >> > flink-playgrounds repo.
> > >> >
> > >> > While we are past the feature freeze I think this is important and
> > would
> > >> > not affect release testing or 1.11's stability. I am volunteering to
> > >> ensure
> > >> > the work is completed before the release.
> > >> >
> > >> > Seth
> > >> >
> > >> > [1]
> > >> >
> > >>
> >
> https://ci.apache.org/projects/flink/flink-docs-master/getting-started/walkthroughs/table_api.html
> > >> > [2]
> > >> >
> > >>
> >
> https://github.com/apache/flink/blob/master/flink-walkthroughs/flink-walkthrough-table-java/src/main/resources/archetype-resources/src/main/java/SpendReport.java
> > >> >
> > >>
> > >>
> > >
> > > --
> > >
> > > Konstantin Knauf
> > >
> > > https://twitter.com/snntrable
> > >
> > > https://github.com/knaufk
> > >
> >
>
>
> --
>
> Konstantin Knauf | Head of Product
>
> [phone]
>
>
> Follow us @VervericaData Ververica <https://www.ververica.com/>
>
>
> --
>
> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> Conference
>
> Stream Processing | Event Driven | Real Time
>
> --
>
> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Ververica GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> (Tony) Cheng
>