(DEPRECATED) Apache Flink Mailing List archive.

[VOTE] FLIP-100: Add Attempt Information

Classic

List

Threaded

23 messages Options

Yadong Xie

[VOTE] FLIP-100: Add Attempt Information

Hi all

I want to start the vote for FLIP-100, which proposes to add attempt
information inside subtask and timeline in web UI.

To help everyone better understand the proposal, we spent some efforts on
making an online POC

Timeline Attempt (click the vertex timeline to see the differences):
previous web:
http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
POC web:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline

Subtask Attempt (click the vertex and switch to subtask tab to see the
differences):
previous web:
http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
POC web:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview

The vote will last for at least 72 hours, following the consensus voting
process.

FLIP wiki:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information

Discussion thread:
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html

Thanks,

Yadong

Kurt Young

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Yadong,

Thanks for the proposal, it's a useful feature, especially for batch jobs.
But according
to the examples you gave, I can't tell whether i got required information
from that.
Can you replace the demo job to a more complex batch job and then we can
see some
differences of start/stop time of different tasks and attempts?

Best,
Kurt

On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]> wrote:

> Hi all
>
> I want to start the vote for FLIP-100, which proposes to add attempt
> information inside subtask and timeline in web UI.
>
> To help everyone better understand the proposal, we spent some efforts on
> making an online POC
>
> Timeline Attempt (click the vertex timeline to see the differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> POC web:
>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>
> Subtask Attempt (click the vertex and switch to subtask tab to see the
> differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> POC web:
>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>
>
> The vote will last for at least 72 hours, following the consensus voting
> process.
>
> FLIP wiki:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
> Discussion thread:
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>
> Thanks,
>
> Yadong
>

Jark Wu-2

Re: [VOTE] FLIP-100: Add Attempt Information

Thanks Yadong,

I think one of the purpose to find the failed attempts is that we want to
look into the logs in the failed node (node may change across attempts).
However, from the POC, I didn't find a log link on each attempts. Maybe we
can add such feature?

Best,
Jark

On Fri, 21 Feb 2020 at 11:51, Kurt Young <[hidden email]> wrote:

> Hi Yadong,
>
> Thanks for the proposal, it's a useful feature, especially for batch jobs.
> But according
> to the examples you gave, I can't tell whether i got required information
> from that.
> Can you replace the demo job to a more complex batch job and then we can
> see some
> differences of start/stop time of different tasks and attempts?
>
> Best,
> Kurt
>
>
> On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]> wrote:
>
> > Hi all
> >
> > I want to start the vote for FLIP-100, which proposes to add attempt
> > information inside subtask and timeline in web UI.
> >
> > To help everyone better understand the proposal, we spent some efforts on
> > making an online POC
> >
> > Timeline Attempt (click the vertex timeline to see the differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >
> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >
> >
> > The vote will last for at least 72 hours, following the consensus voting
> > process.
> >
> > FLIP wiki:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> > Discussion thread:
> >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >
> > Thanks,
> >
> > Yadong
> >
>

Benchao Li

Re: [VOTE] FLIP-100: Add Attempt Information

I agree with Jark.
It's helpful to add a TM logs link for each attempt. Usually there are
multiple TMs per host, so only host name cannot determine the real TM.

Jark Wu <[hidden email]> 于2020年2月21日周五下午2:06写道：

> Thanks Yadong,
>
> I think one of the purpose to find the failed attempts is that we want to
> look into the logs in the failed node (node may change across attempts).
> However, from the POC, I didn't find a log link on each attempts. Maybe we
> can add such feature?
>
> Best,
> Jark
>
> On Fri, 21 Feb 2020 at 11:51, Kurt Young <[hidden email]> wrote:
>
> > Hi Yadong,
> >
> > Thanks for the proposal, it's a useful feature, especially for batch
> jobs.
> > But according
> > to the examples you gave, I can't tell whether i got required information
> > from that.
> > Can you replace the demo job to a more complex batch job and then we can
> > see some
> > differences of start/stop time of different tasks and attempts?
> >
> > Best,
> > Kurt
> >
> >
> > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]> wrote:
> >
> > > Hi all
> > >
> > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > information inside subtask and timeline in web UI.
> > >
> > > To help everyone better understand the proposal, we spent some efforts
> on
> > > making an online POC
> > >
> > > Timeline Attempt (click the vertex timeline to see the differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > >
> > > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > > differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > >
> > >
> > > The vote will last for at least 72 hours, following the consensus
> voting
> > > process.
> > >
> > > FLIP wiki:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > >
> > > Discussion thread:
> > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > >
> > > Thanks,
> > >
> > > Yadong
> > >
> >
>

--

Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: [hidden email]; [hidden email]

jing

Re: [VOTE] FLIP-100: Add Attempt Information

I agree with Jark.
The user can find the previous taskmanager according to the log link in the
history of the attempt.

The reason for adding attempts' information to the timeline is that for
some reason, the job failover failed or took a long time. This problem is
challenging to locate. As the current timeline is a summary of all attempt
information with the attempted timeline, you could find attempts with an
abnormal status.

Benchao Li <[hidden email]> 于2020年2月24日周一上午8:51写道：

> I agree with Jark.
> It's helpful to add a TM logs link for each attempt. Usually there are
> multiple TMs per host, so only host name cannot determine the real TM.
>
> Jark Wu <[hidden email]> 于2020年2月21日周五下午2:06写道：
>
> > Thanks Yadong,
> >
> > I think one of the purpose to find the failed attempts is that we want to
> > look into the logs in the failed node (node may change across attempts).
> > However, from the POC, I didn't find a log link on each attempts. Maybe
> we
> > can add such feature?
> >
> > Best,
> > Jark
> >
> > On Fri, 21 Feb 2020 at 11:51, Kurt Young <[hidden email]> wrote:
> >
> > > Hi Yadong,
> > >
> > > Thanks for the proposal, it's a useful feature, especially for batch
> > jobs.
> > > But according
> > > to the examples you gave, I can't tell whether i got required
> information
> > > from that.
> > > Can you replace the demo job to a more complex batch job and then we
> can
> > > see some
> > > differences of start/stop time of different tasks and attempts?
> > >
> > > Best,
> > > Kurt
> > >
> > >
> > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]>
> wrote:
> > >
> > > > Hi all
> > > >
> > > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > > information inside subtask and timeline in web UI.
> > > >
> > > > To help everyone better understand the proposal, we spent some
> efforts
> > on
> > > > making an online POC
> > > >
> > > > Timeline Attempt (click the vertex timeline to see the differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > >
> > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> the
> > > > differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > >
> > > >
> > > > The vote will last for at least 72 hours, following the consensus
> > voting
> > > > process.
> > > >
> > > > FLIP wiki:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > >
> > > > Discussion thread:
> > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > >
> > > > Thanks,
> > > >
> > > > Yadong
> > > >
> > >
> >
>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: [hidden email]; [hidden email]
>

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Jark and Benchao

The link to taks manager log was added to the attempt timeline now, here is
the new POC url

check it here:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline

thanks for your suggestions!

lining jing <[hidden email]> 于2020年2月24日周一下午5:16写道：

> I agree with Jark.
> The user can find the previous taskmanager according to the log link in the
> history of the attempt.
>
> The reason for adding attempts' information to the timeline is that for
> some reason, the job failover failed or took a long time. This problem is
> challenging to locate. As the current timeline is a summary of all attempt
> information with the attempted timeline, you could find attempts with an
> abnormal status.
>
>
>
> Benchao Li <[hidden email]> 于2020年2月24日周一上午8:51写道：
>
> > I agree with Jark.
> > It's helpful to add a TM logs link for each attempt. Usually there are
> > multiple TMs per host, so only host name cannot determine the real TM.
> >
> > Jark Wu <[hidden email]> 于2020年2月21日周五下午2:06写道：
> >
> > > Thanks Yadong,
> > >
> > > I think one of the purpose to find the failed attempts is that we want
> to
> > > look into the logs in the failed node (node may change across
> attempts).
> > > However, from the POC, I didn't find a log link on each attempts. Maybe
> > we
> > > can add such feature?
> > >
> > > Best,
> > > Jark
> > >
> > > On Fri, 21 Feb 2020 at 11:51, Kurt Young <[hidden email]> wrote:
> > >
> > > > Hi Yadong,
> > > >
> > > > Thanks for the proposal, it's a useful feature, especially for batch
> > > jobs.
> > > > But according
> > > > to the examples you gave, I can't tell whether i got required
> > information
> > > > from that.
> > > > Can you replace the demo job to a more complex batch job and then we
> > can
> > > > see some
> > > > differences of start/stop time of different tasks and attempts?
> > > >
> > > > Best,
> > > > Kurt
> > > >
> > > >
> > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]>
> > wrote:
> > > >
> > > > > Hi all
> > > > >
> > > > > I want to start the vote for FLIP-100, which proposes to add
> attempt
> > > > > information inside subtask and timeline in web UI.
> > > > >
> > > > > To help everyone better understand the proposal, we spent some
> > efforts
> > > on
> > > > > making an online POC
> > > > >
> > > > > Timeline Attempt (click the vertex timeline to see the
> differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > >
> > > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> > the
> > > > > differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > >
> > > > >
> > > > > The vote will last for at least 72 hours, following the consensus
> > > voting
> > > > > process.
> > > > >
> > > > > FLIP wiki:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > >
> > > > > Discussion thread:
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Yadong
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Benchao Li
> > School of Electronics Engineering and Computer Science, Peking University
> > Tel:+86-15650713730
> > Email: [hidden email]; [hidden email]
> >
>

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

In reply to this post by Kurt Young

Hi Kurt

There will be no differences between batch jobs and stream jobs in
subtask-attempt level in the UI
The only differences are in the vertex timeline, I have added a screenshot
of the batch job in the FLIP-100 since the batch job will disappear from
the list after it finished soon.
here is the link:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information

Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：

Benchao Li

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Yadong,

Thanks for the updating. LGTM now.

+1 (non-binding)

Yadong Xie <[hidden email]> 于2020年2月25日周二下午4:41写道：

> Hi Kurt
>
> There will be no differences between batch jobs and stream jobs in
> subtask-attempt level in the UI
> The only differences are in the vertex timeline, I have added a screenshot
> of the batch job in the FLIP-100 since the batch job will disappear from
> the list after it finished soon.
> here is the link:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
>
> Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：
>
> > Hi Yadong,
> >
> > Thanks for the proposal, it's a useful feature, especially for batch
> jobs.
> > But according
> > to the examples you gave, I can't tell whether i got required information
> > from that.
> > Can you replace the demo job to a more complex batch job and then we can
> > see some
> > differences of start/stop time of different tasks and attempts?
> >
> > Best,
> > Kurt
> >
> >
> > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]> wrote:
> >
> > > Hi all
> > >
> > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > information inside subtask and timeline in web UI.
> > >
> > > To help everyone better understand the proposal, we spent some efforts
> on
> > > making an online POC
> > >
> > > Timeline Attempt (click the vertex timeline to see the differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > >
> > > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > > differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > >
> > >
> > > The vote will last for at least 72 hours, following the consensus
> voting
> > > process.
> > >
> > > FLIP wiki:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > >
> > > Discussion thread:
> > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > >
> > > Thanks,
> > >
> > > Yadong
> > >
> >
>

--

Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: [hidden email]; [hidden email]

Till Rohrmann

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Yadong,

thanks for creating this FLIP. I like the idea to make the web-ui
information richer wrt to subtask attempt information.

I have a comment concerning the SubtasksTimesHandler: Should we change the
response type SubtasksTimeInfo so that it simply contains an
array of SubtaskTimeInfo? One could add an attempt range path parameter to
the SubtasksTimesHandler to be able to control which attempts will be
returned.

Cheers,
Till

On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <[hidden email]> wrote:

> Hi Yadong,
>
> Thanks for the updating. LGTM now.
>
> +1 (non-binding)
>
> Yadong Xie <[hidden email]> 于2020年2月25日周二下午4:41写道：
>
> > Hi Kurt
> >
> > There will be no differences between batch jobs and stream jobs in
> > subtask-attempt level in the UI
> > The only differences are in the vertex timeline, I have added a
> screenshot
> > of the batch job in the FLIP-100 since the batch job will disappear from
> > the list after it finished soon.
> > here is the link:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> >
> > Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：
> >
> > > Hi Yadong,
> > >
> > > Thanks for the proposal, it's a useful feature, especially for batch
> > jobs.
> > > But according
> > > to the examples you gave, I can't tell whether i got required
> information
> > > from that.
> > > Can you replace the demo job to a more complex batch job and then we
> can
> > > see some
> > > differences of start/stop time of different tasks and attempts?
> > >
> > > Best,
> > > Kurt
> > >
> > >
> > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]>
> wrote:
> > >
> > > > Hi all
> > > >
> > > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > > information inside subtask and timeline in web UI.
> > > >
> > > > To help everyone better understand the proposal, we spent some
> efforts
> > on
> > > > making an online POC
> > > >
> > > > Timeline Attempt (click the vertex timeline to see the differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > >
> > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> the
> > > > differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > >
> > > >
> > > > The vote will last for at least 72 hours, following the consensus
> > voting
> > > > process.
> > > >
> > > > FLIP wiki:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > >
> > > > Discussion thread:
> > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > >
> > > > Thanks,
> > > >
> > > > Yadong
> > > >
> > >
> >
>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: [hidden email]; [hidden email]
>

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Till

Thanks for your comments.

> I have a comment concerning the SubtasksTimesHandler....

It would be much easier for the frontend to handle a large amount of data
if we have a rest API parameter filter, but in my opinion, the attempt list
data is not large enough that we have to rely on the rest API parameters
paging, we still can handle them all in the frontend.

Users can filter the attempt list by the status(scheduled/created/deploying
and so on) and other keywords(attempt_id and so on) directly in the
frontend since all data are listed from the rest API.
If we move some of the filter parameters to the rest API path parameter,
all the other filter parameters need to be moved too.

I suggest adding an attempt id filter in the UI to help users filter the
desired attempt, and all the filtering process is running inside the
browser side, what do you think about this?

Till Rohrmann <[hidden email]> 于2020年2月25日周二下午11:40写道：

> Hi Yadong,
>
> thanks for creating this FLIP. I like the idea to make the web-ui
> information richer wrt to subtask attempt information.
>
> I have a comment concerning the SubtasksTimesHandler: Should we change the
> response type SubtasksTimeInfo so that it simply contains an
> array of SubtaskTimeInfo? One could add an attempt range path parameter to
> the SubtasksTimesHandler to be able to control which attempts will be
> returned.
>
> Cheers,
> Till
>
> On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <[hidden email]> wrote:
>
> > Hi Yadong,
> >
> > Thanks for the updating. LGTM now.
> >
> > +1 (non-binding)
> >
> > Yadong Xie <[hidden email]> 于2020年2月25日周二下午4:41写道：
> >
> > > Hi Kurt
> > >
> > > There will be no differences between batch jobs and stream jobs in
> > > subtask-attempt level in the UI
> > > The only differences are in the vertex timeline, I have added a
> > screenshot
> > > of the batch job in the FLIP-100 since the batch job will disappear
> from
> > > the list after it finished soon.
> > > here is the link:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > >
> > >
> > > Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：
> > >
> > > > Hi Yadong,
> > > >
> > > > Thanks for the proposal, it's a useful feature, especially for batch
> > > jobs.
> > > > But according
> > > > to the examples you gave, I can't tell whether i got required
> > information
> > > > from that.
> > > > Can you replace the demo job to a more complex batch job and then we
> > can
> > > > see some
> > > > differences of start/stop time of different tasks and attempts?
> > > >
> > > > Best,
> > > > Kurt
> > > >
> > > >
> > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]>
> > wrote:
> > > >
> > > > > Hi all
> > > > >
> > > > > I want to start the vote for FLIP-100, which proposes to add
> attempt
> > > > > information inside subtask and timeline in web UI.
> > > > >
> > > > > To help everyone better understand the proposal, we spent some
> > efforts
> > > on
> > > > > making an online POC
> > > > >
> > > > > Timeline Attempt (click the vertex timeline to see the
> differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > >
> > > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> > the
> > > > > differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > >
> > > > >
> > > > > The vote will last for at least 72 hours, following the consensus
> > > voting
> > > > > process.
> > > > >
> > > > > FLIP wiki:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > >
> > > > > Discussion thread:
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Yadong
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Benchao Li
> > School of Electronics Engineering and Computer Science, Peking University
> > Tel:+86-15650713730
> > Email: [hidden email]; [hidden email]
> >
>

Till Rohrmann

Re: [VOTE] FLIP-100: Add Attempt Information

Fair enough. If this should become a problem we could introduce it later as
well.

What about changing the SubtasksTimeInfo response type into an array of
SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the previous
attempts.

Cheers,
Till

On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <[hidden email]> wrote:

> Hi Till
>
> Thanks for your comments.
>
> > I have a comment concerning the SubtasksTimesHandler....
>
> It would be much easier for the frontend to handle a large amount of data
> if we have a rest API parameter filter, but in my opinion, the attempt list
> data is not large enough that we have to rely on the rest API parameters
> paging, we still can handle them all in the frontend.
>
> Users can filter the attempt list by the status(scheduled/created/deploying
> and so on) and other keywords(attempt_id and so on) directly in the
> frontend since all data are listed from the rest API.
> If we move some of the filter parameters to the rest API path parameter,
> all the other filter parameters need to be moved too.
>
> I suggest adding an attempt id filter in the UI to help users filter the
> desired attempt, and all the filtering process is running inside the
> browser side, what do you think about this?
>
>
>
>
> Till Rohrmann <[hidden email]> 于2020年2月25日周二下午11:40写道：
>
> > Hi Yadong,
> >
> > thanks for creating this FLIP. I like the idea to make the web-ui
> > information richer wrt to subtask attempt information.
> >
> > I have a comment concerning the SubtasksTimesHandler: Should we change
> the
> > response type SubtasksTimeInfo so that it simply contains an
> > array of SubtaskTimeInfo? One could add an attempt range path parameter
> to
> > the SubtasksTimesHandler to be able to control which attempts will be
> > returned.
> >
> > Cheers,
> > Till
> >
> > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <[hidden email]> wrote:
> >
> > > Hi Yadong,
> > >
> > > Thanks for the updating. LGTM now.
> > >
> > > +1 (non-binding)
> > >
> > > Yadong Xie <[hidden email]> 于2020年2月25日周二下午4:41写道：
> > >
> > > > Hi Kurt
> > > >
> > > > There will be no differences between batch jobs and stream jobs in
> > > > subtask-attempt level in the UI
> > > > The only differences are in the vertex timeline, I have added a
> > > screenshot
> > > > of the batch job in the FLIP-100 since the batch job will disappear
> > from
> > > > the list after it finished soon.
> > > > here is the link:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > >
> > > >
> > > > Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：
> > > >
> > > > > Hi Yadong,
> > > > >
> > > > > Thanks for the proposal, it's a useful feature, especially for
> batch
> > > > jobs.
> > > > > But according
> > > > > to the examples you gave, I can't tell whether i got required
> > > information
> > > > > from that.
> > > > > Can you replace the demo job to a more complex batch job and then
> we
> > > can
> > > > > see some
> > > > > differences of start/stop time of different tasks and attempts?
> > > > >
> > > > > Best,
> > > > > Kurt
> > > > >
> > > > >
> > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Hi all
> > > > > >
> > > > > > I want to start the vote for FLIP-100, which proposes to add
> > attempt
> > > > > > information inside subtask and timeline in web UI.
> > > > > >
> > > > > > To help everyone better understand the proposal, we spent some
> > > efforts
> > > > on
> > > > > > making an online POC
> > > > > >
> > > > > > Timeline Attempt (click the vertex timeline to see the
> > differences):
> > > > > > previous web:
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > POC web:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > >
> > > > > > Subtask Attempt (click the vertex and switch to subtask tab to
> see
> > > the
> > > > > > differences):
> > > > > > previous web:
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > POC web:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > >
> > > > > >
> > > > > > The vote will last for at least 72 hours, following the consensus
> > > > voting
> > > > > > process.
> > > > > >
> > > > > > FLIP wiki:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > >
> > > > > > Discussion thread:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Yadong
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Benchao Li
> > > School of Electronics Engineering and Computer Science, Peking
> University
> > > Tel:+86-15650713730
> > > Email: [hidden email]; [hidden email]
> > >
> >
>

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Till

We keep response a flattened SubtaskTimeInfo and an array of
SubtaskTimeInfo to keep the restAPI backward compatible, since Flink users
may still need the API('/jobs/{jobId}/vertices/{vertexId}/subtasktimes') to
get the flattened SubtaskTimeInfo.

If we want to get an array of SubtaskTimeInfo in the API, a new URL needs
to be created other than reuse the old one.

Both solutions are ok for me. What do you think about it?

Till Rohrmann <[hidden email]> 于2020年2月26日周三下午10:53写道：

> Fair enough. If this should become a problem we could introduce it later as
> well.
>
> What about changing the SubtasksTimeInfo response type into an array of
> SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
> flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the previous
> attempts.
>
> Cheers,
> Till
>
> On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <[hidden email]> wrote:
>
> > Hi Till
> >
> > Thanks for your comments.
> >
> > > I have a comment concerning the SubtasksTimesHandler....
> >
> > It would be much easier for the frontend to handle a large amount of data
> > if we have a rest API parameter filter, but in my opinion, the attempt
> list
> > data is not large enough that we have to rely on the rest API parameters
> > paging, we still can handle them all in the frontend.
> >
> > Users can filter the attempt list by the
> status(scheduled/created/deploying
> > and so on) and other keywords(attempt_id and so on) directly in the
> > frontend since all data are listed from the rest API.
> > If we move some of the filter parameters to the rest API path parameter,
> > all the other filter parameters need to be moved too.
> >
> > I suggest adding an attempt id filter in the UI to help users filter the
> > desired attempt, and all the filtering process is running inside the
> > browser side, what do you think about this?
> >
> >
> >
> >
> > Till Rohrmann <[hidden email]> 于2020年2月25日周二下午11:40写道：
> >
> > > Hi Yadong,
> > >
> > > thanks for creating this FLIP. I like the idea to make the web-ui
> > > information richer wrt to subtask attempt information.
> > >
> > > I have a comment concerning the SubtasksTimesHandler: Should we change
> > the
> > > response type SubtasksTimeInfo so that it simply contains an
> > > array of SubtaskTimeInfo? One could add an attempt range path parameter
> > to
> > > the SubtasksTimesHandler to be able to control which attempts will be
> > > returned.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <[hidden email]>
> wrote:
> > >
> > > > Hi Yadong,
> > > >
> > > > Thanks for the updating. LGTM now.
> > > >
> > > > +1 (non-binding)
> > > >
> > > > Yadong Xie <[hidden email]> 于2020年2月25日周二下午4:41写道：
> > > >
> > > > > Hi Kurt
> > > > >
> > > > > There will be no differences between batch jobs and stream jobs in
> > > > > subtask-attempt level in the UI
> > > > > The only differences are in the vertex timeline, I have added a
> > > > screenshot
> > > > > of the batch job in the FLIP-100 since the batch job will disappear
> > > from
> > > > > the list after it finished soon.
> > > > > here is the link:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > >
> > > > >
> > > > > Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：
> > > > >
> > > > > > Hi Yadong,
> > > > > >
> > > > > > Thanks for the proposal, it's a useful feature, especially for
> > batch
> > > > > jobs.
> > > > > > But according
> > > > > > to the examples you gave, I can't tell whether i got required
> > > > information
> > > > > > from that.
> > > > > > Can you replace the demo job to a more complex batch job and then
> > we
> > > > can
> > > > > > see some
> > > > > > differences of start/stop time of different tasks and attempts?
> > > > > >
> > > > > > Best,
> > > > > > Kurt
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <[hidden email]>
> > > > wrote:
> > > > > >
> > > > > > > Hi all
> > > > > > >
> > > > > > > I want to start the vote for FLIP-100, which proposes to add
> > > attempt
> > > > > > > information inside subtask and timeline in web UI.
> > > > > > >
> > > > > > > To help everyone better understand the proposal, we spent some
> > > > efforts
> > > > > on
> > > > > > > making an online POC
> > > > > > >
> > > > > > > Timeline Attempt (click the vertex timeline to see the
> > > differences):
> > > > > > > previous web:
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > POC web:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > >
> > > > > > > Subtask Attempt (click the vertex and switch to subtask tab to
> > see
> > > > the
> > > > > > > differences):
> > > > > > > previous web:
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > POC web:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > >
> > > > > > >
> > > > > > > The vote will last for at least 72 hours, following the
> consensus
> > > > > voting
> > > > > > > process.
> > > > > > >
> > > > > > > FLIP wiki:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > >
> > > > > > > Discussion thread:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Yadong
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Benchao Li
> > > > School of Electronics Engineering and Computer Science, Peking
> > University
> > > > Tel:+86-15650713730
> > > > Email: [hidden email]; [hidden email]
> > > >
> > >
> >
>

Till Rohrmann

Re: [VOTE] FLIP-100: Add Attempt Information

I think your proposal is better because then we don't have to bump the REST
API version.

Cheers,
Till

On Thu, Feb 27, 2020 at 8:02 AM Yadong Xie <[hidden email]> wrote:

> Hi Till
>
> We keep response a flattened SubtaskTimeInfo and an array of
> SubtaskTimeInfo to keep the restAPI backward compatible, since Flink users
> may still need the API('/jobs/{jobId}/vertices/{vertexId}/subtasktimes') to
> get the flattened SubtaskTimeInfo.
>
> If we want to get an array of SubtaskTimeInfo in the API, a new URL needs
> to be created other than reuse the old one.
>
> Both solutions are ok for me. What do you think about it?
>
> Till Rohrmann <[hidden email]> 于2020年2月26日周三下午10:53写道：
>
> > Fair enough. If this should become a problem we could introduce it later
> as
> > well.
> >
> > What about changing the SubtasksTimeInfo response type into an array of
> > SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
> > flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the
> previous
> > attempts.
> >
> > Cheers,
> > Till
> >
> > On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <[hidden email]> wrote:
> >
> > > Hi Till
> > >
> > > Thanks for your comments.
> > >
> > > > I have a comment concerning the SubtasksTimesHandler....
> > >
> > > It would be much easier for the frontend to handle a large amount of
> data
> > > if we have a rest API parameter filter, but in my opinion, the attempt
> > list
> > > data is not large enough that we have to rely on the rest API
> parameters
> > > paging, we still can handle them all in the frontend.
> > >
> > > Users can filter the attempt list by the
> > status(scheduled/created/deploying
> > > and so on) and other keywords(attempt_id and so on) directly in the
> > > frontend since all data are listed from the rest API.
> > > If we move some of the filter parameters to the rest API path
> parameter,
> > > all the other filter parameters need to be moved too.
> > >
> > > I suggest adding an attempt id filter in the UI to help users filter
> the
> > > desired attempt, and all the filtering process is running inside the
> > > browser side, what do you think about this?
> > >
> > >
> > >
> > >
> > > Till Rohrmann <[hidden email]> 于2020年2月25日周二下午11:40写道：
> > >
> > > > Hi Yadong,
> > > >
> > > > thanks for creating this FLIP. I like the idea to make the web-ui
> > > > information richer wrt to subtask attempt information.
> > > >
> > > > I have a comment concerning the SubtasksTimesHandler: Should we
> change
> > > the
> > > > response type SubtasksTimeInfo so that it simply contains an
> > > > array of SubtaskTimeInfo? One could add an attempt range path
> parameter
> > > to
> > > > the SubtasksTimesHandler to be able to control which attempts will be
> > > > returned.
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <[hidden email]>
> > wrote:
> > > >
> > > > > Hi Yadong,
> > > > >
> > > > > Thanks for the updating. LGTM now.
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Yadong Xie <[hidden email]> 于2020年2月25日周二下午4:41写道：
> > > > >
> > > > > > Hi Kurt
> > > > > >
> > > > > > There will be no differences between batch jobs and stream jobs
> in
> > > > > > subtask-attempt level in the UI
> > > > > > The only differences are in the vertex timeline, I have added a
> > > > > screenshot
> > > > > > of the batch job in the FLIP-100 since the batch job will
> disappear
> > > > from
> > > > > > the list after it finished soon.
> > > > > > here is the link:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > >
> > > > > >
> > > > > > Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：
> > > > > >
> > > > > > > Hi Yadong,
> > > > > > >
> > > > > > > Thanks for the proposal, it's a useful feature, especially for
> > > batch
> > > > > > jobs.
> > > > > > > But according
> > > > > > > to the examples you gave, I can't tell whether i got required
> > > > > information
> > > > > > > from that.
> > > > > > > Can you replace the demo job to a more complex batch job and
> then
> > > we
> > > > > can
> > > > > > > see some
> > > > > > > differences of start/stop time of different tasks and attempts?
> > > > > > >
> > > > > > > Best,
> > > > > > > Kurt
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <
> [hidden email]>
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi all
> > > > > > > >
> > > > > > > > I want to start the vote for FLIP-100, which proposes to add
> > > > attempt
> > > > > > > > information inside subtask and timeline in web UI.
> > > > > > > >
> > > > > > > > To help everyone better understand the proposal, we spent
> some
> > > > > efforts
> > > > > > on
> > > > > > > > making an online POC
> > > > > > > >
> > > > > > > > Timeline Attempt (click the vertex timeline to see the
> > > > differences):
> > > > > > > > previous web:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > > POC web:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > >
> > > > > > > > Subtask Attempt (click the vertex and switch to subtask tab
> to
> > > see
> > > > > the
> > > > > > > > differences):
> > > > > > > > previous web:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > > POC web:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > >
> > > > > > > >
> > > > > > > > The vote will last for at least 72 hours, following the
> > consensus
> > > > > > voting
> > > > > > > > process.
> > > > > > > >
> > > > > > > > FLIP wiki:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > > >
> > > > > > > > Discussion thread:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Yadong
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Benchao Li
> > > > > School of Electronics Engineering and Computer Science, Peking
> > > University
> > > > > Tel:+86-15650713730
> > > > > Email: [hidden email]; [hidden email]
> > > > >
> > > >
> > >
> >
>

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

Hi everyone

A subtask filter and an attempt id filter was added to the attempt timeline
page now

check the POC here:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline

FLIP doc was also updated:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information

Till Rohrmann <[hidden email]> 于2020年2月27日周四下午9:49写道：

> I think your proposal is better because then we don't have to bump the REST
> API version.
>
> Cheers,
> Till
>
> On Thu, Feb 27, 2020 at 8:02 AM Yadong Xie <[hidden email]> wrote:
>
> > Hi Till
> >
> > We keep response a flattened SubtaskTimeInfo and an array of
> > SubtaskTimeInfo to keep the restAPI backward compatible, since Flink
> users
> > may still need the API('/jobs/{jobId}/vertices/{vertexId}/subtasktimes')
> to
> > get the flattened SubtaskTimeInfo.
> >
> > If we want to get an array of SubtaskTimeInfo in the API, a new URL needs
> > to be created other than reuse the old one.
> >
> > Both solutions are ok for me. What do you think about it?
> >
> > Till Rohrmann <[hidden email]> 于2020年2月26日周三下午10:53写道：
> >
> > > Fair enough. If this should become a problem we could introduce it
> later
> > as
> > > well.
> > >
> > > What about changing the SubtasksTimeInfo response type into an array of
> > > SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
> > > flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the
> > previous
> > > attempts.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <[hidden email]>
> wrote:
> > >
> > > > Hi Till
> > > >
> > > > Thanks for your comments.
> > > >
> > > > > I have a comment concerning the SubtasksTimesHandler....
> > > >
> > > > It would be much easier for the frontend to handle a large amount of
> > data
> > > > if we have a rest API parameter filter, but in my opinion, the
> attempt
> > > list
> > > > data is not large enough that we have to rely on the rest API
> > parameters
> > > > paging, we still can handle them all in the frontend.
> > > >
> > > > Users can filter the attempt list by the
> > > status(scheduled/created/deploying
> > > > and so on) and other keywords(attempt_id and so on) directly in the
> > > > frontend since all data are listed from the rest API.
> > > > If we move some of the filter parameters to the rest API path
> > parameter,
> > > > all the other filter parameters need to be moved too.
> > > >
> > > > I suggest adding an attempt id filter in the UI to help users filter
> > the
> > > > desired attempt, and all the filtering process is running inside the
> > > > browser side, what do you think about this?
> > > >
> > > >
> > > >
> > > >
> > > > Till Rohrmann <[hidden email]> 于2020年2月25日周二下午11:40写道：
> > > >
> > > > > Hi Yadong,
> > > > >
> > > > > thanks for creating this FLIP. I like the idea to make the web-ui
> > > > > information richer wrt to subtask attempt information.
> > > > >
> > > > > I have a comment concerning the SubtasksTimesHandler: Should we
> > change
> > > > the
> > > > > response type SubtasksTimeInfo so that it simply contains an
> > > > > array of SubtaskTimeInfo? One could add an attempt range path
> > parameter
> > > > to
> > > > > the SubtasksTimesHandler to be able to control which attempts will
> be
> > > > > returned.
> > > > >
> > > > > Cheers,
> > > > > Till
> > > > >
> > > > > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Hi Yadong,
> > > > > >
> > > > > > Thanks for the updating. LGTM now.
> > > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Yadong Xie <[hidden email]> 于2020年2月25日周二下午4:41写道：
> > > > > >
> > > > > > > Hi Kurt
> > > > > > >
> > > > > > > There will be no differences between batch jobs and stream jobs
> > in
> > > > > > > subtask-attempt level in the UI
> > > > > > > The only differences are in the vertex timeline, I have added a
> > > > > > screenshot
> > > > > > > of the batch job in the FLIP-100 since the batch job will
> > disappear
> > > > > from
> > > > > > > the list after it finished soon.
> > > > > > > here is the link:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > >
> > > > > > >
> > > > > > > Kurt Young <[hidden email]> 于2020年2月21日周五上午11:51写道：
> > > > > > >
> > > > > > > > Hi Yadong,
> > > > > > > >
> > > > > > > > Thanks for the proposal, it's a useful feature, especially
> for
> > > > batch
> > > > > > > jobs.
> > > > > > > > But according
> > > > > > > > to the examples you gave, I can't tell whether i got required
> > > > > > information
> > > > > > > > from that.
> > > > > > > > Can you replace the demo job to a more complex batch job and
> > then
> > > > we
> > > > > > can
> > > > > > > > see some
> > > > > > > > differences of start/stop time of different tasks and
> attempts?
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Kurt
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <
> > [hidden email]>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all
> > > > > > > > >
> > > > > > > > > I want to start the vote for FLIP-100, which proposes to
> add
> > > > > attempt
> > > > > > > > > information inside subtask and timeline in web UI.
> > > > > > > > >
> > > > > > > > > To help everyone better understand the proposal, we spent
> > some
> > > > > > efforts
> > > > > > > on
> > > > > > > > > making an online POC
> > > > > > > > >
> > > > > > > > > Timeline Attempt (click the vertex timeline to see the
> > > > > differences):
> > > > > > > > > previous web:
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > > > POC web:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > > >
> > > > > > > > > Subtask Attempt (click the vertex and switch to subtask tab
> > to
> > > > see
> > > > > > the
> > > > > > > > > differences):
> > > > > > > > > previous web:
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > > > POC web:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > The vote will last for at least 72 hours, following the
> > > consensus
> > > > > > > voting
> > > > > > > > > process.
> > > > > > > > >
> > > > > > > > > FLIP wiki:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > > > >
> > > > > > > > > Discussion thread:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Yadong
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Benchao Li
> > > > > > School of Electronics Engineering and Computer Science, Peking
> > > > University
> > > > > > Tel:+86-15650713730
> > > > > > Email: [hidden email]; [hidden email]
> > > > > >
> > > > >
> > > >
> > >
> >
>

Gary Yao-4

Re: [VOTE] FLIP-100: Add Attempt Information

In reply to this post by Yadong Xie

Hi Yadong,

Thanks for driving this FLIP. I have a few questions/remarks:

* Why are we duplicating the subtask index in the objects that are
stored in the attempts-time-info array? I thought that all objects in the
same array share the same subtask index.
* Are we confident that the attempts-time-info array does not grow too
large during the lifetime of a job? Should the size of the array be limited?
* Have we considered placing the historic attempts in the same array as
the current attempts, i.e., flatten the arrays? One could toggle the
historic attempts on and off with a query parameter.
* I think 'attempt-history' would be a better name instead of
'attempts-time-info'.

Let me know what you think.

Best,
Gary

On Thu, Feb 20, 2020 at 10:46 AM Yadong Xie <[hidden email]> wrote:

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

> * Why are we duplicating the subtask index in the objects that are
stored in the attempts-time-info array? I thought that all objects in the
same array share the same subtask index.
yes, I agree with you that subtask could be removed in the
`attempts-time-info`

> * Are we confident that the attempts-time-info array does not grow too
large during the lifetime of a job? Should the size of the array be limited?

no, I think it is a similar situation to taskmanager/jobmanager log API,
all data response from the rest API could grow too large, but add query
params in the RestAPI would also bring pressure to the server.

> * Have we considered placing the historic attempts in the same array as
the current attempts, i.e., flatten the arrays? One could toggle the
historic attempts on and off with a query parameter.

it is a good idea considering the previous situation, this would be useful
not blocking the previous feature even if the history becomes too large, we
will implement this and update the POC later.

> * I think 'attempt-history' would be a better name instead of
'attempts-time-info'.

Yes, I agree with you, we will update it later.

Gary Yao <[hidden email]> 于2020年3月2日周一下午10:50写道：

> Hi Yadong,
>
> Thanks for driving this FLIP. I have a few questions/remarks:
>
> * Why are we duplicating the subtask index in the objects that are
> stored in the attempts-time-info array? I thought that all objects in the
> same array share the same subtask index.
> * Are we confident that the attempts-time-info array does not grow too
> large during the lifetime of a job? Should the size of the array be
> limited?
> * Have we considered placing the historic attempts in the same array as
> the current attempts, i.e., flatten the arrays? One could toggle the
> historic attempts on and off with a query parameter.
> * I think 'attempt-history' would be a better name instead of
> 'attempts-time-info'.
>
> Let me know what you think.
>
> Best,
> Gary
>
> On Thu, Feb 20, 2020 at 10:46 AM Yadong Xie <[hidden email]> wrote:
>
> > Hi all
> >
> > I want to start the vote for FLIP-100, which proposes to add attempt
> > information inside subtask and timeline in web UI.
> >
> > To help everyone better understand the proposal, we spent some efforts on
> > making an online POC
> >
> > Timeline Attempt (click the vertex timeline to see the differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >
> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >
> >
> > The vote will last for at least 72 hours, following the consensus voting
> > process.
> >
> > FLIP wiki:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> > Discussion thread:
> >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >
> > Thanks,
> >
> > Yadong
> >
>

Chesnay Schepler-3

Re: [VOTE] FLIP-100: Add Attempt Information

In reply to this post by Yadong Xie

I suggest to cancel this vote.
Several discussion items have been brought up during the vote, some of
which are still unresolved, others which resulted in changes to the
proposal.

My conclusion is that this proposal needs more discussions.

On 20/02/2020 10:46, Yadong Xie wrote:

> Hi all
>
> I want to start the vote for FLIP-100, which proposes to add attempt
> information inside subtask and timeline in web UI.
>
> To help everyone better understand the proposal, we spent some efforts on
> making an online POC
>
> Timeline Attempt (click the vertex timeline to see the differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> POC web:
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>
> Subtask Attempt (click the vertex and switch to subtask tab to see the
> differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> POC web:
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>
>
> The vote will last for at least 72 hours, following the consensus voting
> process.
>
> FLIP wiki:
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
> Discussion thread:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>
> Thanks,
>
> Yadong
>

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Chesnay

most discussions in this vote are about the more feature/demo request in
POC or discussion about response format, the main proposal the web UI part
which is not changed

and the discussion about the response is converging, the response format
discussion could happen either here or at the code review stage, which
would be a minor change from my point of view.

Chesnay Schepler <[hidden email]> 于2020年3月3日周二下午8:20写道：

> I suggest to cancel this vote.
> Several discussion items have been brought up during the vote, some of
> which are still unresolved, others which resulted in changes to the
> proposal.
>
> My conclusion is that this proposal needs more discussions.
>
>
> On 20/02/2020 10:46, Yadong Xie wrote:
> > Hi all
> >
> > I want to start the vote for FLIP-100, which proposes to add attempt
> > information inside subtask and timeline in web UI.
> >
> > To help everyone better understand the proposal, we spent some efforts on
> > making an online POC
> >
> > Timeline Attempt (click the vertex timeline to see the differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > POC web:
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >
> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > POC web:
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >
> >
> > The vote will last for at least 72 hours, following the consensus voting
> > process.
> >
> > FLIP wiki:
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> > Discussion thread:
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >
> > Thanks,
> >
> > Yadong
> >
>
>

Yadong Xie

Re: [VOTE] FLIP-100: Add Attempt Information

Hi all

The rest API part has been updated with Gary and Till's suggestions
here is the link:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information

Yadong Xie <[hidden email]> 于2020年3月3日周二下午9:14写道：

> Hi Chesnay
>
> most discussions in this vote are about the more feature/demo request in
> POC or discussion about response format, the main proposal the web UI part
> which is not changed
>
> and the discussion about the response is converging, the response format
> discussion could happen either here or at the code review stage, which
> would be a minor change from my point of view.
>
> Chesnay Schepler <[hidden email]> 于2020年3月3日周二下午8:20写道：
>
>> I suggest to cancel this vote.
>> Several discussion items have been brought up during the vote, some of
>> which are still unresolved, others which resulted in changes to the
>> proposal.
>>
>> My conclusion is that this proposal needs more discussions.
>>
>>
>> On 20/02/2020 10:46, Yadong Xie wrote:
>> > Hi all
>> >
>> > I want to start the vote for FLIP-100, which proposes to add attempt
>> > information inside subtask and timeline in web UI.
>> >
>> > To help everyone better understand the proposal, we spent some efforts
>> on
>> > making an online POC
>> >
>> > Timeline Attempt (click the vertex timeline to see the differences):
>> > previous web:
>> >
>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
>> > POC web:
>> >
>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>> >
>> > Subtask Attempt (click the vertex and switch to subtask tab to see the
>> > differences):
>> > previous web:
>> >
>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
>> > POC web:
>> >
>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>> >
>> >
>> > The vote will last for at least 72 hours, following the consensus voting
>> > process.
>> >
>> > FLIP wiki:
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>> >
>> > Discussion thread:
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>> >
>> > Thanks,
>> >
>> > Yadong
>> >
>>
>>

Gary Yao-4

Re: [VOTE] FLIP-100: Add Attempt Information

Hi Yadong,

Thank you for updating the wiki page.

Only one minor suggestion – I would change:

> If show-history is true return the information of attempt.

to

> If show-history is true, information for all attempts including
previous ones will be returned

That being said, FLIP-100 looks good to me. From my side there is not
anything
else to discuss.

@Kurt and @Jark: Can you look into the improvements that have been made
since
the last time you looked at the PoC? If you are happy, we can restart the
voting.

Best,
Gary

On Tue, Mar 3, 2020 at 2:34 PM Yadong Xie <[hidden email]> wrote:

> Hi all
>
> The rest API part has been updated with Gary and Till's suggestions
> here is the link:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
> Yadong Xie <[hidden email]> 于2020年3月3日周二下午9:14写道：
>
> > Hi Chesnay
> >
> > most discussions in this vote are about the more feature/demo request in
> > POC or discussion about response format, the main proposal the web UI
> part
> > which is not changed
> >
> > and the discussion about the response is converging, the response format
> > discussion could happen either here or at the code review stage, which
> > would be a minor change from my point of view.
> >
> > Chesnay Schepler <[hidden email]> 于2020年3月3日周二下午8:20写道：
> >
> >> I suggest to cancel this vote.
> >> Several discussion items have been brought up during the vote, some of
> >> which are still unresolved, others which resulted in changes to the
> >> proposal.
> >>
> >> My conclusion is that this proposal needs more discussions.
> >>
> >>
> >> On 20/02/2020 10:46, Yadong Xie wrote:
> >> > Hi all
> >> >
> >> > I want to start the vote for FLIP-100, which proposes to add attempt
> >> > information inside subtask and timeline in web UI.
> >> >
> >> > To help everyone better understand the proposal, we spent some efforts
> >> on
> >> > making an online POC
> >> >
> >> > Timeline Attempt (click the vertex timeline to see the differences):
> >> > previous web:
> >> >
> >>
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> >> > POC web:
> >> >
> >>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >> >
> >> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> >> > differences):
> >> > previous web:
> >> >
> >>
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> >> > POC web:
> >> >
> >>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >> >
> >> >
> >> > The vote will last for at least 72 hours, following the consensus
> voting
> >> > process.
> >> >
> >> > FLIP wiki:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >> >
> >> > Discussion thread:
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >> >
> >> > Thanks,
> >> >
> >> > Yadong
> >> >
> >>
> >>
>