Hi all
Are there any plans to upgrade the webclient interface? It could certainly use an upgrade compared to the much better dashboard . If so, is integrating them into one the best way to go? We can have a tab under Job Manager on the dashboard. I can share screenshots of the UI I designed last night, if it's not a terrible idea to do this. :') Cheers! Sachin |
I like the idea in general! Not sure if integrating it into JobMangaer
Webinterface is good or not... What would be the pros/cons for standalone vs. integrated? -Matthias On 11/05/2015 05:23 AM, Sachin Goel wrote: > Hi all > Are there any plans to upgrade the webclient interface? It could certainly > use an upgrade compared to the much better dashboard . > If so, is integrating them into one the best way to go? We can have a tab > under Job Manager on the dashboard. > > I can share screenshots of the UI I designed last night, if it's not a > terrible idea to do this. :') > > Cheers! > Sachin > |
The reason why we didn’t want to have it in the JobManager interface by default is that it could be a security concern. I also see, however, that security in general is not very strong right now.
> On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> wrote: > > I like the idea in general! Not sure if integrating it into JobMangaer > Webinterface is good or not... What would be the pros/cons for > standalone vs. integrated? > > -Matthias > > On 11/05/2015 05:23 AM, Sachin Goel wrote: >> Hi all >> Are there any plans to upgrade the webclient interface? It could certainly >> use an upgrade compared to the much better dashboard . >> If so, is integrating them into one the best way to go? We can have a tab >> under Job Manager on the dashboard. >> >> I can share screenshots of the UI I designed last night, if it's not a >> terrible idea to do this. :') >> >> Cheers! >> Sachin >> > |
The current authentication mechanism is via htaccess, in the webclient. The
dashboard however isn't secured at all. -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek <[hidden email]> wrote: > The reason why we didn’t want to have it in the JobManager interface by > default is that it could be a security concern. I also see, however, that > security in general is not very strong right now. > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> wrote: > > > > I like the idea in general! Not sure if integrating it into JobMangaer > > Webinterface is good or not... What would be the pros/cons for > > standalone vs. integrated? > > > > -Matthias > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > >> Hi all > >> Are there any plans to upgrade the webclient interface? It could > certainly > >> use an upgrade compared to the much better dashboard . > >> If so, is integrating them into one the best way to go? We can have a > tab > >> under Job Manager on the dashboard. > >> > >> I can share screenshots of the UI I designed last night, if it's not a > >> terrible idea to do this. :') > >> > >> Cheers! > >> Sachin > >> > > > > |
Hi,
I personally like the idea of allowing users to submit a job from the web interface a lot. In particular for users new to the system, its a very convenient feature. We would also get rid of this ugly, old web client :) Regarding security: We can always add a configuration parameter which allows users to disable the feature. Please share the screenshot with us :) On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel <[hidden email]> wrote: > The current authentication mechanism is via htaccess, in the webclient. The > dashboard however isn't secured at all. > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek <[hidden email]> > wrote: > > > The reason why we didn’t want to have it in the JobManager interface by > > default is that it could be a security concern. I also see, however, that > > security in general is not very strong right now. > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> wrote: > > > > > > I like the idea in general! Not sure if integrating it into JobMangaer > > > Webinterface is good or not... What would be the pros/cons for > > > standalone vs. integrated? > > > > > > -Matthias > > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > > >> Hi all > > >> Are there any plans to upgrade the webclient interface? It could > > certainly > > >> use an upgrade compared to the much better dashboard . > > >> If so, is integrating them into one the best way to go? We can have a > > tab > > >> under Job Manager on the dashboard. > > >> > > >> I can share screenshots of the UI I designed last night, if it's not a > > >> terrible idea to do this. :') > > >> > > >> Cheers! > > >> Sachin > > >> > > > > > > > > |
+1 to remove the old web client in favor of an improved dashboard (not so
urgent however) On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger <[hidden email]> wrote: > Hi, > I personally like the idea of allowing users to submit a job from the web > interface a lot. In particular for users new to the system, its a very > convenient feature. > We would also get rid of this ugly, old web client :) > > Regarding security: We can always add a configuration parameter which > allows users to disable the feature. > > Please share the screenshot with us :) > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel <[hidden email]> > wrote: > > > The current authentication mechanism is via htaccess, in the webclient. > The > > dashboard however isn't secured at all. > > > > -- Sachin Goel > > Computer Science, IIT Delhi > > m. +91-9871457685 > > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek <[hidden email]> > > wrote: > > > > > The reason why we didn’t want to have it in the JobManager interface by > > > default is that it could be a security concern. I also see, however, > that > > > security in general is not very strong right now. > > > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> wrote: > > > > > > > > I like the idea in general! Not sure if integrating it into > JobMangaer > > > > Webinterface is good or not... What would be the pros/cons for > > > > standalone vs. integrated? > > > > > > > > -Matthias > > > > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > > > >> Hi all > > > >> Are there any plans to upgrade the webclient interface? It could > > > certainly > > > >> use an upgrade compared to the much better dashboard . > > > >> If so, is integrating them into one the best way to go? We can have > a > > > tab > > > >> under Job Manager on the dashboard. > > > >> > > > >> I can share screenshots of the UI I designed last night, if it's > not a > > > >> terrible idea to do this. :') > > > >> > > > >> Cheers! > > > >> Sachin > > > >> > > > > > > > > > > > > > |
+1 The web client needs to go.
I don't really see how how a submission page on the job manager would make security worse. At the moment, anyone who sends an Akka message to the job manager, can execute arbitrary code.. On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier <[hidden email]> wrote: > +1 to remove the old web client in favor of an improved dashboard (not so > urgent however) > > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger <[hidden email]> > wrote: > > > Hi, > > I personally like the idea of allowing users to submit a job from the web > > interface a lot. In particular for users new to the system, its a very > > convenient feature. > > We would also get rid of this ugly, old web client :) > > > > Regarding security: We can always add a configuration parameter which > > allows users to disable the feature. > > > > Please share the screenshot with us :) > > > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel <[hidden email]> > > wrote: > > > > > The current authentication mechanism is via htaccess, in the webclient. > > The > > > dashboard however isn't secured at all. > > > > > > -- Sachin Goel > > > Computer Science, IIT Delhi > > > m. +91-9871457685 > > > > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek <[hidden email]> > > > wrote: > > > > > > > The reason why we didn’t want to have it in the JobManager interface > by > > > > default is that it could be a security concern. I also see, however, > > that > > > > security in general is not very strong right now. > > > > > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> > wrote: > > > > > > > > > > I like the idea in general! Not sure if integrating it into > > JobMangaer > > > > > Webinterface is good or not... What would be the pros/cons for > > > > > standalone vs. integrated? > > > > > > > > > > -Matthias > > > > > > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > > > > >> Hi all > > > > >> Are there any plans to upgrade the webclient interface? It could > > > > certainly > > > > >> use an upgrade compared to the much better dashboard . > > > > >> If so, is integrating them into one the best way to go? We can > have > > a > > > > tab > > > > >> under Job Manager on the dashboard. > > > > >> > > > > >> I can share screenshots of the UI I designed last night, if it's > > not a > > > > >> terrible idea to do this. :') > > > > >> > > > > >> Cheers! > > > > >> Sachin > > > > >> > > > > > > > > > > > > > > > > > > > |
I agree with Max, but still, its extremely convenient (and much easier than
crafting akka messages) to have a web interface which accepts jar files and executes the main method ;) On Thu, Nov 5, 2015 at 11:42 AM, Maximilian Michels <[hidden email]> wrote: > +1 The web client needs to go. > > I don't really see how how a submission page on the job manager would make > security worse. At the moment, anyone who sends an Akka message to the job > manager, can execute arbitrary code.. > > On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier <[hidden email]> > wrote: > > > +1 to remove the old web client in favor of an improved dashboard (not so > > urgent however) > > > > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger <[hidden email]> > > wrote: > > > > > Hi, > > > I personally like the idea of allowing users to submit a job from the > web > > > interface a lot. In particular for users new to the system, its a very > > > convenient feature. > > > We would also get rid of this ugly, old web client :) > > > > > > Regarding security: We can always add a configuration parameter which > > > allows users to disable the feature. > > > > > > Please share the screenshot with us :) > > > > > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel <[hidden email] > > > > > wrote: > > > > > > > The current authentication mechanism is via htaccess, in the > webclient. > > > The > > > > dashboard however isn't secured at all. > > > > > > > > -- Sachin Goel > > > > Computer Science, IIT Delhi > > > > m. +91-9871457685 > > > > > > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < > [hidden email]> > > > > wrote: > > > > > > > > > The reason why we didn’t want to have it in the JobManager > interface > > by > > > > > default is that it could be a security concern. I also see, > however, > > > that > > > > > security in general is not very strong right now. > > > > > > > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> > > wrote: > > > > > > > > > > > > I like the idea in general! Not sure if integrating it into > > > JobMangaer > > > > > > Webinterface is good or not... What would be the pros/cons for > > > > > > standalone vs. integrated? > > > > > > > > > > > > -Matthias > > > > > > > > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > > > > > >> Hi all > > > > > >> Are there any plans to upgrade the webclient interface? It could > > > > > certainly > > > > > >> use an upgrade compared to the much better dashboard . > > > > > >> If so, is integrating them into one the best way to go? We can > > have > > > a > > > > > tab > > > > > >> under Job Manager on the dashboard. > > > > > >> > > > > > >> I can share screenshots of the UI I designed last night, if it's > > > not a > > > > > >> terrible idea to do this. :') > > > > > >> > > > > > >> Cheers! > > > > > >> Sachin > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > |
In reply to this post by mxm
Okay. I think that does it. I've already designed the basic interface, although the major part for uploading jars and actually running the job are left. :') Here are some screenshots: -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> wrote: +1 The web client needs to go. |
Hey Sachin, I think you can not upload images to apache mailing lists.
On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel <[hidden email]> wrote: > Okay. I think that does it. I've already designed the basic interface, > although the major part for uploading jars and actually running the job are > left. :') > Here are some screenshots: > [image: Inline image 2][image: Inline image 1][image: Inline image 3] > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> wrote: > >> +1 The web client needs to go. >> >> I don't really see how how a submission page on the job manager would make >> security worse. At the moment, anyone who sends an Akka message to the job >> manager, can execute arbitrary code.. >> >> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier <[hidden email] >> > >> wrote: >> >> > +1 to remove the old web client in favor of an improved dashboard (not >> so >> > urgent however) >> > >> > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger <[hidden email]> >> > wrote: >> > >> > > Hi, >> > > I personally like the idea of allowing users to submit a job from the >> web >> > > interface a lot. In particular for users new to the system, its a very >> > > convenient feature. >> > > We would also get rid of this ugly, old web client :) >> > > >> > > Regarding security: We can always add a configuration parameter which >> > > allows users to disable the feature. >> > > >> > > Please share the screenshot with us :) >> > > >> > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < >> [hidden email]> >> > > wrote: >> > > >> > > > The current authentication mechanism is via htaccess, in the >> webclient. >> > > The >> > > > dashboard however isn't secured at all. >> > > > >> > > > -- Sachin Goel >> > > > Computer Science, IIT Delhi >> > > > m. +91-9871457685 >> > > > >> > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < >> [hidden email]> >> > > > wrote: >> > > > >> > > > > The reason why we didn’t want to have it in the JobManager >> interface >> > by >> > > > > default is that it could be a security concern. I also see, >> however, >> > > that >> > > > > security in general is not very strong right now. >> > > > > >> > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> >> > wrote: >> > > > > > >> > > > > > I like the idea in general! Not sure if integrating it into >> > > JobMangaer >> > > > > > Webinterface is good or not... What would be the pros/cons for >> > > > > > standalone vs. integrated? >> > > > > > >> > > > > > -Matthias >> > > > > > >> > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: >> > > > > >> Hi all >> > > > > >> Are there any plans to upgrade the webclient interface? It >> could >> > > > > certainly >> > > > > >> use an upgrade compared to the much better dashboard . >> > > > > >> If so, is integrating them into one the best way to go? We can >> > have >> > > a >> > > > > tab >> > > > > >> under Job Manager on the dashboard. >> > > > > >> >> > > > > >> I can share screenshots of the UI I designed last night, if >> it's >> > > not a >> > > > > >> terrible idea to do this. :') >> > > > > >> >> > > > > >> Cheers! >> > > > > >> Sachin >> > > > > >> >> > > > > > >> > > > > >> > > > > >> > > > >> > > >> > >> > > |
In reply to this post by Robert Metzger
Security should definitely be in our minds big time.
We think we need to add secure cookies to akka very soon (makes akka connections encrypted) and use encryption on the netty as well. To that end, we should not introduce more loopholes now. We can add an upload/execute section to the dashboard, but it would need to be activated / deactivated by a flag in the cluster, so secure installations can suppress this functionality. I suggest to have it activated by default in standalone mode / YARN session mode, and deactivated by default in the YARN-per-job mode. On Thu, Nov 5, 2015 at 11:46 AM, Robert Metzger <[hidden email]> wrote: > I agree with Max, but still, its extremely convenient (and much easier than > crafting akka messages) to have a web interface which accepts jar files and > executes the main method ;) > > > On Thu, Nov 5, 2015 at 11:42 AM, Maximilian Michels <[hidden email]> > wrote: > > > +1 The web client needs to go. > > > > I don't really see how how a submission page on the job manager would > make > > security worse. At the moment, anyone who sends an Akka message to the > job > > manager, can execute arbitrary code.. > > > > On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < > [hidden email]> > > wrote: > > > > > +1 to remove the old web client in favor of an improved dashboard (not > so > > > urgent however) > > > > > > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger <[hidden email]> > > > wrote: > > > > > > > Hi, > > > > I personally like the idea of allowing users to submit a job from the > > web > > > > interface a lot. In particular for users new to the system, its a > very > > > > convenient feature. > > > > We would also get rid of this ugly, old web client :) > > > > > > > > Regarding security: We can always add a configuration parameter which > > > > allows users to disable the feature. > > > > > > > > Please share the screenshot with us :) > > > > > > > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < > [hidden email] > > > > > > > wrote: > > > > > > > > > The current authentication mechanism is via htaccess, in the > > webclient. > > > > The > > > > > dashboard however isn't secured at all. > > > > > > > > > > -- Sachin Goel > > > > > Computer Science, IIT Delhi > > > > > m. +91-9871457685 > > > > > > > > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < > > [hidden email]> > > > > > wrote: > > > > > > > > > > > The reason why we didn’t want to have it in the JobManager > > interface > > > by > > > > > > default is that it could be a security concern. I also see, > > however, > > > > that > > > > > > security in general is not very strong right now. > > > > > > > > > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> > > > wrote: > > > > > > > > > > > > > > I like the idea in general! Not sure if integrating it into > > > > JobMangaer > > > > > > > Webinterface is good or not... What would be the pros/cons for > > > > > > > standalone vs. integrated? > > > > > > > > > > > > > > -Matthias > > > > > > > > > > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > > > > > > >> Hi all > > > > > > >> Are there any plans to upgrade the webclient interface? It > could > > > > > > certainly > > > > > > >> use an upgrade compared to the much better dashboard . > > > > > > >> If so, is integrating them into one the best way to go? We can > > > have > > > > a > > > > > > tab > > > > > > >> under Job Manager on the dashboard. > > > > > > >> > > > > > > >> I can share screenshots of the UI I designed last night, if > it's > > > > not a > > > > > > >> terrible idea to do this. :') > > > > > > >> > > > > > > >> Cheers! > > > > > > >> Sachin > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > |
In reply to this post by Robert Metzger
Ah. Sharing the links then:
https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> wrote: > Hey Sachin, I think you can not upload images to apache mailing lists. > > On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel <[hidden email]> > wrote: > > > Okay. I think that does it. I've already designed the basic interface, > > although the major part for uploading jars and actually running the job > are > > left. :') > > Here are some screenshots: > > [image: Inline image 2][image: Inline image 1][image: Inline image 3] > > > > -- Sachin Goel > > Computer Science, IIT Delhi > > m. +91-9871457685 > > > > On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> > wrote: > > > >> +1 The web client needs to go. > >> > >> I don't really see how how a submission page on the job manager would > make > >> security worse. At the moment, anyone who sends an Akka message to the > job > >> manager, can execute arbitrary code.. > >> > >> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < > [hidden email] > >> > > >> wrote: > >> > >> > +1 to remove the old web client in favor of an improved dashboard (not > >> so > >> > urgent however) > >> > > >> > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger <[hidden email]> > >> > wrote: > >> > > >> > > Hi, > >> > > I personally like the idea of allowing users to submit a job from > the > >> web > >> > > interface a lot. In particular for users new to the system, its a > very > >> > > convenient feature. > >> > > We would also get rid of this ugly, old web client :) > >> > > > >> > > Regarding security: We can always add a configuration parameter > which > >> > > allows users to disable the feature. > >> > > > >> > > Please share the screenshot with us :) > >> > > > >> > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < > >> [hidden email]> > >> > > wrote: > >> > > > >> > > > The current authentication mechanism is via htaccess, in the > >> webclient. > >> > > The > >> > > > dashboard however isn't secured at all. > >> > > > > >> > > > -- Sachin Goel > >> > > > Computer Science, IIT Delhi > >> > > > m. +91-9871457685 > >> > > > > >> > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < > >> [hidden email]> > >> > > > wrote: > >> > > > > >> > > > > The reason why we didn’t want to have it in the JobManager > >> interface > >> > by > >> > > > > default is that it could be a security concern. I also see, > >> however, > >> > > that > >> > > > > security in general is not very strong right now. > >> > > > > > >> > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> > >> > wrote: > >> > > > > > > >> > > > > > I like the idea in general! Not sure if integrating it into > >> > > JobMangaer > >> > > > > > Webinterface is good or not... What would be the pros/cons for > >> > > > > > standalone vs. integrated? > >> > > > > > > >> > > > > > -Matthias > >> > > > > > > >> > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > >> > > > > >> Hi all > >> > > > > >> Are there any plans to upgrade the webclient interface? It > >> could > >> > > > > certainly > >> > > > > >> use an upgrade compared to the much better dashboard . > >> > > > > >> If so, is integrating them into one the best way to go? We > can > >> > have > >> > > a > >> > > > > tab > >> > > > > >> under Job Manager on the dashboard. > >> > > > > >> > >> > > > > >> I can share screenshots of the UI I designed last night, if > >> it's > >> > > not a > >> > > > > >> terrible idea to do this. :') > >> > > > > >> > >> > > > > >> Cheers! > >> > > > > >> Sachin > >> > > > > >> > >> > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > > > > |
Hi Stephan
I agree. However, as and when we secure the Dashboard, everything should get secured in one go. Or is a secure dashboard not much desired? [By secure, I mean some form of authentication for accessing.] -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 4:23 PM, Sachin Goel <[hidden email]> wrote: > Ah. Sharing the links then: > > https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> > wrote: > >> Hey Sachin, I think you can not upload images to apache mailing lists. >> >> On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel <[hidden email]> >> wrote: >> >> > Okay. I think that does it. I've already designed the basic interface, >> > although the major part for uploading jars and actually running the job >> are >> > left. :') >> > Here are some screenshots: >> > [image: Inline image 2][image: Inline image 1][image: Inline image 3] >> > >> > -- Sachin Goel >> > Computer Science, IIT Delhi >> > m. +91-9871457685 >> > >> > On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> >> wrote: >> > >> >> +1 The web client needs to go. >> >> >> >> I don't really see how how a submission page on the job manager would >> make >> >> security worse. At the moment, anyone who sends an Akka message to the >> job >> >> manager, can execute arbitrary code.. >> >> >> >> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < >> [hidden email] >> >> > >> >> wrote: >> >> >> >> > +1 to remove the old web client in favor of an improved dashboard >> (not >> >> so >> >> > urgent however) >> >> > >> >> > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger <[hidden email] >> > >> >> > wrote: >> >> > >> >> > > Hi, >> >> > > I personally like the idea of allowing users to submit a job from >> the >> >> web >> >> > > interface a lot. In particular for users new to the system, its a >> very >> >> > > convenient feature. >> >> > > We would also get rid of this ugly, old web client :) >> >> > > >> >> > > Regarding security: We can always add a configuration parameter >> which >> >> > > allows users to disable the feature. >> >> > > >> >> > > Please share the screenshot with us :) >> >> > > >> >> > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < >> >> [hidden email]> >> >> > > wrote: >> >> > > >> >> > > > The current authentication mechanism is via htaccess, in the >> >> webclient. >> >> > > The >> >> > > > dashboard however isn't secured at all. >> >> > > > >> >> > > > -- Sachin Goel >> >> > > > Computer Science, IIT Delhi >> >> > > > m. +91-9871457685 >> >> > > > >> >> > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < >> >> [hidden email]> >> >> > > > wrote: >> >> > > > >> >> > > > > The reason why we didn’t want to have it in the JobManager >> >> interface >> >> > by >> >> > > > > default is that it could be a security concern. I also see, >> >> however, >> >> > > that >> >> > > > > security in general is not very strong right now. >> >> > > > > >> >> > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email]> >> >> > wrote: >> >> > > > > > >> >> > > > > > I like the idea in general! Not sure if integrating it into >> >> > > JobMangaer >> >> > > > > > Webinterface is good or not... What would be the pros/cons >> for >> >> > > > > > standalone vs. integrated? >> >> > > > > > >> >> > > > > > -Matthias >> >> > > > > > >> >> > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: >> >> > > > > >> Hi all >> >> > > > > >> Are there any plans to upgrade the webclient interface? It >> >> could >> >> > > > > certainly >> >> > > > > >> use an upgrade compared to the much better dashboard . >> >> > > > > >> If so, is integrating them into one the best way to go? We >> can >> >> > have >> >> > > a >> >> > > > > tab >> >> > > > > >> under Job Manager on the dashboard. >> >> > > > > >> >> >> > > > > >> I can share screenshots of the UI I designed last night, if >> >> it's >> >> > > not a >> >> > > > > >> terrible idea to do this. :') >> >> > > > > >> >> >> > > > > >> Cheers! >> >> > > > > >> Sachin >> >> > > > > >> >> >> > > > > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> > >> > >> > > |
In reply to this post by Sachin Goel
@Robert: I'm in favor of having a submission page which can be deactivated.
My point was simply that we don't have proper security support right now. I agree with Stephan that we'll need to support it! @Stephan: Good suggestion to disable the submission page for the per-job YARN cluster. @Sachin: Looks nice. Probably there should also be an option to view the plan like in the web client. Should not be so hard since we already have the visualization in place on other pages. On Thu, Nov 5, 2015 at 11:53 AM, Sachin Goel <[hidden email]> wrote: > Ah. Sharing the links then: > > https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> > wrote: > > > Hey Sachin, I think you can not upload images to apache mailing lists. > > > > On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel <[hidden email]> > > wrote: > > > > > Okay. I think that does it. I've already designed the basic interface, > > > although the major part for uploading jars and actually running the job > > are > > > left. :') > > > Here are some screenshots: > > > [image: Inline image 2][image: Inline image 1][image: Inline image 3] > > > > > > -- Sachin Goel > > > Computer Science, IIT Delhi > > > m. +91-9871457685 > > > > > > On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> > > wrote: > > > > > >> +1 The web client needs to go. > > >> > > >> I don't really see how how a submission page on the job manager would > > make > > >> security worse. At the moment, anyone who sends an Akka message to the > > job > > >> manager, can execute arbitrary code.. > > >> > > >> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < > > [hidden email] > > >> > > > >> wrote: > > >> > > >> > +1 to remove the old web client in favor of an improved dashboard > (not > > >> so > > >> > urgent however) > > >> > > > >> > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger < > [hidden email]> > > >> > wrote: > > >> > > > >> > > Hi, > > >> > > I personally like the idea of allowing users to submit a job from > > the > > >> web > > >> > > interface a lot. In particular for users new to the system, its a > > very > > >> > > convenient feature. > > >> > > We would also get rid of this ugly, old web client :) > > >> > > > > >> > > Regarding security: We can always add a configuration parameter > > which > > >> > > allows users to disable the feature. > > >> > > > > >> > > Please share the screenshot with us :) > > >> > > > > >> > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < > > >> [hidden email]> > > >> > > wrote: > > >> > > > > >> > > > The current authentication mechanism is via htaccess, in the > > >> webclient. > > >> > > The > > >> > > > dashboard however isn't secured at all. > > >> > > > > > >> > > > -- Sachin Goel > > >> > > > Computer Science, IIT Delhi > > >> > > > m. +91-9871457685 > > >> > > > > > >> > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < > > >> [hidden email]> > > >> > > > wrote: > > >> > > > > > >> > > > > The reason why we didn’t want to have it in the JobManager > > >> interface > > >> > by > > >> > > > > default is that it could be a security concern. I also see, > > >> however, > > >> > > that > > >> > > > > security in general is not very strong right now. > > >> > > > > > > >> > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax <[hidden email] > > > > >> > wrote: > > >> > > > > > > > >> > > > > > I like the idea in general! Not sure if integrating it into > > >> > > JobMangaer > > >> > > > > > Webinterface is good or not... What would be the pros/cons > for > > >> > > > > > standalone vs. integrated? > > >> > > > > > > > >> > > > > > -Matthias > > >> > > > > > > > >> > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > > >> > > > > >> Hi all > > >> > > > > >> Are there any plans to upgrade the webclient interface? It > > >> could > > >> > > > > certainly > > >> > > > > >> use an upgrade compared to the much better dashboard . > > >> > > > > >> If so, is integrating them into one the best way to go? We > > can > > >> > have > > >> > > a > > >> > > > > tab > > >> > > > > >> under Job Manager on the dashboard. > > >> > > > > >> > > >> > > > > >> I can share screenshots of the UI I designed last night, if > > >> it's > > >> > > not a > > >> > > > > >> terrible idea to do this. :') > > >> > > > > >> > > >> > > > > >> Cheers! > > >> > > > > >> Sachin > > >> > > > > >> > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > > |
Disabling the access for secure installation: Pretty easy. Writing it right
now. @mxm: Thanks. I haven't implemented any handlers for actually running the job. I hope to finish it by this weekend. One quick question: The runtime monitor will be running the job itself, i.e., the main method of program's entry class. Will that be a major issue, that one thread is performing this work [although the server is capable to handling several requests in parallel]? -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 4:28 PM, Maximilian Michels <[hidden email]> wrote: > @Robert: I'm in favor of having a submission page which can be deactivated. > My point was simply that we don't have proper security support right now. I > agree with Stephan that we'll need to support it! > > @Stephan: Good suggestion to disable the submission page for the per-job > YARN cluster. > > @Sachin: Looks nice. Probably there should also be an option to view the > plan like in the web client. Should not be so hard since we already have > the visualization in place on other pages. > > On Thu, Nov 5, 2015 at 11:53 AM, Sachin Goel <[hidden email]> > wrote: > > > Ah. Sharing the links then: > > > > > https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing > > > > > https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing > > > > > https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing > > > > -- Sachin Goel > > Computer Science, IIT Delhi > > m. +91-9871457685 > > > > On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> > > wrote: > > > > > Hey Sachin, I think you can not upload images to apache mailing lists. > > > > > > On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel <[hidden email] > > > > > wrote: > > > > > > > Okay. I think that does it. I've already designed the basic > interface, > > > > although the major part for uploading jars and actually running the > job > > > are > > > > left. :') > > > > Here are some screenshots: > > > > [image: Inline image 2][image: Inline image 1][image: Inline image 3] > > > > > > > > -- Sachin Goel > > > > Computer Science, IIT Delhi > > > > m. +91-9871457685 > > > > > > > > On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> > > > wrote: > > > > > > > >> +1 The web client needs to go. > > > >> > > > >> I don't really see how how a submission page on the job manager > would > > > make > > > >> security worse. At the moment, anyone who sends an Akka message to > the > > > job > > > >> manager, can execute arbitrary code.. > > > >> > > > >> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < > > > [hidden email] > > > >> > > > > >> wrote: > > > >> > > > >> > +1 to remove the old web client in favor of an improved dashboard > > (not > > > >> so > > > >> > urgent however) > > > >> > > > > >> > On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger < > > [hidden email]> > > > >> > wrote: > > > >> > > > > >> > > Hi, > > > >> > > I personally like the idea of allowing users to submit a job > from > > > the > > > >> web > > > >> > > interface a lot. In particular for users new to the system, its > a > > > very > > > >> > > convenient feature. > > > >> > > We would also get rid of this ugly, old web client :) > > > >> > > > > > >> > > Regarding security: We can always add a configuration parameter > > > which > > > >> > > allows users to disable the feature. > > > >> > > > > > >> > > Please share the screenshot with us :) > > > >> > > > > > >> > > On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < > > > >> [hidden email]> > > > >> > > wrote: > > > >> > > > > > >> > > > The current authentication mechanism is via htaccess, in the > > > >> webclient. > > > >> > > The > > > >> > > > dashboard however isn't secured at all. > > > >> > > > > > > >> > > > -- Sachin Goel > > > >> > > > Computer Science, IIT Delhi > > > >> > > > m. +91-9871457685 > > > >> > > > > > > >> > > > On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < > > > >> [hidden email]> > > > >> > > > wrote: > > > >> > > > > > > >> > > > > The reason why we didn’t want to have it in the JobManager > > > >> interface > > > >> > by > > > >> > > > > default is that it could be a security concern. I also see, > > > >> however, > > > >> > > that > > > >> > > > > security in general is not very strong right now. > > > >> > > > > > > > >> > > > > > On 05 Nov 2015, at 09:58, Matthias J. Sax < > [hidden email] > > > > > > >> > wrote: > > > >> > > > > > > > > >> > > > > > I like the idea in general! Not sure if integrating it > into > > > >> > > JobMangaer > > > >> > > > > > Webinterface is good or not... What would be the pros/cons > > for > > > >> > > > > > standalone vs. integrated? > > > >> > > > > > > > > >> > > > > > -Matthias > > > >> > > > > > > > > >> > > > > > On 11/05/2015 05:23 AM, Sachin Goel wrote: > > > >> > > > > >> Hi all > > > >> > > > > >> Are there any plans to upgrade the webclient interface? > It > > > >> could > > > >> > > > > certainly > > > >> > > > > >> use an upgrade compared to the much better dashboard . > > > >> > > > > >> If so, is integrating them into one the best way to go? > We > > > can > > > >> > have > > > >> > > a > > > >> > > > > tab > > > >> > > > > >> under Job Manager on the dashboard. > > > >> > > > > >> > > > >> > > > > >> I can share screenshots of the UI I designed last night, > if > > > >> it's > > > >> > > not a > > > >> > > > > >> terrible idea to do this. :') > > > >> > > > > >> > > > >> > > > > >> Cheers! > > > >> > > > > >> Sachin > > > >> > > > > >> > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > > > > > |
Looks good. How would you display a jar that contains multiple programs
and entry classes? Right now, WebClient can handle this: https://ci.apache.org/projects/flink/flink-docs-master/apis/web_client.html The Manifest entries "Main-Class" and "program-class" can have a comma separated list of entry classes. Furthermore, if interface ProgramDescription is implemented by entry class, a tooltip shows the description right now. And you should go with "Flink Options" instead of "Flink Arguments". That was the consensus in WebClient too, in order to distinguish both clearly. -Matthias On 11/05/2015 12:04 PM, Sachin Goel wrote: > Disabling the access for secure installation: Pretty easy. Writing it right > now. > > @mxm: Thanks. I haven't implemented any handlers for actually running the > job. I hope to finish it by this weekend. > > One quick question: The runtime monitor will be running the job itself, > i.e., the main method of program's entry class. Will that be a major issue, > that one thread is performing this work [although the server is capable to > handling several requests in parallel]? > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 4:28 PM, Maximilian Michels <[hidden email]> wrote: > >> @Robert: I'm in favor of having a submission page which can be deactivated. >> My point was simply that we don't have proper security support right now. I >> agree with Stephan that we'll need to support it! >> >> @Stephan: Good suggestion to disable the submission page for the per-job >> YARN cluster. >> >> @Sachin: Looks nice. Probably there should also be an option to view the >> plan like in the web client. Should not be so hard since we already have >> the visualization in place on other pages. >> >> On Thu, Nov 5, 2015 at 11:53 AM, Sachin Goel <[hidden email]> >> wrote: >> >>> Ah. Sharing the links then: >>> >>> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing >>> >>> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing >>> >>> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing >>> >>> -- Sachin Goel >>> Computer Science, IIT Delhi >>> m. +91-9871457685 >>> >>> On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> >>> wrote: >>> >>>> Hey Sachin, I think you can not upload images to apache mailing lists. >>>> >>>> On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel <[hidden email] >>> >>>> wrote: >>>> >>>>> Okay. I think that does it. I've already designed the basic >> interface, >>>>> although the major part for uploading jars and actually running the >> job >>>> are >>>>> left. :') >>>>> Here are some screenshots: >>>>> [image: Inline image 2][image: Inline image 1][image: Inline image 3] >>>>> >>>>> -- Sachin Goel >>>>> Computer Science, IIT Delhi >>>>> m. +91-9871457685 >>>>> >>>>> On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> >>>> wrote: >>>>> >>>>>> +1 The web client needs to go. >>>>>> >>>>>> I don't really see how how a submission page on the job manager >> would >>>> make >>>>>> security worse. At the moment, anyone who sends an Akka message to >> the >>>> job >>>>>> manager, can execute arbitrary code.. >>>>>> >>>>>> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < >>>> [hidden email] >>>>>>> >>>>>> wrote: >>>>>> >>>>>>> +1 to remove the old web client in favor of an improved dashboard >>> (not >>>>>> so >>>>>>> urgent however) >>>>>>> >>>>>>> On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger < >>> [hidden email]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> I personally like the idea of allowing users to submit a job >> from >>>> the >>>>>> web >>>>>>>> interface a lot. In particular for users new to the system, its >> a >>>> very >>>>>>>> convenient feature. >>>>>>>> We would also get rid of this ugly, old web client :) >>>>>>>> >>>>>>>> Regarding security: We can always add a configuration parameter >>>> which >>>>>>>> allows users to disable the feature. >>>>>>>> >>>>>>>> Please share the screenshot with us :) >>>>>>>> >>>>>>>> On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < >>>>>> [hidden email]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> The current authentication mechanism is via htaccess, in the >>>>>> webclient. >>>>>>>> The >>>>>>>>> dashboard however isn't secured at all. >>>>>>>>> >>>>>>>>> -- Sachin Goel >>>>>>>>> Computer Science, IIT Delhi >>>>>>>>> m. +91-9871457685 >>>>>>>>> >>>>>>>>> On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < >>>>>> [hidden email]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> The reason why we didn’t want to have it in the JobManager >>>>>> interface >>>>>>> by >>>>>>>>>> default is that it could be a security concern. I also see, >>>>>> however, >>>>>>>> that >>>>>>>>>> security in general is not very strong right now. >>>>>>>>>> >>>>>>>>>>> On 05 Nov 2015, at 09:58, Matthias J. Sax < >> [hidden email] >>>> >>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I like the idea in general! Not sure if integrating it >> into >>>>>>>> JobMangaer >>>>>>>>>>> Webinterface is good or not... What would be the pros/cons >>> for >>>>>>>>>>> standalone vs. integrated? >>>>>>>>>>> >>>>>>>>>>> -Matthias >>>>>>>>>>> >>>>>>>>>>> On 11/05/2015 05:23 AM, Sachin Goel wrote: >>>>>>>>>>>> Hi all >>>>>>>>>>>> Are there any plans to upgrade the webclient interface? >> It >>>>>> could >>>>>>>>>> certainly >>>>>>>>>>>> use an upgrade compared to the much better dashboard . >>>>>>>>>>>> If so, is integrating them into one the best way to go? >> We >>>> can >>>>>>> have >>>>>>>> a >>>>>>>>>> tab >>>>>>>>>>>> under Job Manager on the dashboard. >>>>>>>>>>>> >>>>>>>>>>>> I can share screenshots of the UI I designed last night, >> if >>>>>> it's >>>>>>>> not a >>>>>>>>>>>> terrible idea to do this. :') >>>>>>>>>>>> >>>>>>>>>>>> Cheers! >>>>>>>>>>>> Sachin >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> > |
I plan to implement all features of the existing web client. :) I think
multiple programs will be best displayed as a list in the Entry Class column. -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 5:36 PM, Matthias J. Sax <[hidden email]> wrote: > Looks good. How would you display a jar that contains multiple programs > and entry classes? Right now, WebClient can handle this: > https://ci.apache.org/projects/flink/flink-docs-master/apis/web_client.html > > The Manifest entries "Main-Class" and "program-class" can have a comma > separated list of entry classes. Furthermore, if interface > ProgramDescription is implemented by entry class, a tooltip shows the > description right now. > > And you should go with "Flink Options" instead of "Flink Arguments". > That was the consensus in WebClient too, in order to distinguish both > clearly. > > -Matthias > > On 11/05/2015 12:04 PM, Sachin Goel wrote: > > Disabling the access for secure installation: Pretty easy. Writing it > right > > now. > > > > @mxm: Thanks. I haven't implemented any handlers for actually running the > > job. I hope to finish it by this weekend. > > > > One quick question: The runtime monitor will be running the job itself, > > i.e., the main method of program's entry class. Will that be a major > issue, > > that one thread is performing this work [although the server is capable > to > > handling several requests in parallel]? > > > > -- Sachin Goel > > Computer Science, IIT Delhi > > m. +91-9871457685 > > > > On Thu, Nov 5, 2015 at 4:28 PM, Maximilian Michels <[hidden email]> > wrote: > > > >> @Robert: I'm in favor of having a submission page which can be > deactivated. > >> My point was simply that we don't have proper security support right > now. I > >> agree with Stephan that we'll need to support it! > >> > >> @Stephan: Good suggestion to disable the submission page for the per-job > >> YARN cluster. > >> > >> @Sachin: Looks nice. Probably there should also be an option to view the > >> plan like in the web client. Should not be so hard since we already have > >> the visualization in place on other pages. > >> > >> On Thu, Nov 5, 2015 at 11:53 AM, Sachin Goel <[hidden email]> > >> wrote: > >> > >>> Ah. Sharing the links then: > >>> > >>> > >> > https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing > >>> > >>> > >> > https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing > >>> > >>> > >> > https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing > >>> > >>> -- Sachin Goel > >>> Computer Science, IIT Delhi > >>> m. +91-9871457685 > >>> > >>> On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> > >>> wrote: > >>> > >>>> Hey Sachin, I think you can not upload images to apache mailing lists. > >>>> > >>>> On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel < > [hidden email] > >>> > >>>> wrote: > >>>> > >>>>> Okay. I think that does it. I've already designed the basic > >> interface, > >>>>> although the major part for uploading jars and actually running the > >> job > >>>> are > >>>>> left. :') > >>>>> Here are some screenshots: > >>>>> [image: Inline image 2][image: Inline image 1][image: Inline image 3] > >>>>> > >>>>> -- Sachin Goel > >>>>> Computer Science, IIT Delhi > >>>>> m. +91-9871457685 > >>>>> > >>>>> On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> > >>>> wrote: > >>>>> > >>>>>> +1 The web client needs to go. > >>>>>> > >>>>>> I don't really see how how a submission page on the job manager > >> would > >>>> make > >>>>>> security worse. At the moment, anyone who sends an Akka message to > >> the > >>>> job > >>>>>> manager, can execute arbitrary code.. > >>>>>> > >>>>>> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < > >>>> [hidden email] > >>>>>>> > >>>>>> wrote: > >>>>>> > >>>>>>> +1 to remove the old web client in favor of an improved dashboard > >>> (not > >>>>>> so > >>>>>>> urgent however) > >>>>>>> > >>>>>>> On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger < > >>> [hidden email]> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Hi, > >>>>>>>> I personally like the idea of allowing users to submit a job > >> from > >>>> the > >>>>>> web > >>>>>>>> interface a lot. In particular for users new to the system, its > >> a > >>>> very > >>>>>>>> convenient feature. > >>>>>>>> We would also get rid of this ugly, old web client :) > >>>>>>>> > >>>>>>>> Regarding security: We can always add a configuration parameter > >>>> which > >>>>>>>> allows users to disable the feature. > >>>>>>>> > >>>>>>>> Please share the screenshot with us :) > >>>>>>>> > >>>>>>>> On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < > >>>>>> [hidden email]> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> The current authentication mechanism is via htaccess, in the > >>>>>> webclient. > >>>>>>>> The > >>>>>>>>> dashboard however isn't secured at all. > >>>>>>>>> > >>>>>>>>> -- Sachin Goel > >>>>>>>>> Computer Science, IIT Delhi > >>>>>>>>> m. +91-9871457685 > >>>>>>>>> > >>>>>>>>> On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < > >>>>>> [hidden email]> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> The reason why we didn’t want to have it in the JobManager > >>>>>> interface > >>>>>>> by > >>>>>>>>>> default is that it could be a security concern. I also see, > >>>>>> however, > >>>>>>>> that > >>>>>>>>>> security in general is not very strong right now. > >>>>>>>>>> > >>>>>>>>>>> On 05 Nov 2015, at 09:58, Matthias J. Sax < > >> [hidden email] > >>>> > >>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> I like the idea in general! Not sure if integrating it > >> into > >>>>>>>> JobMangaer > >>>>>>>>>>> Webinterface is good or not... What would be the pros/cons > >>> for > >>>>>>>>>>> standalone vs. integrated? > >>>>>>>>>>> > >>>>>>>>>>> -Matthias > >>>>>>>>>>> > >>>>>>>>>>> On 11/05/2015 05:23 AM, Sachin Goel wrote: > >>>>>>>>>>>> Hi all > >>>>>>>>>>>> Are there any plans to upgrade the webclient interface? > >> It > >>>>>> could > >>>>>>>>>> certainly > >>>>>>>>>>>> use an upgrade compared to the much better dashboard . > >>>>>>>>>>>> If so, is integrating them into one the best way to go? > >> We > >>>> can > >>>>>>> have > >>>>>>>> a > >>>>>>>>>> tab > >>>>>>>>>>>> under Job Manager on the dashboard. > >>>>>>>>>>>> > >>>>>>>>>>>> I can share screenshots of the UI I designed last night, > >> if > >>>>>> it's > >>>>>>>> not a > >>>>>>>>>>>> terrible idea to do this. :') > >>>>>>>>>>>> > >>>>>>>>>>>> Cheers! > >>>>>>>>>>>> Sachin > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>> > >>> > >> > > > > |
Update: I've finished most of the work on this.
Here are a few screenshots: https://drive.google.com/file/d/0B0Xlv1qTgpjDRlMxRkQ5OG5PRUk/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDczJwLU13ZWM5anM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDbXlwbWhVelFFRWM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDX2VlWkl6ZTRTLWM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDZHNFWFFBVlg1dmM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDZEZHdE9pS1V0RVE/view?usp=sharing If you want to try it out, here's my branch: https://github.com/sachingoel0101/flink/tree/client Just make sure to copy the jars to "webclient-jobs" in this format: <13 digit timestamp>_<jar_name>.jar. [Uploading functionality is not yet implemented.] I also made the decision to only allow two flink options: parallelism and entry class. I think this is reasonable because we do not need anything related to yarn here. Web monitor already has the gateway to job manager. Any comments or review is welcome. :) I'll likely be able to finish up by the end of this week. Cheers! Sachin -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 5:46 PM, Sachin Goel <[hidden email]> wrote: > I plan to implement all features of the existing web client. :) I think > multiple programs will be best displayed as a list in the Entry Class > column. > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 5:36 PM, Matthias J. Sax <[hidden email]> wrote: > >> Looks good. How would you display a jar that contains multiple programs >> and entry classes? Right now, WebClient can handle this: >> >> https://ci.apache.org/projects/flink/flink-docs-master/apis/web_client.html >> >> The Manifest entries "Main-Class" and "program-class" can have a comma >> separated list of entry classes. Furthermore, if interface >> ProgramDescription is implemented by entry class, a tooltip shows the >> description right now. >> >> And you should go with "Flink Options" instead of "Flink Arguments". >> That was the consensus in WebClient too, in order to distinguish both >> clearly. >> >> -Matthias >> >> On 11/05/2015 12:04 PM, Sachin Goel wrote: >> > Disabling the access for secure installation: Pretty easy. Writing it >> right >> > now. >> > >> > @mxm: Thanks. I haven't implemented any handlers for actually running >> the >> > job. I hope to finish it by this weekend. >> > >> > One quick question: The runtime monitor will be running the job itself, >> > i.e., the main method of program's entry class. Will that be a major >> issue, >> > that one thread is performing this work [although the server is capable >> to >> > handling several requests in parallel]? >> > >> > -- Sachin Goel >> > Computer Science, IIT Delhi >> > m. +91-9871457685 >> > >> > On Thu, Nov 5, 2015 at 4:28 PM, Maximilian Michels <[hidden email]> >> wrote: >> > >> >> @Robert: I'm in favor of having a submission page which can be >> deactivated. >> >> My point was simply that we don't have proper security support right >> now. I >> >> agree with Stephan that we'll need to support it! >> >> >> >> @Stephan: Good suggestion to disable the submission page for the >> per-job >> >> YARN cluster. >> >> >> >> @Sachin: Looks nice. Probably there should also be an option to view >> the >> >> plan like in the web client. Should not be so hard since we already >> have >> >> the visualization in place on other pages. >> >> >> >> On Thu, Nov 5, 2015 at 11:53 AM, Sachin Goel <[hidden email] >> > >> >> wrote: >> >> >> >>> Ah. Sharing the links then: >> >>> >> >>> >> >> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing >> >>> >> >>> >> >> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing >> >>> >> >>> >> >> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing >> >>> >> >>> -- Sachin Goel >> >>> Computer Science, IIT Delhi >> >>> m. +91-9871457685 >> >>> >> >>> On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> >> >>> wrote: >> >>> >> >>>> Hey Sachin, I think you can not upload images to apache mailing >> lists. >> >>>> >> >>>> On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel < >> [hidden email] >> >>> >> >>>> wrote: >> >>>> >> >>>>> Okay. I think that does it. I've already designed the basic >> >> interface, >> >>>>> although the major part for uploading jars and actually running the >> >> job >> >>>> are >> >>>>> left. :') >> >>>>> Here are some screenshots: >> >>>>> [image: Inline image 2][image: Inline image 1][image: Inline image >> 3] >> >>>>> >> >>>>> -- Sachin Goel >> >>>>> Computer Science, IIT Delhi >> >>>>> m. +91-9871457685 >> >>>>> >> >>>>> On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> >> >>>> wrote: >> >>>>> >> >>>>>> +1 The web client needs to go. >> >>>>>> >> >>>>>> I don't really see how how a submission page on the job manager >> >> would >> >>>> make >> >>>>>> security worse. At the moment, anyone who sends an Akka message to >> >> the >> >>>> job >> >>>>>> manager, can execute arbitrary code.. >> >>>>>> >> >>>>>> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < >> >>>> [hidden email] >> >>>>>>> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> +1 to remove the old web client in favor of an improved dashboard >> >>> (not >> >>>>>> so >> >>>>>>> urgent however) >> >>>>>>> >> >>>>>>> On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger < >> >>> [hidden email]> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> Hi, >> >>>>>>>> I personally like the idea of allowing users to submit a job >> >> from >> >>>> the >> >>>>>> web >> >>>>>>>> interface a lot. In particular for users new to the system, its >> >> a >> >>>> very >> >>>>>>>> convenient feature. >> >>>>>>>> We would also get rid of this ugly, old web client :) >> >>>>>>>> >> >>>>>>>> Regarding security: We can always add a configuration parameter >> >>>> which >> >>>>>>>> allows users to disable the feature. >> >>>>>>>> >> >>>>>>>> Please share the screenshot with us :) >> >>>>>>>> >> >>>>>>>> On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < >> >>>>>> [hidden email]> >> >>>>>>>> wrote: >> >>>>>>>> >> >>>>>>>>> The current authentication mechanism is via htaccess, in the >> >>>>>> webclient. >> >>>>>>>> The >> >>>>>>>>> dashboard however isn't secured at all. >> >>>>>>>>> >> >>>>>>>>> -- Sachin Goel >> >>>>>>>>> Computer Science, IIT Delhi >> >>>>>>>>> m. +91-9871457685 >> >>>>>>>>> >> >>>>>>>>> On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < >> >>>>>> [hidden email]> >> >>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> The reason why we didn’t want to have it in the JobManager >> >>>>>> interface >> >>>>>>> by >> >>>>>>>>>> default is that it could be a security concern. I also see, >> >>>>>> however, >> >>>>>>>> that >> >>>>>>>>>> security in general is not very strong right now. >> >>>>>>>>>> >> >>>>>>>>>>> On 05 Nov 2015, at 09:58, Matthias J. Sax < >> >> [hidden email] >> >>>> >> >>>>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>> I like the idea in general! Not sure if integrating it >> >> into >> >>>>>>>> JobMangaer >> >>>>>>>>>>> Webinterface is good or not... What would be the pros/cons >> >>> for >> >>>>>>>>>>> standalone vs. integrated? >> >>>>>>>>>>> >> >>>>>>>>>>> -Matthias >> >>>>>>>>>>> >> >>>>>>>>>>> On 11/05/2015 05:23 AM, Sachin Goel wrote: >> >>>>>>>>>>>> Hi all >> >>>>>>>>>>>> Are there any plans to upgrade the webclient interface? >> >> It >> >>>>>> could >> >>>>>>>>>> certainly >> >>>>>>>>>>>> use an upgrade compared to the much better dashboard . >> >>>>>>>>>>>> If so, is integrating them into one the best way to go? >> >> We >> >>>> can >> >>>>>>> have >> >>>>>>>> a >> >>>>>>>>>> tab >> >>>>>>>>>>>> under Job Manager on the dashboard. >> >>>>>>>>>>>> >> >>>>>>>>>>>> I can share screenshots of the UI I designed last night, >> >> if >> >>>>>> it's >> >>>>>>>> not a >> >>>>>>>>>>>> terrible idea to do this. :') >> >>>>>>>>>>>> >> >>>>>>>>>>>> Cheers! >> >>>>>>>>>>>> Sachin >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> > >> >> > |
Hi Sachin,
great work and very quickly! Just a question: do you already manage a jar contain multiple jobs (multiple main classes or Program interface implementation)? I have also a suggestion for a very nice feature: right now there's the ParameterTool API that centralize and standardize the passing of parameters to the program. With a little bit of effort I think it should be possible to show to the user an automatically computed list of required and optional parameters, with their default value (if any) and the possibility to check if the passed parameter respect the specified type. What do you think? Update: I've finished most of the work on this. Here are a few screenshots: https://drive.google.com/file/d/0B0Xlv1qTgpjDRlMxRkQ5OG5PRUk/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDczJwLU13ZWM5anM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDbXlwbWhVelFFRWM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDX2VlWkl6ZTRTLWM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDZHNFWFFBVlg1dmM/view?usp=sharing https://drive.google.com/file/d/0B0Xlv1qTgpjDZEZHdE9pS1V0RVE/view?usp=sharing If you want to try it out, here's my branch: https://github.com/sachingoel0101/flink/tree/client Just make sure to copy the jars to "webclient-jobs" in this format: <13 digit timestamp>_<jar_name>.jar. [Uploading functionality is not yet implemented.] I also made the decision to only allow two flink options: parallelism and entry class. I think this is reasonable because we do not need anything related to yarn here. Web monitor already has the gateway to job manager. Any comments or review is welcome. :) I'll likely be able to finish up by the end of this week. Cheers! Sachin -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Thu, Nov 5, 2015 at 5:46 PM, Sachin Goel <[hidden email]> wrote: > I plan to implement all features of the existing web client. :) I think > multiple programs will be best displayed as a list in the Entry Class > column. > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 5:36 PM, Matthias J. Sax <[hidden email]> wrote: > >> Looks good. How would you display a jar that contains multiple programs >> and entry classes? Right now, WebClient can handle this: >> >> >> >> The Manifest entries "Main-Class" and "program-class" can have a comma >> separated list of entry classes. Furthermore, if interface >> ProgramDescription is implemented by entry class, a tooltip shows the >> description right now. >> >> And you should go with "Flink Options" instead of "Flink Arguments". >> That was the consensus in WebClient too, in order to distinguish both >> clearly. >> >> -Matthias >> >> On 11/05/2015 12:04 PM, Sachin Goel wrote: >> > Disabling the access for secure installation: Pretty easy. Writing it >> right >> > now. >> > >> > @mxm: Thanks. I haven't implemented any handlers for actually running >> the >> > job. I hope to finish it by this weekend. >> > >> > One quick question: The runtime monitor will be running the job itself, >> > i.e., the main method of program's entry class. Will that be a major >> issue, >> > that one thread is performing this work [although the server is capable >> to >> > handling several requests in parallel]? >> > >> > -- Sachin Goel >> > Computer Science, IIT Delhi >> > m. +91-9871457685 >> > >> > On Thu, Nov 5, 2015 at 4:28 PM, Maximilian Michels <[hidden email]> >> wrote: >> > >> >> @Robert: I'm in favor of having a submission page which can be >> deactivated. >> >> My point was simply that we don't have proper security support right >> now. I >> >> agree with Stephan that we'll need to support it! >> >> >> >> @Stephan: Good suggestion to disable the submission page for the >> per-job >> >> YARN cluster. >> >> >> >> @Sachin: Looks nice. Probably there should also be an option to view >> the >> >> plan like in the web client. Should not be so hard since we already >> have >> >> the visualization in place on other pages. >> >> >> >> On Thu, Nov 5, 2015 at 11:53 AM, Sachin Goel <[hidden email] >> > >> >> wrote: >> >> >> >>> Ah. Sharing the links then: >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing >> >>> >> >>> >> >> >> https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing >> >>> >> >>> -- Sachin Goel >> >>> Computer Science, IIT Delhi >> >>> m. +91-9871457685 >> >>> >> >>> On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email]> >> >>> wrote: >> >>> >> >>>> Hey Sachin, I think you can not upload images to apache mailing >> lists. >> >>>> >> >>>> On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel < >> [hidden email] >> >>> >> >>>> wrote: >> >>>> >> >>>>> Okay. I think that does it. I've already designed the basic >> >> interface, >> >>>>> although the major part for uploading jars and actually running the >> >> job >> >>>> are >> >>>>> left. :') >> >>>>> Here are some screenshots: >> >>>>> [image: Inline image 2][image: Inline image 1][image: Inline image >> 3] >> >>>>> >> >>>>> -- Sachin Goel >> >>>>> Computer Science, IIT Delhi >> >>>>> m. +91-9871457685 >> >>>>> >> >>>>> On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels <[hidden email]> >> >>>> wrote: >> >>>>> >> >>>>>> +1 The web client needs to go. >> >>>>>> >> >>>>>> I don't really see how how a submission page on the job manager >> >> would >> >>>> make >> >>>>>> security worse. At the moment, anyone who sends an Akka message to >> >> the >> >>>> job >> >>>>>> manager, can execute arbitrary code.. >> >>>>>> >> >>>>>> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < >> >>>> [hidden email] >> >>>>>>> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> +1 to remove the old web client in favor of an improved dashboard >> >>> (not >> >>>>>> so >> >>>>>>> urgent however) >> >>>>>>> >> >>>>>>> On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger < >> >>> [hidden email]> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> Hi, >> >>>>>>>> I personally like the idea of allowing users to submit a job >> >> from >> >>>> the >> >>>>>> web >> >>>>>>>> interface a lot. In particular for users new to the system, its >> >> a >> >>>> very >> >>>>>>>> convenient feature. >> >>>>>>>> We would also get rid of this ugly, old web client :) >> >>>>>>>> >> >>>>>>>> Regarding security: We can always add a configuration parameter >> >>>> which >> >>>>>>>> allows users to disable the feature. >> >>>>>>>> >> >>>>>>>> Please share the screenshot with us :) >> >>>>>>>> >> >>>>>>>> On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < >> >>>>>> [hidden email]> >> >>>>>>>> wrote: >> >>>>>>>> >> >>>>>>>>> The current authentication mechanism is via htaccess, in the >> >>>>>> webclient. >> >>>>>>>> The >> >>>>>>>>> dashboard however isn't secured at all. >> >>>>>>>>> >> >>>>>>>>> -- Sachin Goel >> >>>>>>>>> Computer Science, IIT Delhi >> >>>>>>>>> m. +91-9871457685 >> >>>>>>>>> >> >>>>>>>>> On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < >> >>>>>> [hidden email]> >> >>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> The reason why we didn’t want to have it in the JobManager >> >>>>>> interface >> >>>>>>> by >> >>>>>>>>>> default is that it could be a security concern. I also see, >> >>>>>> however, >> >>>>>>>> that >> >>>>>>>>>> security in general is not very strong right now. >> >>>>>>>>>> >> >>>>>>>>>>> On 05 Nov 2015, at 09:58, Matthias J. Sax < >> >> [hidden email] >> >>>> >> >>>>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>> I like the idea in general! Not sure if integrating it >> >> into >> >>>>>>>> JobMangaer >> >>>>>>>>>>> Webinterface is good or not... What would be the pros/cons >> >>> for >> >>>>>>>>>>> standalone vs. integrated? >> >>>>>>>>>>> >> >>>>>>>>>>> -Matthias >> >>>>>>>>>>> >> >>>>>>>>>>> On 11/05/2015 05:23 AM, Sachin Goel wrote: >> >>>>>>>>>>>> Hi all >> >>>>>>>>>>>> Are there any plans to upgrade the webclient interface? >> >> It >> >>>>>> could >> >>>>>>>>>> certainly >> >>>>>>>>>>>> use an upgrade compared to the much better dashboard . >> >>>>>>>>>>>> If so, is integrating them into one the best way to go? >> >> We >> >>>> can >> >>>>>>> have >> >>>>>>>> a >> >>>>>>>>>> tab >> >>>>>>>>>>>> under Job Manager on the dashboard. >> >>>>>>>>>>>> >> >>>>>>>>>>>> I can share screenshots of the UI I designed last night, >> >> if >> >>>>>> it's >> >>>>>>>> not a >> >>>>>>>>>>>> terrible idea to do this. :') >> >>>>>>>>>>>> >> >>>>>>>>>>>> Cheers! >> >>>>>>>>>>>> Sachin >> >>>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> > >> >> > |
Hi Flavio
Yes. Multiple classes are reported properly. And having a Parameter interface sounds nice, but IMO it'll make the UI too cluttered. What can be done is to, say, search for a method named `getDescription` and display that to the user on frontend? Right now, we only do that for Program interface extending classes. A little bit of reflection magic will allow so for Interactive programs too. I also looked at the ParameterTool API, but it doesn't appear to be used much anywhere. Although I think there's some work going on this. I'm not sure if that's been merged yet. Also, managing required parameters, and their types etc. is somewhat frustrating in JavaScript. Everything has to be rechecked at the Server again anyways. I opened a PR just now: https://github.com/apache/flink/pull/1338. Let me know what you guys think. :) Cheers! Sachin -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Sat, Nov 7, 2015 at 5:11 AM, Flavio Pompermaier <[hidden email]> wrote: > Hi Sachin, > great work and very quickly! > Just a question: do you already manage a jar contain multiple jobs > (multiple main classes or Program interface implementation)? > I have also a suggestion for a very nice feature: right now there's the > ParameterTool API that centralize and standardize the passing of parameters > to the program. With a little bit of effort I think it should be possible > to show to the user an automatically computed list of required and optional > parameters, with their default value (if any) and the possibility to check > if the passed parameter respect the specified type. > What do you think? > Update: I've finished most of the work on this. > Here are a few screenshots: > > https://drive.google.com/file/d/0B0Xlv1qTgpjDRlMxRkQ5OG5PRUk/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDczJwLU13ZWM5anM/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDbXlwbWhVelFFRWM/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDX2VlWkl6ZTRTLWM/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDZHNFWFFBVlg1dmM/view?usp=sharing > > https://drive.google.com/file/d/0B0Xlv1qTgpjDZEZHdE9pS1V0RVE/view?usp=sharing > > If you want to try it out, here's my branch: > https://github.com/sachingoel0101/flink/tree/client > Just make sure to copy the jars to "webclient-jobs" in this format: <13 > digit timestamp>_<jar_name>.jar. [Uploading functionality is not yet > implemented.] > > I also made the decision to only allow two flink options: parallelism and > entry class. I think this is reasonable because we do not need anything > related to yarn here. Web monitor already has the gateway to job manager. > > Any comments or review is welcome. :) > > I'll likely be able to finish up by the end of this week. > > Cheers! > Sachin > > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Thu, Nov 5, 2015 at 5:46 PM, Sachin Goel <[hidden email]> > wrote: > > > I plan to implement all features of the existing web client. :) I think > > multiple programs will be best displayed as a list in the Entry Class > > column. > > > > -- Sachin Goel > > Computer Science, IIT Delhi > > m. +91-9871457685 > > > > On Thu, Nov 5, 2015 at 5:36 PM, Matthias J. Sax <[hidden email]> > wrote: > > > >> Looks good. How would you display a jar that contains multiple programs > >> and entry classes? Right now, WebClient can handle this: > >> > >> > https://ci.apache.org/projects/flink/flink-docs-master/apis/web_client.html > >> > >> The Manifest entries "Main-Class" and "program-class" can have a comma > >> separated list of entry classes. Furthermore, if interface > >> ProgramDescription is implemented by entry class, a tooltip shows the > >> description right now. > >> > >> And you should go with "Flink Options" instead of "Flink Arguments". > >> That was the consensus in WebClient too, in order to distinguish both > >> clearly. > >> > >> -Matthias > >> > >> On 11/05/2015 12:04 PM, Sachin Goel wrote: > >> > Disabling the access for secure installation: Pretty easy. Writing it > >> right > >> > now. > >> > > >> > @mxm: Thanks. I haven't implemented any handlers for actually running > >> the > >> > job. I hope to finish it by this weekend. > >> > > >> > One quick question: The runtime monitor will be running the job > itself, > >> > i.e., the main method of program's entry class. Will that be a major > >> issue, > >> > that one thread is performing this work [although the server is > capable > >> to > >> > handling several requests in parallel]? > >> > > >> > -- Sachin Goel > >> > Computer Science, IIT Delhi > >> > m. +91-9871457685 > >> > > >> > On Thu, Nov 5, 2015 at 4:28 PM, Maximilian Michels <[hidden email]> > >> wrote: > >> > > >> >> @Robert: I'm in favor of having a submission page which can be > >> deactivated. > >> >> My point was simply that we don't have proper security support right > >> now. I > >> >> agree with Stephan that we'll need to support it! > >> >> > >> >> @Stephan: Good suggestion to disable the submission page for the > >> per-job > >> >> YARN cluster. > >> >> > >> >> @Sachin: Looks nice. Probably there should also be an option to view > >> the > >> >> plan like in the web client. Should not be so hard since we already > >> have > >> >> the visualization in place on other pages. > >> >> > >> >> On Thu, Nov 5, 2015 at 11:53 AM, Sachin Goel < > [hidden email] > >> > > >> >> wrote: > >> >> > >> >>> Ah. Sharing the links then: > >> >>> > >> >>> > >> >> > >> > > https://drive.google.com/file/d/0B0Xlv1qTgpjDT0ZCd0pSU3VYMUU/view?usp=sharing > >> >>> > >> >>> > >> >> > >> > > https://drive.google.com/file/d/0B0Xlv1qTgpjDeG51azRRc0FJYjQ/view?usp=sharing > >> >>> > >> >>> > >> >> > >> > > https://drive.google.com/file/d/0B0Xlv1qTgpjDSHZXQ0h5S0FBS3c/view?usp=sharing > >> >>> > >> >>> -- Sachin Goel > >> >>> Computer Science, IIT Delhi > >> >>> m. +91-9871457685 > >> >>> > >> >>> On Thu, Nov 5, 2015 at 4:18 PM, Robert Metzger <[hidden email] > > > >> >>> wrote: > >> >>> > >> >>>> Hey Sachin, I think you can not upload images to apache mailing > >> lists. > >> >>>> > >> >>>> On Thu, Nov 5, 2015 at 11:46 AM, Sachin Goel < > >> [hidden email] > >> >>> > >> >>>> wrote: > >> >>>> > >> >>>>> Okay. I think that does it. I've already designed the basic > >> >> interface, > >> >>>>> although the major part for uploading jars and actually running > the > >> >> job > >> >>>> are > >> >>>>> left. :') > >> >>>>> Here are some screenshots: > >> >>>>> [image: Inline image 2][image: Inline image 1][image: Inline image > >> 3] > >> >>>>> > >> >>>>> -- Sachin Goel > >> >>>>> Computer Science, IIT Delhi > >> >>>>> m. +91-9871457685 > >> >>>>> > >> >>>>> On Thu, Nov 5, 2015 at 4:12 PM, Maximilian Michels < > [hidden email]> > >> >>>> wrote: > >> >>>>> > >> >>>>>> +1 The web client needs to go. > >> >>>>>> > >> >>>>>> I don't really see how how a submission page on the job manager > >> >> would > >> >>>> make > >> >>>>>> security worse. At the moment, anyone who sends an Akka message > to > >> >> the > >> >>>> job > >> >>>>>> manager, can execute arbitrary code.. > >> >>>>>> > >> >>>>>> On Thu, Nov 5, 2015 at 11:16 AM, Flavio Pompermaier < > >> >>>> [hidden email] > >> >>>>>>> > >> >>>>>> wrote: > >> >>>>>> > >> >>>>>>> +1 to remove the old web client in favor of an improved > dashboard > >> >>> (not > >> >>>>>> so > >> >>>>>>> urgent however) > >> >>>>>>> > >> >>>>>>> On Thu, Nov 5, 2015 at 11:11 AM, Robert Metzger < > >> >>> [hidden email]> > >> >>>>>>> wrote: > >> >>>>>>> > >> >>>>>>>> Hi, > >> >>>>>>>> I personally like the idea of allowing users to submit a job > >> >> from > >> >>>> the > >> >>>>>> web > >> >>>>>>>> interface a lot. In particular for users new to the system, its > >> >> a > >> >>>> very > >> >>>>>>>> convenient feature. > >> >>>>>>>> We would also get rid of this ugly, old web client :) > >> >>>>>>>> > >> >>>>>>>> Regarding security: We can always add a configuration parameter > >> >>>> which > >> >>>>>>>> allows users to disable the feature. > >> >>>>>>>> > >> >>>>>>>> Please share the screenshot with us :) > >> >>>>>>>> > >> >>>>>>>> On Thu, Nov 5, 2015 at 10:08 AM, Sachin Goel < > >> >>>>>> [hidden email]> > >> >>>>>>>> wrote: > >> >>>>>>>> > >> >>>>>>>>> The current authentication mechanism is via htaccess, in the > >> >>>>>> webclient. > >> >>>>>>>> The > >> >>>>>>>>> dashboard however isn't secured at all. > >> >>>>>>>>> > >> >>>>>>>>> -- Sachin Goel > >> >>>>>>>>> Computer Science, IIT Delhi > >> >>>>>>>>> m. +91-9871457685 > >> >>>>>>>>> > >> >>>>>>>>> On Thu, Nov 5, 2015 at 2:33 PM, Aljoscha Krettek < > >> >>>>>> [hidden email]> > >> >>>>>>>>> wrote: > >> >>>>>>>>> > >> >>>>>>>>>> The reason why we didn’t want to have it in the JobManager > >> >>>>>> interface > >> >>>>>>> by > >> >>>>>>>>>> default is that it could be a security concern. I also see, > >> >>>>>> however, > >> >>>>>>>> that > >> >>>>>>>>>> security in general is not very strong right now. > >> >>>>>>>>>> > >> >>>>>>>>>>> On 05 Nov 2015, at 09:58, Matthias J. Sax < > >> >> [hidden email] > >> >>>> > >> >>>>>>> wrote: > >> >>>>>>>>>>> > >> >>>>>>>>>>> I like the idea in general! Not sure if integrating it > >> >> into > >> >>>>>>>> JobMangaer > >> >>>>>>>>>>> Webinterface is good or not... What would be the pros/cons > >> >>> for > >> >>>>>>>>>>> standalone vs. integrated? > >> >>>>>>>>>>> > >> >>>>>>>>>>> -Matthias > >> >>>>>>>>>>> > >> >>>>>>>>>>> On 11/05/2015 05:23 AM, Sachin Goel wrote: > >> >>>>>>>>>>>> Hi all > >> >>>>>>>>>>>> Are there any plans to upgrade the webclient interface? > >> >> It > >> >>>>>> could > >> >>>>>>>>>> certainly > >> >>>>>>>>>>>> use an upgrade compared to the much better dashboard . > >> >>>>>>>>>>>> If so, is integrating them into one the best way to go? > >> >> We > >> >>>> can > >> >>>>>>> have > >> >>>>>>>> a > >> >>>>>>>>>> tab > >> >>>>>>>>>>>> under Job Manager on the dashboard. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> I can share screenshots of the UI I designed last night, > >> >> if > >> >>>>>> it's > >> >>>>>>>> not a > >> >>>>>>>>>>>> terrible idea to do this. :') > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Cheers! > >> >>>>>>>>>>>> Sachin > >> >>>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>>> > >> >>>>> > >> >>>>> > >> >>>> > >> >>> > >> >> > >> > > >> > >> > > > |
Free forum by Nabble | Edit this page |