http://deprecated-apache-flink-mailing-list-archive.368.s1.nabble.com/DISCUSS-Dashboard-HistoryServer-authentication-tp50993p51093.html
the Flink cluster. Would this be possible? The beauty of this setup is in
> Thanks for giving options to fulfil the need.
>
> Users are looking for a solution where users can be identified on the
> whole cluster and restrict access to resources/actions.
> A good example for such an action is cancelling other users running jobs.
>
> * SSL does provide mutual authentication but when authentication passed
> there is no user based on restrictions can be made.
> * The less problematic part is that generating/maintaining short time
> valid certificates would be a hard (that's the reason KDC like servers
> exist).
> Having long time valid certificates would widen the attack surface but
> since the first concern is there this is just a cosmetic issue.
>
> All in all using TLS certificates is not sufficient in these environments
> unfortunately.
>
> BR,
> G
>
>
> On Thu, Jun 3, 2021 at 12:49 PM Till Rohrmann <
[hidden email]>
> wrote:
>
>> Thanks for the information Gabor. If it is about securing the
>> communication between the REST client and the REST server, then Flink
>> already supports enabling mutual SSL authentication [1]. Would this be
>> enough to secure the communication and to pass an audit?
>>
>> [1]
>>
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/security/security-ssl/#external--rest-connectivity>>
>> Cheers,
>> Till
>>
>> On Thu, Jun 3, 2021 at 10:33 AM Gabor Somogyi <
[hidden email]>
>> wrote:
>>
>>> Hi Till,
>>>
>>> Since I'm working in security area 10+ years let me share my thought.
>>> I would like to emphasise there are experts better than me but I have
>>> some
>>> basics.
>>> The discussion is open and not trying to tell alone things...
>>>
>>> > I mean if an attacker can get access to one of the machines, then it
>>> should also be possible to obtain the right Kerberos token.
>>> Not necessarily. For example if one gets access to a specific user's
>>> credentials then it's not possible to compromise other user's jobs, data,
>>> etc...
>>> Security is like an onion, the more layers has been added the more time
>>> an
>>> attacker needs to proceed.
>>> At the end of the day if one is in, then most probably can find the way
>>> but
>>> this time is normally enough to sysadmins or security experts to
>>> close down the system and minimize the damage.
>>>
>>> The other thing is that all tokens has a timeout and if the token is
>>> invalid then the attacker can't proceed further.
>>>
>>> > Is Kerberos also the standard authentication protocol for Kubernetes
>>> deployments?
>>> Kerberos is an industry standard which is cloud/deployment agnostic and
>>> it
>>> can be used in any deployments including k8s.
>>> The main intention is to use kerberos in k8s deployments too since we're
>>> going this direction as well.
>>> Please see how Spark does this:
>>>
>>>
https://spark.apache.org/docs/latest/security.html#secure-interaction-with-kubernetes>>>
>>> Last but not least the most important reason to add at least one strong
>>> authentication is that we have users who has
>>> hard requirements on this. They're doing security audits and if they fail
>>> then it's deal breaking.
>>> That is why we have added kerberos at the first place. Unfortunately we
>>> can't name them in this public list, however
>>> the customers who specifically asked for this were mainly in the banking
>>> and telco sector.
>>>
>>> BR,
>>> G
>>>
>>>
>>> On Thu, Jun 3, 2021 at 9:20 AM Till Rohrmann <
[hidden email]>
>>> wrote:
>>>
>>> > Thanks for updating the document Márton. Why is it that banks will
>>> > consider it more secure if Flink comes with Kerberos authentication
>>> > (assuming a properly secured setup)? I mean if an attacker can get
>>> access
>>> > to one of the machines, then it should also be possible to obtain the
>>> right
>>> > Kerberos token.
>>> >
>>> > I am not an authentication expert and that's why I wanted to ask what
>>> are
>>> > other authentication protocols other than Kerberos? Why did we select
>>> > Kerberos and not any other authentication protocol? Maybe you can list
>>> the
>>> > pros and cons for the different protocols. Is Kerberos also the
>>> standard
>>> > authentication protocol for Kubernetes deployments? If not, what would
>>> be
>>> > the answer when deploying on K8s?
>>> >
>>> > Cheers,
>>> > Till
>>> >
>>> > On Wed, Jun 2, 2021 at 12:07 PM Gabor Somogyi <
>>>
[hidden email]>
>>> > wrote:
>>> >
>>> >> Hi team,
>>> >>
>>> >> Happy to be here and hope I can provide quality additions in the
>>> future.
>>> >>
>>> >> Thank you all for helpful the suggestions!
>>> >> Considering them the FLIP has been modified and the work continues on
>>> the
>>> >> already existing Jira.
>>> >>
>>> >> BR,
>>> >> G
>>> >>
>>> >>
>>> >> On Wed, Jun 2, 2021 at 11:23 AM Márton Balassi <
>>>
[hidden email]>
>>> >> wrote:
>>> >>
>>> >>> Thanks, Chesney - I totally missed that. Answered on the ticket too,
>>> let
>>> >>> us continue there then.
>>> >>>
>>> >>> Till, I agree that we should keep this codepath as slim as possible.
>>> It
>>> >>> is an important design decision that we aim to keep the list of
>>> >>> authentication protocols to a minimum. We believe that this should
>>> not be a
>>> >>> primary concern of Flink and a trusted proxy service (for example
>>> Apache
>>> >>> Knox) should be used to enable a multitude of enduser authentication
>>> >>> mechanisms. The bare minimum of authentication mechanisms to support
>>> >>> consequently consist of a single strong authentication protocol for
>>> which
>>> >>> Kerberos is the enterprise solution and HTTP Basic primary for
>>> development
>>> >>> and light-weight scenarios.
>>> >>>
>>> >>> Added the above wording to G's doc.
>>> >>>
>>> >>>
>>>
https://docs.google.com/document/d/1NMPeJ9H0G49TGy3AzTVVJVKmYC0okwOtqLTSPnGqzHw/edit>>> >>>
>>> >>>
>>> >>>
>>> >>> On Tue, Jun 1, 2021 at 11:47 AM Chesnay Schepler <
[hidden email]
>>> >
>>> >>> wrote:
>>> >>>
>>> >>>> There's a related effort:
>>> >>>>
https://issues.apache.org/jira/browse/FLINK-21108>>> >>>>
>>> >>>> On 6/1/2021 10:14 AM, Till Rohrmann wrote:
>>> >>>> > Hi Gabor, welcome to the Flink community!
>>> >>>> >
>>> >>>> > Thanks for sharing this proposal with the community Márton. In
>>> >>>> general, I
>>> >>>> > agree that authentication is missing and that this is required for
>>> >>>> using
>>> >>>> > Flink within an enterprise. The thing I am wondering is whether
>>> this
>>> >>>> > feature strictly needs to be implemented inside of Flink or
>>> whether a
>>> >>>> proxy
>>> >>>> > setup could do the job? Have you considered this option? If yes,
>>> then
>>> >>>> it
>>> >>>> > would be good to list it under the point of rejected alternatives.
>>> >>>> >
>>> >>>> > I do see the benefit of implementing this feature inside of Flink
>>> if
>>> >>>> many
>>> >>>> > users need it. If not, then it might be easier for the project to
>>> not
>>> >>>> > increase the surface area since it makes the overall maintenance
>>> >>>> harder.
>>> >>>> >
>>> >>>> > Cheers,
>>> >>>> > Till
>>> >>>> >
>>> >>>> > On Mon, May 31, 2021 at 4:57 PM Márton Balassi <
>>>
[hidden email]>
>>> >>>> wrote:
>>> >>>> >
>>> >>>> >> Hi team,
>>> >>>> >>
>>> >>>> >> Firstly I would like to introduce Gabor or G [1] for short to the
>>> >>>> >> community, he is a Spark committer who has recently transitioned
>>> to
>>> >>>> the
>>> >>>> >> Flink Engineering team at Cloudera and is looking forward to
>>> >>>> contributing
>>> >>>> >> to Apache Flink. Previously G primarily focused on Spark
>>> Streaming
>>> >>>> and
>>> >>>> >> security.
>>> >>>> >>
>>> >>>> >> Based on requests from our customers G has implemented Kerberos
>>> and
>>> >>>> HTTP
>>> >>>> >> Basic Authentication for the Flink Dashboard and HistoryServer.
>>> >>>> Previously
>>> >>>> >> lacked an authentication story.
>>> >>>> >>
>>> >>>> >> We are looking to contribute this functionality back to the
>>> >>>> community, we
>>> >>>> >> believe that given Flink's maturity there should be a common code
>>> >>>> solution
>>> >>>> >> for this general pattern.
>>> >>>> >>
>>> >>>> >> We are looking forward to your feedback on G's design. [2]
>>> >>>> >>
>>> >>>> >> [1]
http://gaborsomogyi.com/>>> >>>> >> [2]
>>> >>>> >>
>>> >>>> >>
>>> >>>>
>>>
https://docs.google.com/document/d/1NMPeJ9H0G49TGy3AzTVVJVKmYC0okwOtqLTSPnGqzHw/edit>>> >>>> >>
>>> >>>>
>>> >>>>
>>>
>>