[DISCUSS] Improve documentation / tooling around security of Flink

[Robert Metzger]

Hi all,

There was recently a private report to the Flink PMC, as well as publicly
[1] about Flink's ability to execute arbitrary code. In scenarios where
Flink is accessible by somebody unauthorized, this can lead to issues.
The PMC received a similar report in November 2018.

I believe it would be good to warn our users a bit more prominently about
the risks of accidentally opening up Flink to the public internet, or other
unauthorized entities.

I have collected the following potential solutions discussed so far:

a) Add a check-security.sh script, or a check into the frontend if the
JobManager can be reached on the public internet
b) Add a prominent warning to the download page
c) add an opt-out warning to the Flink logs / UI that can be disabled via
the config.
d) Bind the REST endpoint to localhost only, by default


I'm curious to hear if others have other ideas what to do.
I personally like to kick things off with b).


Best,
Robert


[1] https://twitter.com/pyn3rd/status/1197397475897692160

[Chesnay Schepler-3]

Another proposal that was brought up was to provide a script for
generating an SSL certificate with the distribution.

On 12/12/2019 17:45, Robert Metzger wrote:

> Hi all,
>
> There was recently a private report to the Flink PMC, as well as publicly
> [1] about Flink's ability to execute arbitrary code. In scenarios where
> Flink is accessible by somebody unauthorized, this can lead to issues.
> The PMC received a similar report in November 2018.
>
> I believe it would be good to warn our users a bit more prominently about
> the risks of accidentally opening up Flink to the public internet, or other
> unauthorized entities.
>
> I have collected the following potential solutions discussed so far:
>
> a) Add a check-security.sh script, or a check into the frontend if the
> JobManager can be reached on the public internet
> b) Add a prominent warning to the download page
> c) add an opt-out warning to the Flink logs / UI that can be disabled via
> the config.
> d) Bind the REST endpoint to localhost only, by default
>
>
> I'm curious to hear if others have other ideas what to do.
> I personally like to kick things off with b).
>
>
> Best,
> Robert
>
>
> [1] https://twitter.com/pyn3rd/status/1197397475897692160
>

[Konstantin Knauf-3]

Hi Robert,

we could also add a warning (or a general "security" section) to the
"production readiness checklist" in the documentation.

Generally, I like d) in combination with an informative log message. Do you
think this would cause a lot of friction?

Cheers,

Konstantin

On Fri, Dec 13, 2019 at 2:06 PM Chesnay Schepler <[hidden email]> wrote:

> Another proposal that was brought up was to provide a script for
> generating an SSL certificate with the distribution.
>
> On 12/12/2019 17:45, Robert Metzger wrote:
> > Hi all,
> >
> > There was recently a private report to the Flink PMC, as well as publicly
> > [1] about Flink's ability to execute arbitrary code. In scenarios where
> > Flink is accessible by somebody unauthorized, this can lead to issues.
> > The PMC received a similar report in November 2018.
> >
> > I believe it would be good to warn our users a bit more prominently about
> > the risks of accidentally opening up Flink to the public internet, or
> other
> > unauthorized entities.
> >
> > I have collected the following potential solutions discussed so far:
> >
> > a) Add a check-security.sh script, or a check into the frontend if the
> > JobManager can be reached on the public internet
> > b) Add a prominent warning to the download page
> > c) add an opt-out warning to the Flink logs / UI that can be disabled via
> > the config.
> > d) Bind the REST endpoint to localhost only, by default
> >
> >
> > I'm curious to hear if others have other ideas what to do.
> > I personally like to kick things off with b).
> >
> >
> > Best,
> > Robert
> >
> >
> > [1] https://twitter.com/pyn3rd/status/1197397475897692160
> >
>
>

--

Konstantin Knauf | Solutions Architect

[phone]


Follow us @VervericaData Ververica <https://www.ververica.com/>


--

Join Flink Forward <https://flink-forward.org/> - The Apache Flink
Conference

Stream Processing | Event Driven | Real Time

--

Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany

--
Ververica GmbH
Registered at Amtsgericht Charlottenburg: HRB 158244 B
Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
(Tony) Cheng

[Robert Metzger]

Hey,
changes to the network configuration often cause unforeseen trouble, in
particular with things like Kubernetes, Docker etc., and the "onboarding
experience" might suffer due to this.

Updated list:
a) Add a check-security.sh script, or a check into the frontend if the
JobManager can be reached on the public internet
b) Add a prominent warning to the download page and the production
readiness checklist
c) add an opt-out warning to the Flink logs / UI that can be disabled via
the config.
d) Bind the REST endpoint to localhost only, by default
e) provide a script for generating an SSL certificate with the distribution.

On Sun, Dec 15, 2019 at 4:01 PM Konstantin Knauf <[hidden email]>
wrote:

> Hi Robert,
>
> we could also add a warning (or a general "security" section) to the
> "production readiness checklist" in the documentation.
>
> Generally, I like d) in combination with an informative log message. Do
> you think this would cause a lot of friction?
>
> Cheers,
>
> Konstantin
>
> On Fri, Dec 13, 2019 at 2:06 PM Chesnay Schepler <[hidden email]>
> wrote:
>
>> Another proposal that was brought up was to provide a script for
>> generating an SSL certificate with the distribution.
>>
>> On 12/12/2019 17:45, Robert Metzger wrote:
>> > Hi all,
>> >
>> > There was recently a private report to the Flink PMC, as well as
>> publicly
>> > [1] about Flink's ability to execute arbitrary code. In scenarios where
>> > Flink is accessible by somebody unauthorized, this can lead to issues.
>> > The PMC received a similar report in November 2018.
>> >
>> > I believe it would be good to warn our users a bit more prominently
>> about
>> > the risks of accidentally opening up Flink to the public internet, or
>> other
>> > unauthorized entities.
>> >
>> > I have collected the following potential solutions discussed so far:
>> >
>> > a) Add a check-security.sh script, or a check into the frontend if the
>> > JobManager can be reached on the public internet
>> > b) Add a prominent warning to the download page
>> > c) add an opt-out warning to the Flink logs / UI that can be disabled
>> via
>> > the config.
>> > d) Bind the REST endpoint to localhost only, by default
>> >
>> >
>> > I'm curious to hear if others have other ideas what to do.
>> > I personally like to kick things off with b).
>> >
>> >
>> > Best,
>> > Robert
>> >
>> >
>> > [1] https://twitter.com/pyn3rd/status/1197397475897692160
>> >
>>
>>
>
> --
>
> Konstantin Knauf | Solutions Architect
>
> [phone]
>
>
> Follow us @VervericaData Ververica <https://www.ververica.com/>
>
>
> --
>
> Join Flink Forward <https://flink-forward.org/> - The Apache Flink
> Conference
>
> Stream Processing | Event Driven | Real Time
>
> --
>
> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany
>
> --
> Ververica GmbH
> Registered at Amtsgericht Charlottenburg: HRB 158244 B
> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji
> (Tony) Cheng
>