[HEADS UP] Introducing ConfigOptions

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[HEADS UP] Introducing ConfigOptions

Stephan Ewen
Hi all!

A few weeks back we introduced a new way to define configuration parameters.
I would like to encourage everyone to use that new pattern for all new
options that we create, and lazily migrate existing parameters to that
pattern.

The current way of maintaining keys, defaults, and deprecated keys manually
in the "ConfigConstants" class started to become unmaintainable.

The new approach is based on the "ConfigOption" class and defines keys,
deprecated keys, and default values together. For example:

ConfigOption<Double> threshold = ConfigOptions
    .key("cpu.utilization.threshold")
    .defaultValue(0.9).
    .withDeprecatedKeys("cpu.threshold");


Have a look at this class for a complete example:
https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/configuration/HighAvailabilityOptions.java


Greetings,
Stephan
mxm
Reply | Threaded
Open this post in threaded view
|

Re: [HEADS UP] Introducing ConfigOptions

mxm
+1 I like it a lot. Much clearer and better maintainable!

Do we have a plan to migrate ConfigConstants to the new ConfigOptions?
Where do we maintain a list of all config options?

-Max


On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen <[hidden email]> wrote:

> Hi all!
>
> A few weeks back we introduced a new way to define configuration parameters.
> I would like to encourage everyone to use that new pattern for all new
> options that we create, and lazily migrate existing parameters to that
> pattern.
>
> The current way of maintaining keys, defaults, and deprecated keys manually
> in the "ConfigConstants" class started to become unmaintainable.
>
> The new approach is based on the "ConfigOption" class and defines keys,
> deprecated keys, and default values together. For example:
>
> ConfigOption<Double> threshold = ConfigOptions
>     .key("cpu.utilization.threshold")
>     .defaultValue(0.9).
>     .withDeprecatedKeys("cpu.threshold");
>
>
> Have a look at this class for a complete example:
> https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/configuration/HighAvailabilityOptions.java
>
>
> Greetings,
> Stephan
Reply | Threaded
Open this post in threaded view
|

Re: [HEADS UP] Introducing ConfigOptions

Stephan Ewen
Hey!

Here is the parent issue (with per component sub-issues) for the option
migration.

https://issues.apache.org/jira/browse/FLINK-4765

Stephan


On Thu, Oct 27, 2016 at 11:06 AM, Maximilian Michels <[hidden email]> wrote:

> +1 I like it a lot. Much clearer and better maintainable!
>
> Do we have a plan to migrate ConfigConstants to the new ConfigOptions?
> Where do we maintain a list of all config options?
>
> -Max
>
>
> On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen <[hidden email]> wrote:
> > Hi all!
> >
> > A few weeks back we introduced a new way to define configuration
> parameters.
> > I would like to encourage everyone to use that new pattern for all new
> > options that we create, and lazily migrate existing parameters to that
> > pattern.
> >
> > The current way of maintaining keys, defaults, and deprecated keys
> manually
> > in the "ConfigConstants" class started to become unmaintainable.
> >
> > The new approach is based on the "ConfigOption" class and defines keys,
> > deprecated keys, and default values together. For example:
> >
> > ConfigOption<Double> threshold = ConfigOptions
> >     .key("cpu.utilization.threshold")
> >     .defaultValue(0.9).
> >     .withDeprecatedKeys("cpu.threshold");
> >
> >
> > Have a look at this class for a complete example:
> > https://github.com/apache/flink/blob/master/flink-core/
> src/main/java/org/apache/flink/configuration/HighAvailabilityOptions.java
> >
> >
> > Greetings,
> > Stephan
>
Reply | Threaded
Open this post in threaded view
|

Re: [HEADS UP] Introducing ConfigOptions

Robert Metzger
+1 good idea.

I'm wondering whether we can extend this a bit to include a description of
the configuration options as well. This way, we can generate the
configuration page for the documentation from the code. This makes
maintenance for developers really easy.
This is also the approach Apache Kafka is following:
http://kafka.apache.org/documentation#brokerconfigs (and
https://github.com/apache/kafka/blob/60ad6d727861a87fa756918a7be7547e9b1f4c3d/clients/src/main/java/org/apache/kafka/clients/consumer/ConsumerConfig.java
)


On Thu, Oct 27, 2016 at 2:52 PM, Stephan Ewen <[hidden email]> wrote:

> Hey!
>
> Here is the parent issue (with per component sub-issues) for the option
> migration.
>
> https://issues.apache.org/jira/browse/FLINK-4765
>
> Stephan
>
>
> On Thu, Oct 27, 2016 at 11:06 AM, Maximilian Michels <[hidden email]>
> wrote:
>
> > +1 I like it a lot. Much clearer and better maintainable!
> >
> > Do we have a plan to migrate ConfigConstants to the new ConfigOptions?
> > Where do we maintain a list of all config options?
> >
> > -Max
> >
> >
> > On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen <[hidden email]> wrote:
> > > Hi all!
> > >
> > > A few weeks back we introduced a new way to define configuration
> > parameters.
> > > I would like to encourage everyone to use that new pattern for all new
> > > options that we create, and lazily migrate existing parameters to that
> > > pattern.
> > >
> > > The current way of maintaining keys, defaults, and deprecated keys
> > manually
> > > in the "ConfigConstants" class started to become unmaintainable.
> > >
> > > The new approach is based on the "ConfigOption" class and defines keys,
> > > deprecated keys, and default values together. For example:
> > >
> > > ConfigOption<Double> threshold = ConfigOptions
> > >     .key("cpu.utilization.threshold")
> > >     .defaultValue(0.9).
> > >     .withDeprecatedKeys("cpu.threshold");
> > >
> > >
> > > Have a look at this class for a complete example:
> > > https://github.com/apache/flink/blob/master/flink-core/
> > src/main/java/org/apache/flink/configuration/
> HighAvailabilityOptions.java
> > >
> > >
> > > Greetings,
> > > Stephan
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [HEADS UP] Introducing ConfigOptions

Stephan Ewen
I think adding descriptions to the options and auto-generating the config
docs is a very good followup.

How about we do that after we migrated all options? Then we can add the
docs generation logic in that step as well.

On Thu, Oct 27, 2016 at 5:22 PM, Robert Metzger <[hidden email]> wrote:

> +1 good idea.
>
> I'm wondering whether we can extend this a bit to include a description of
> the configuration options as well. This way, we can generate the
> configuration page for the documentation from the code. This makes
> maintenance for developers really easy.
> This is also the approach Apache Kafka is following:
> http://kafka.apache.org/documentation#brokerconfigs (and
> https://github.com/apache/kafka/blob/60ad6d727861a87fa756918a7be754
> 7e9b1f4c3d/clients/src/main/java/org/apache/kafka/clients/
> consumer/ConsumerConfig.java
> )
>
>
> On Thu, Oct 27, 2016 at 2:52 PM, Stephan Ewen <[hidden email]> wrote:
>
> > Hey!
> >
> > Here is the parent issue (with per component sub-issues) for the option
> > migration.
> >
> > https://issues.apache.org/jira/browse/FLINK-4765
> >
> > Stephan
> >
> >
> > On Thu, Oct 27, 2016 at 11:06 AM, Maximilian Michels <[hidden email]>
> > wrote:
> >
> > > +1 I like it a lot. Much clearer and better maintainable!
> > >
> > > Do we have a plan to migrate ConfigConstants to the new ConfigOptions?
> > > Where do we maintain a list of all config options?
> > >
> > > -Max
> > >
> > >
> > > On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen <[hidden email]>
> wrote:
> > > > Hi all!
> > > >
> > > > A few weeks back we introduced a new way to define configuration
> > > parameters.
> > > > I would like to encourage everyone to use that new pattern for all
> new
> > > > options that we create, and lazily migrate existing parameters to
> that
> > > > pattern.
> > > >
> > > > The current way of maintaining keys, defaults, and deprecated keys
> > > manually
> > > > in the "ConfigConstants" class started to become unmaintainable.
> > > >
> > > > The new approach is based on the "ConfigOption" class and defines
> keys,
> > > > deprecated keys, and default values together. For example:
> > > >
> > > > ConfigOption<Double> threshold = ConfigOptions
> > > >     .key("cpu.utilization.threshold")
> > > >     .defaultValue(0.9).
> > > >     .withDeprecatedKeys("cpu.threshold");
> > > >
> > > >
> > > > Have a look at this class for a complete example:
> > > > https://github.com/apache/flink/blob/master/flink-core/
> > > src/main/java/org/apache/flink/configuration/
> > HighAvailabilityOptions.java
> > > >
> > > >
> > > > Greetings,
> > > > Stephan
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [HEADS UP] Introducing ConfigOptions

伍翀(云邪)
+1

The automatic configuration page generating sounds really cool.

- Jark Wu

> 在 2016年10月28日,上午12:23,Stephan Ewen <[hidden email]> 写道:
>
> I think adding descriptions to the options and auto-generating the config
> docs is a very good followup.
>
> How about we do that after we migrated all options? Then we can add the
> docs generation logic in that step as well.
>
> On Thu, Oct 27, 2016 at 5:22 PM, Robert Metzger <[hidden email]> wrote:
>
>> +1 good idea.
>>
>> I'm wondering whether we can extend this a bit to include a description of
>> the configuration options as well. This way, we can generate the
>> configuration page for the documentation from the code. This makes
>> maintenance for developers really easy.
>> This is also the approach Apache Kafka is following:
>> http://kafka.apache.org/documentation#brokerconfigs (and
>> https://github.com/apache/kafka/blob/60ad6d727861a87fa756918a7be754
>> 7e9b1f4c3d/clients/src/main/java/org/apache/kafka/clients/
>> consumer/ConsumerConfig.java
>> )
>>
>>
>> On Thu, Oct 27, 2016 at 2:52 PM, Stephan Ewen <[hidden email]> wrote:
>>
>>> Hey!
>>>
>>> Here is the parent issue (with per component sub-issues) for the option
>>> migration.
>>>
>>> https://issues.apache.org/jira/browse/FLINK-4765
>>>
>>> Stephan
>>>
>>>
>>> On Thu, Oct 27, 2016 at 11:06 AM, Maximilian Michels <[hidden email]>
>>> wrote:
>>>
>>>> +1 I like it a lot. Much clearer and better maintainable!
>>>>
>>>> Do we have a plan to migrate ConfigConstants to the new ConfigOptions?
>>>> Where do we maintain a list of all config options?
>>>>
>>>> -Max
>>>>
>>>>
>>>> On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen <[hidden email]>
>> wrote:
>>>>> Hi all!
>>>>>
>>>>> A few weeks back we introduced a new way to define configuration
>>>> parameters.
>>>>> I would like to encourage everyone to use that new pattern for all
>> new
>>>>> options that we create, and lazily migrate existing parameters to
>> that
>>>>> pattern.
>>>>>
>>>>> The current way of maintaining keys, defaults, and deprecated keys
>>>> manually
>>>>> in the "ConfigConstants" class started to become unmaintainable.
>>>>>
>>>>> The new approach is based on the "ConfigOption" class and defines
>> keys,
>>>>> deprecated keys, and default values together. For example:
>>>>>
>>>>> ConfigOption<Double> threshold = ConfigOptions
>>>>>    .key("cpu.utilization.threshold")
>>>>>    .defaultValue(0.9).
>>>>>    .withDeprecatedKeys("cpu.threshold");
>>>>>
>>>>>
>>>>> Have a look at this class for a complete example:
>>>>> https://github.com/apache/flink/blob/master/flink-core/
>>>> src/main/java/org/apache/flink/configuration/
>>> HighAvailabilityOptions.java
>>>>>
>>>>>
>>>>> Greetings,
>>>>> Stephan
>>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: [HEADS UP] Introducing ConfigOptions

Robert Metzger
You are right Stephan. The doc migration will be easier when we use the new
logic for all keys.

On Fri, Oct 28, 2016 at 4:27 AM, Jark Wu <[hidden email]> wrote:

> +1
>
> The automatic configuration page generating sounds really cool.
>
> - Jark Wu
>
> > 在 2016年10月28日,上午12:23,Stephan Ewen <[hidden email]> 写道:
> >
> > I think adding descriptions to the options and auto-generating the config
> > docs is a very good followup.
> >
> > How about we do that after we migrated all options? Then we can add the
> > docs generation logic in that step as well.
> >
> > On Thu, Oct 27, 2016 at 5:22 PM, Robert Metzger <[hidden email]>
> wrote:
> >
> >> +1 good idea.
> >>
> >> I'm wondering whether we can extend this a bit to include a description
> of
> >> the configuration options as well. This way, we can generate the
> >> configuration page for the documentation from the code. This makes
> >> maintenance for developers really easy.
> >> This is also the approach Apache Kafka is following:
> >> http://kafka.apache.org/documentation#brokerconfigs (and
> >> https://github.com/apache/kafka/blob/60ad6d727861a87fa756918a7be754
> >> 7e9b1f4c3d/clients/src/main/java/org/apache/kafka/clients/
> >> consumer/ConsumerConfig.java
> >> )
> >>
> >>
> >> On Thu, Oct 27, 2016 at 2:52 PM, Stephan Ewen <[hidden email]> wrote:
> >>
> >>> Hey!
> >>>
> >>> Here is the parent issue (with per component sub-issues) for the option
> >>> migration.
> >>>
> >>> https://issues.apache.org/jira/browse/FLINK-4765
> >>>
> >>> Stephan
> >>>
> >>>
> >>> On Thu, Oct 27, 2016 at 11:06 AM, Maximilian Michels <[hidden email]>
> >>> wrote:
> >>>
> >>>> +1 I like it a lot. Much clearer and better maintainable!
> >>>>
> >>>> Do we have a plan to migrate ConfigConstants to the new ConfigOptions?
> >>>> Where do we maintain a list of all config options?
> >>>>
> >>>> -Max
> >>>>
> >>>>
> >>>> On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen <[hidden email]>
> >> wrote:
> >>>>> Hi all!
> >>>>>
> >>>>> A few weeks back we introduced a new way to define configuration
> >>>> parameters.
> >>>>> I would like to encourage everyone to use that new pattern for all
> >> new
> >>>>> options that we create, and lazily migrate existing parameters to
> >> that
> >>>>> pattern.
> >>>>>
> >>>>> The current way of maintaining keys, defaults, and deprecated keys
> >>>> manually
> >>>>> in the "ConfigConstants" class started to become unmaintainable.
> >>>>>
> >>>>> The new approach is based on the "ConfigOption" class and defines
> >> keys,
> >>>>> deprecated keys, and default values together. For example:
> >>>>>
> >>>>> ConfigOption<Double> threshold = ConfigOptions
> >>>>>    .key("cpu.utilization.threshold")
> >>>>>    .defaultValue(0.9).
> >>>>>    .withDeprecatedKeys("cpu.threshold");
> >>>>>
> >>>>>
> >>>>> Have a look at this class for a complete example:
> >>>>> https://github.com/apache/flink/blob/master/flink-core/
> >>>> src/main/java/org/apache/flink/configuration/
> >>> HighAvailabilityOptions.java
> >>>>>
> >>>>>
> >>>>> Greetings,
> >>>>> Stephan
> >>>>
> >>>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [HEADS UP] Introducing ConfigOptions

Till Rohrmann
+1 for the ConfigOptions and the automatic configuration description
generation once we've migrated all config keys.

Cheers,
Till

On Fri, Oct 28, 2016 at 2:18 PM, Robert Metzger <[hidden email]> wrote:

> You are right Stephan. The doc migration will be easier when we use the new
> logic for all keys.
>
> On Fri, Oct 28, 2016 at 4:27 AM, Jark Wu <[hidden email]>
> wrote:
>
> > +1
> >
> > The automatic configuration page generating sounds really cool.
> >
> > - Jark Wu
> >
> > > 在 2016年10月28日,上午12:23,Stephan Ewen <[hidden email]> 写道:
> > >
> > > I think adding descriptions to the options and auto-generating the
> config
> > > docs is a very good followup.
> > >
> > > How about we do that after we migrated all options? Then we can add the
> > > docs generation logic in that step as well.
> > >
> > > On Thu, Oct 27, 2016 at 5:22 PM, Robert Metzger <[hidden email]>
> > wrote:
> > >
> > >> +1 good idea.
> > >>
> > >> I'm wondering whether we can extend this a bit to include a
> description
> > of
> > >> the configuration options as well. This way, we can generate the
> > >> configuration page for the documentation from the code. This makes
> > >> maintenance for developers really easy.
> > >> This is also the approach Apache Kafka is following:
> > >> http://kafka.apache.org/documentation#brokerconfigs (and
> > >> https://github.com/apache/kafka/blob/60ad6d727861a87fa756918a7be754
> > >> 7e9b1f4c3d/clients/src/main/java/org/apache/kafka/clients/
> > >> consumer/ConsumerConfig.java
> > >> )
> > >>
> > >>
> > >> On Thu, Oct 27, 2016 at 2:52 PM, Stephan Ewen <[hidden email]>
> wrote:
> > >>
> > >>> Hey!
> > >>>
> > >>> Here is the parent issue (with per component sub-issues) for the
> option
> > >>> migration.
> > >>>
> > >>> https://issues.apache.org/jira/browse/FLINK-4765
> > >>>
> > >>> Stephan
> > >>>
> > >>>
> > >>> On Thu, Oct 27, 2016 at 11:06 AM, Maximilian Michels <[hidden email]
> >
> > >>> wrote:
> > >>>
> > >>>> +1 I like it a lot. Much clearer and better maintainable!
> > >>>>
> > >>>> Do we have a plan to migrate ConfigConstants to the new
> ConfigOptions?
> > >>>> Where do we maintain a list of all config options?
> > >>>>
> > >>>> -Max
> > >>>>
> > >>>>
> > >>>> On Wed, Oct 26, 2016 at 2:36 PM, Stephan Ewen <[hidden email]>
> > >> wrote:
> > >>>>> Hi all!
> > >>>>>
> > >>>>> A few weeks back we introduced a new way to define configuration
> > >>>> parameters.
> > >>>>> I would like to encourage everyone to use that new pattern for all
> > >> new
> > >>>>> options that we create, and lazily migrate existing parameters to
> > >> that
> > >>>>> pattern.
> > >>>>>
> > >>>>> The current way of maintaining keys, defaults, and deprecated keys
> > >>>> manually
> > >>>>> in the "ConfigConstants" class started to become unmaintainable.
> > >>>>>
> > >>>>> The new approach is based on the "ConfigOption" class and defines
> > >> keys,
> > >>>>> deprecated keys, and default values together. For example:
> > >>>>>
> > >>>>> ConfigOption<Double> threshold = ConfigOptions
> > >>>>>    .key("cpu.utilization.threshold")
> > >>>>>    .defaultValue(0.9).
> > >>>>>    .withDeprecatedKeys("cpu.threshold");
> > >>>>>
> > >>>>>
> > >>>>> Have a look at this class for a complete example:
> > >>>>> https://github.com/apache/flink/blob/master/flink-core/
> > >>>> src/main/java/org/apache/flink/configuration/
> > >>> HighAvailabilityOptions.java
> > >>>>>
> > >>>>>
> > >>>>> Greetings,
> > >>>>> Stephan
> > >>>>
> > >>>
> > >>
> >
> >
>