[DISCUSS] Rename the SQL ANY type to OPAQUE type

[Timo Walther-2]

Hi everyone,

Stephan pointed out that our naming of a generic/blackbox/opaque type in
SQL might be not intuitive for users. As the term ANY rather describes a
"super-class of all types" which is not the case in our type system. Our
current ANY type stands for a type that is just a blackbox within SQL,
serialized by some custom serializer, that can only be modified within UDFs.

I also gathered feedback from a training instructor and native English
speaker (David in CC) where I received the following:

"The way I’m thinking about this is this: there’s a concept here that
people have to become aware of, which is that Flink SQL is able to
operate generically on opaquely typed things — and folks need to be able
to connect what they see in code examples, etc. with this concept (which
they may be unaware of initially).
I feel like ANY misses the mark a little bit, but isn’t particularly
bad. I do worry that it may cause some confusion about its purpose and
power. I think OPAQUE would more clearly express what’s going on."

Also resources like Wikipedia [1] show that this terminology is common:

"a data type whose concrete data structure is not defined [...] its
values can only be manipulated by calling subroutines that have access
to the missing information"

I would therefore vote for refactoring the type name because it is not
used much yet.

Implications are:

- a new parser keyword "OPAQUE" and changed SQL parser

- changes for logical type root, logical type visitors, and their usages

What do you think?

Thanks,

Timo

[1] https://en.wikipedia.org/wiki/Opaque_data_type

[Xuefu Z]

Thanks to Timo for bringing up an interesting topic.

Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It
suits pretty well to glasses, thought. :)) Anyway, could we just use
"UNKNOWN", which is more explicit and true reflects its nature?

Thanks,
Xuefu


On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]> wrote:

> Hi everyone,
>
> Stephan pointed out that our naming of a generic/blackbox/opaque type in
> SQL might be not intuitive for users. As the term ANY rather describes a
> "super-class of all types" which is not the case in our type system. Our
> current ANY type stands for a type that is just a blackbox within SQL,
> serialized by some custom serializer, that can only be modified within
> UDFs.
>
> I also gathered feedback from a training instructor and native English
> speaker (David in CC) where I received the following:
>
> "The way I’m thinking about this is this: there’s a concept here that
> people have to become aware of, which is that Flink SQL is able to
> operate generically on opaquely typed things — and folks need to be able
> to connect what they see in code examples, etc. with this concept (which
> they may be unaware of initially).
> I feel like ANY misses the mark a little bit, but isn’t particularly
> bad. I do worry that it may cause some confusion about its purpose and
> power. I think OPAQUE would more clearly express what’s going on."
>
> Also resources like Wikipedia [1] show that this terminology is common:
>
> "a data type whose concrete data structure is not defined [...] its
> values can only be manipulated by calling subroutines that have access
> to the missing information"
>
> I would therefore vote for refactoring the type name because it is not
> used much yet.
>
> Implications are:
>
> - a new parser keyword "OPAQUE" and changed SQL parser
>
> - changes for logical type root, logical type visitors, and their usages
>
> What do you think?
>
> Thanks,
>
> Timo
>
> [1] https://en.wikipedia.org/wiki/Opaque_data_type
>
>
>

--
Xuefu Zhang

"In Honey We Trust!"

[Aljoscha Krettek-2]

I prefer OPAQUE compared to ANY because any is often the root object in an object hierarchy and would indicate to users the wrong thing.

Aljoscha

> On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
>
> Thanks to Timo for bringing up an interesting topic.
>
> Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It
> suits pretty well to glasses, thought. :)) Anyway, could we just use
> "UNKNOWN", which is more explicit and true reflects its nature?
>
> Thanks,
> Xuefu
>
>
> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]> wrote:
>
>> Hi everyone,
>>
>> Stephan pointed out that our naming of a generic/blackbox/opaque type in
>> SQL might be not intuitive for users. As the term ANY rather describes a
>> "super-class of all types" which is not the case in our type system. Our
>> current ANY type stands for a type that is just a blackbox within SQL,
>> serialized by some custom serializer, that can only be modified within
>> UDFs.
>>
>> I also gathered feedback from a training instructor and native English
>> speaker (David in CC) where I received the following:
>>
>> "The way I’m thinking about this is this: there’s a concept here that
>> people have to become aware of, which is that Flink SQL is able to
>> operate generically on opaquely typed things — and folks need to be able
>> to connect what they see in code examples, etc. with this concept (which
>> they may be unaware of initially).
>> I feel like ANY misses the mark a little bit, but isn’t particularly
>> bad. I do worry that it may cause some confusion about its purpose and
>> power. I think OPAQUE would more clearly express what’s going on."
>>
>> Also resources like Wikipedia [1] show that this terminology is common:
>>
>> "a data type whose concrete data structure is not defined [...] its
>> values can only be manipulated by calling subroutines that have access
>> to the missing information"
>>
>> I would therefore vote for refactoring the type name because it is not
>> used much yet.
>>
>> Implications are:
>>
>> - a new parser keyword "OPAQUE" and changed SQL parser
>>
>> - changes for logical type root, logical type visitors, and their usages
>>
>> What do you think?
>>
>> Thanks,
>>
>> Timo
>>
>> [1] https://en.wikipedia.org/wiki/Opaque_data_type
>>
>>
>>
>
> --
> Xuefu Zhang
>
> "In Honey We Trust!"

[Jark Wu-2]

+1 to rename ANY.

I don't have strong opinion on the new name. I think "OPAQUE" is fine,
because it is introduced in IBM Informix and Oracle.
In Informix, it says [1]:

"An opaque data type is fully encapsulated; the database server does not
know about the internal format of an opaque data type.
Therefore, the database server cannot make assumptions about how to access
a column having an opaque data type.
The database developer defines a data structure that holds the opaque-type
information and support functions
that tell the database server how to access this data structure."

So, I think "opaque" is fine here.

Another option is "RAW" which is introduced by Oracle [2] represents data
that is not to be interpreted by system .

Best,
Jark

[1]:
https://www.ibm.com/support/knowledgecenter/en/SSGU8G_11.70.0/com.ibm.esqlc.doc/ids_esqlc_0357.htm
[2]:
https://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT613

On Mon, 21 Oct 2019 at 15:49, Aljoscha Krettek <[hidden email]> wrote:

> I prefer OPAQUE compared to ANY because any is often the root object in an
> object hierarchy and would indicate to users the wrong thing.
>
> Aljoscha
>
> > On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
> >
> > Thanks to Timo for bringing up an interesting topic.
> >
> > Personally, "OPAQUE" doesn't seem very intuitive with respect to types.
> (It
> > suits pretty well to glasses, thought. :)) Anyway, could we just use
> > "UNKNOWN", which is more explicit and true reflects its nature?
> >
> > Thanks,
> > Xuefu
> >
> >
> > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]> wrote:
> >
> >> Hi everyone,
> >>
> >> Stephan pointed out that our naming of a generic/blackbox/opaque type in
> >> SQL might be not intuitive for users. As the term ANY rather describes a
> >> "super-class of all types" which is not the case in our type system. Our
> >> current ANY type stands for a type that is just a blackbox within SQL,
> >> serialized by some custom serializer, that can only be modified within
> >> UDFs.
> >>
> >> I also gathered feedback from a training instructor and native English
> >> speaker (David in CC) where I received the following:
> >>
> >> "The way I’m thinking about this is this: there’s a concept here that
> >> people have to become aware of, which is that Flink SQL is able to
> >> operate generically on opaquely typed things — and folks need to be able
> >> to connect what they see in code examples, etc. with this concept (which
> >> they may be unaware of initially).
> >> I feel like ANY misses the mark a little bit, but isn’t particularly
> >> bad. I do worry that it may cause some confusion about its purpose and
> >> power. I think OPAQUE would more clearly express what’s going on."
> >>
> >> Also resources like Wikipedia [1] show that this terminology is common:
> >>
> >> "a data type whose concrete data structure is not defined [...] its
> >> values can only be manipulated by calling subroutines that have access
> >> to the missing information"
> >>
> >> I would therefore vote for refactoring the type name because it is not
> >> used much yet.
> >>
> >> Implications are:
> >>
> >> - a new parser keyword "OPAQUE" and changed SQL parser
> >>
> >> - changes for logical type root, logical type visitors, and their usages
> >>
> >> What do you think?
> >>
> >> Thanks,
> >>
> >> Timo
> >>
> >> [1] https://en.wikipedia.org/wiki/Opaque_data_type
> >>
> >>
> >>
> >
> > --
> > Xuefu Zhang
> >
> > "In Honey We Trust!"
>
>

[Kurt Young]

OPAQUE seems to be a little bit advanced to a lot non-english
speakers (including me). I think Xuefu raised a good alternative:
UNKNOWN. What do you think about it?

Best,
Kurt


On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek <[hidden email]>
wrote:

> I prefer OPAQUE compared to ANY because any is often the root object in an
> object hierarchy and would indicate to users the wrong thing.
>
> Aljoscha
>
> > On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
> >
> > Thanks to Timo for bringing up an interesting topic.
> >
> > Personally, "OPAQUE" doesn't seem very intuitive with respect to types.
> (It
> > suits pretty well to glasses, thought. :)) Anyway, could we just use
> > "UNKNOWN", which is more explicit and true reflects its nature?
> >
> > Thanks,
> > Xuefu
> >
> >
> > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]> wrote:
> >
> >> Hi everyone,
> >>
> >> Stephan pointed out that our naming of a generic/blackbox/opaque type in
> >> SQL might be not intuitive for users. As the term ANY rather describes a
> >> "super-class of all types" which is not the case in our type system. Our
> >> current ANY type stands for a type that is just a blackbox within SQL,
> >> serialized by some custom serializer, that can only be modified within
> >> UDFs.
> >>
> >> I also gathered feedback from a training instructor and native English
> >> speaker (David in CC) where I received the following:
> >>
> >> "The way I’m thinking about this is this: there’s a concept here that
> >> people have to become aware of, which is that Flink SQL is able to
> >> operate generically on opaquely typed things — and folks need to be able
> >> to connect what they see in code examples, etc. with this concept (which
> >> they may be unaware of initially).
> >> I feel like ANY misses the mark a little bit, but isn’t particularly
> >> bad. I do worry that it may cause some confusion about its purpose and
> >> power. I think OPAQUE would more clearly express what’s going on."
> >>
> >> Also resources like Wikipedia [1] show that this terminology is common:
> >>
> >> "a data type whose concrete data structure is not defined [...] its
> >> values can only be manipulated by calling subroutines that have access
> >> to the missing information"
> >>
> >> I would therefore vote for refactoring the type name because it is not
> >> used much yet.
> >>
> >> Implications are:
> >>
> >> - a new parser keyword "OPAQUE" and changed SQL parser
> >>
> >> - changes for logical type root, logical type visitors, and their usages
> >>
> >> What do you think?
> >>
> >> Thanks,
> >>
> >> Timo
> >>
> >> [1] https://en.wikipedia.org/wiki/Opaque_data_type
> >>
> >>
> >>
> >
> > --
> > Xuefu Zhang
> >
> > "In Honey We Trust!"
>
>

[PaulLam]

Hi,

IMHO, `UNKNOWN` does not fully reflects the situation here, because the types are
actually “known” to users, and users just want to leave them out of Flink type system.

+1 for `RAW`, for it's more intuitive than `OPAQUE`.

Best,
Paul Lam

> 在 2019年10月21日，16:43，Kurt Young <[hidden email]> 写道：
>
> OPAQUE seems to be a little bit advanced to a lot non-english
> speakers (including me). I think Xuefu raised a good alternative:
> UNKNOWN. What do you think about it?
>
> Best,
> Kurt
>
>
> On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek <[hidden email]>
> wrote:
>
>> I prefer OPAQUE compared to ANY because any is often the root object in an
>> object hierarchy and would indicate to users the wrong thing.
>>
>> Aljoscha
>>
>>> On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
>>>
>>> Thanks to Timo for bringing up an interesting topic.
>>>
>>> Personally, "OPAQUE" doesn't seem very intuitive with respect to types.
>> (It
>>> suits pretty well to glasses, thought. :)) Anyway, could we just use
>>> "UNKNOWN", which is more explicit and true reflects its nature?
>>>
>>> Thanks,
>>> Xuefu
>>>
>>>
>>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> Stephan pointed out that our naming of a generic/blackbox/opaque type in
>>>> SQL might be not intuitive for users. As the term ANY rather describes a
>>>> "super-class of all types" which is not the case in our type system. Our
>>>> current ANY type stands for a type that is just a blackbox within SQL,
>>>> serialized by some custom serializer, that can only be modified within
>>>> UDFs.
>>>>
>>>> I also gathered feedback from a training instructor and native English
>>>> speaker (David in CC) where I received the following:
>>>>
>>>> "The way I’m thinking about this is this: there’s a concept here that
>>>> people have to become aware of, which is that Flink SQL is able to
>>>> operate generically on opaquely typed things — and folks need to be able
>>>> to connect what they see in code examples, etc. with this concept (which
>>>> they may be unaware of initially).
>>>> I feel like ANY misses the mark a little bit, but isn’t particularly
>>>> bad. I do worry that it may cause some confusion about its purpose and
>>>> power. I think OPAQUE would more clearly express what’s going on."
>>>>
>>>> Also resources like Wikipedia [1] show that this terminology is common:
>>>>
>>>> "a data type whose concrete data structure is not defined [...] its
>>>> values can only be manipulated by calling subroutines that have access
>>>> to the missing information"
>>>>
>>>> I would therefore vote for refactoring the type name because it is not
>>>> used much yet.
>>>>
>>>> Implications are:
>>>>
>>>> - a new parser keyword "OPAQUE" and changed SQL parser
>>>>
>>>> - changes for logical type root, logical type visitors, and their usages
>>>>
>>>> What do you think?
>>>>
>>>> Thanks,
>>>>
>>>> Timo
>>>>
>>>> [1] https://en.wikipedia.org/wiki/Opaque_data_type
>>>>
>>>>
>>>>
>>>
>>> --
>>> Xuefu Zhang
>>>
>>> "In Honey We Trust!"
>>
>>

[Jark Wu-2]

I also think `UNKNOWN` is not suitable here.
Because we already have `UNKNOWN` value in SQL, i.e. the three-valued logic
(TRUE, FALSE, UNKNOWN) of BOOLEAN type.
It will confuse users here, what's the relationship between them.

Best,
Jark

On Mon, 21 Oct 2019 at 17:53, Paul Lam <[hidden email]> wrote:

> Hi,
>
> IMHO, `UNKNOWN` does not fully reflects the situation here, because the
> types are
> actually “known” to users, and users just want to leave them out of Flink
> type system.
>
> +1 for `RAW`, for it's more intuitive than `OPAQUE`.
>
> Best,
> Paul Lam
>
> > 在 2019年10月21日，16:43，Kurt Young <[hidden email]> 写道：
> >
> > OPAQUE seems to be a little bit advanced to a lot non-english
> > speakers (including me). I think Xuefu raised a good alternative:
> > UNKNOWN. What do you think about it?
> >
> > Best,
> > Kurt
> >
> >
> > On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek <[hidden email]>
> > wrote:
> >
> >> I prefer OPAQUE compared to ANY because any is often the root object in
> an
> >> object hierarchy and would indicate to users the wrong thing.
> >>
> >> Aljoscha
> >>
> >>> On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
> >>>
> >>> Thanks to Timo for bringing up an interesting topic.
> >>>
> >>> Personally, "OPAQUE" doesn't seem very intuitive with respect to types.
> >> (It
> >>> suits pretty well to glasses, thought. :)) Anyway, could we just use
> >>> "UNKNOWN", which is more explicit and true reflects its nature?
> >>>
> >>> Thanks,
> >>> Xuefu
> >>>
> >>>
> >>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]>
> wrote:
> >>>
> >>>> Hi everyone,
> >>>>
> >>>> Stephan pointed out that our naming of a generic/blackbox/opaque type
> in
> >>>> SQL might be not intuitive for users. As the term ANY rather
> describes a
> >>>> "super-class of all types" which is not the case in our type system.
> Our
> >>>> current ANY type stands for a type that is just a blackbox within SQL,
> >>>> serialized by some custom serializer, that can only be modified within
> >>>> UDFs.
> >>>>
> >>>> I also gathered feedback from a training instructor and native English
> >>>> speaker (David in CC) where I received the following:
> >>>>
> >>>> "The way I’m thinking about this is this: there’s a concept here that
> >>>> people have to become aware of, which is that Flink SQL is able to
> >>>> operate generically on opaquely typed things — and folks need to be
> able
> >>>> to connect what they see in code examples, etc. with this concept
> (which
> >>>> they may be unaware of initially).
> >>>> I feel like ANY misses the mark a little bit, but isn’t particularly
> >>>> bad. I do worry that it may cause some confusion about its purpose and
> >>>> power. I think OPAQUE would more clearly express what’s going on."
> >>>>
> >>>> Also resources like Wikipedia [1] show that this terminology is
> common:
> >>>>
> >>>> "a data type whose concrete data structure is not defined [...] its
> >>>> values can only be manipulated by calling subroutines that have access
> >>>> to the missing information"
> >>>>
> >>>> I would therefore vote for refactoring the type name because it is not
> >>>> used much yet.
> >>>>
> >>>> Implications are:
> >>>>
> >>>> - a new parser keyword "OPAQUE" and changed SQL parser
> >>>>
> >>>> - changes for logical type root, logical type visitors, and their
> usages
> >>>>
> >>>> What do you think?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Timo
> >>>>
> >>>> [1] https://en.wikipedia.org/wiki/Opaque_data_type
> >>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Xuefu Zhang
> >>>
> >>> "In Honey We Trust!"
> >>
> >>
>
>

[Timo Walther-2]

I would also avoid `UNKNOWN` because of the mentioned reasons.

I'm fine with `RAW`. I will wait another day or two until I conclude the
discussion.

Thanks,
Timo


On 21.10.19 12:23, Jark Wu wrote:

> I also think `UNKNOWN` is not suitable here.
> Because we already have `UNKNOWN` value in SQL, i.e. the three-valued logic
> (TRUE, FALSE, UNKNOWN) of BOOLEAN type.
> It will confuse users here, what's the relationship between them.
>
> Best,
> Jark
>
> On Mon, 21 Oct 2019 at 17:53, Paul Lam <[hidden email]> wrote:
>
>> Hi,
>>
>> IMHO, `UNKNOWN` does not fully reflects the situation here, because the
>> types are
>> actually “known” to users, and users just want to leave them out of Flink
>> type system.
>>
>> +1 for `RAW`, for it's more intuitive than `OPAQUE`.
>>
>> Best,
>> Paul Lam
>>
>>> 在 2019年10月21日，16:43，Kurt Young <[hidden email]> 写道：
>>>
>>> OPAQUE seems to be a little bit advanced to a lot non-english
>>> speakers (including me). I think Xuefu raised a good alternative:
>>> UNKNOWN. What do you think about it?
>>>
>>> Best,
>>> Kurt
>>>
>>>
>>> On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek <[hidden email]>
>>> wrote:
>>>
>>>> I prefer OPAQUE compared to ANY because any is often the root object in
>> an
>>>> object hierarchy and would indicate to users the wrong thing.
>>>>
>>>> Aljoscha
>>>>
>>>>> On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
>>>>>
>>>>> Thanks to Timo for bringing up an interesting topic.
>>>>>
>>>>> Personally, "OPAQUE" doesn't seem very intuitive with respect to types.
>>>> (It
>>>>> suits pretty well to glasses, thought. :)) Anyway, could we just use
>>>>> "UNKNOWN", which is more explicit and true reflects its nature?
>>>>>
>>>>> Thanks,
>>>>> Xuefu
>>>>>
>>>>>
>>>>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]>
>> wrote:
>>>>>> Hi everyone,
>>>>>>
>>>>>> Stephan pointed out that our naming of a generic/blackbox/opaque type
>> in
>>>>>> SQL might be not intuitive for users. As the term ANY rather
>> describes a
>>>>>> "super-class of all types" which is not the case in our type system.
>> Our
>>>>>> current ANY type stands for a type that is just a blackbox within SQL,
>>>>>> serialized by some custom serializer, that can only be modified within
>>>>>> UDFs.
>>>>>>
>>>>>> I also gathered feedback from a training instructor and native English
>>>>>> speaker (David in CC) where I received the following:
>>>>>>
>>>>>> "The way I’m thinking about this is this: there’s a concept here that
>>>>>> people have to become aware of, which is that Flink SQL is able to
>>>>>> operate generically on opaquely typed things — and folks need to be
>> able
>>>>>> to connect what they see in code examples, etc. with this concept
>> (which
>>>>>> they may be unaware of initially).
>>>>>> I feel like ANY misses the mark a little bit, but isn’t particularly
>>>>>> bad. I do worry that it may cause some confusion about its purpose and
>>>>>> power. I think OPAQUE would more clearly express what’s going on."
>>>>>>
>>>>>> Also resources like Wikipedia [1] show that this terminology is
>> common:
>>>>>> "a data type whose concrete data structure is not defined [...] its
>>>>>> values can only be manipulated by calling subroutines that have access
>>>>>> to the missing information"
>>>>>>
>>>>>> I would therefore vote for refactoring the type name because it is not
>>>>>> used much yet.
>>>>>>
>>>>>> Implications are:
>>>>>>
>>>>>> - a new parser keyword "OPAQUE" and changed SQL parser
>>>>>>
>>>>>> - changes for logical type root, logical type visitors, and their
>> usages
>>>>>> What do you think?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Timo
>>>>>>
>>>>>> [1] https://en.wikipedia.org/wiki/Opaque_data_type
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Xuefu Zhang
>>>>>
>>>>> "In Honey We Trust!"
>>>>
>>

[Kurt Young]

+1 to RAW, if there's no better candidate comes up.

Best,
Kurt


On Mon, Oct 21, 2019 at 9:25 PM Timo Walther <[hidden email]> wrote:

> I would also avoid `UNKNOWN` because of the mentioned reasons.
>
> I'm fine with `RAW`. I will wait another day or two until I conclude the
> discussion.
>
> Thanks,
> Timo
>
>
> On 21.10.19 12:23, Jark Wu wrote:
> > I also think `UNKNOWN` is not suitable here.
> > Because we already have `UNKNOWN` value in SQL, i.e. the three-valued
> logic
> > (TRUE, FALSE, UNKNOWN) of BOOLEAN type.
> > It will confuse users here, what's the relationship between them.
> >
> > Best,
> > Jark
> >
> > On Mon, 21 Oct 2019 at 17:53, Paul Lam <[hidden email]> wrote:
> >
> >> Hi,
> >>
> >> IMHO, `UNKNOWN` does not fully reflects the situation here, because the
> >> types are
> >> actually “known” to users, and users just want to leave them out of
> Flink
> >> type system.
> >>
> >> +1 for `RAW`, for it's more intuitive than `OPAQUE`.
> >>
> >> Best,
> >> Paul Lam
> >>
> >>> 在 2019年10月21日，16:43，Kurt Young <[hidden email]> 写道：
> >>>
> >>> OPAQUE seems to be a little bit advanced to a lot non-english
> >>> speakers (including me). I think Xuefu raised a good alternative:
> >>> UNKNOWN. What do you think about it?
> >>>
> >>> Best,
> >>> Kurt
> >>>
> >>>
> >>> On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek <[hidden email]>
> >>> wrote:
> >>>
> >>>> I prefer OPAQUE compared to ANY because any is often the root object
> in
> >> an
> >>>> object hierarchy and would indicate to users the wrong thing.
> >>>>
> >>>> Aljoscha
> >>>>
> >>>>> On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
> >>>>>
> >>>>> Thanks to Timo for bringing up an interesting topic.
> >>>>>
> >>>>> Personally, "OPAQUE" doesn't seem very intuitive with respect to
> types.
> >>>> (It
> >>>>> suits pretty well to glasses, thought. :)) Anyway, could we just use
> >>>>> "UNKNOWN", which is more explicit and true reflects its nature?
> >>>>>
> >>>>> Thanks,
> >>>>> Xuefu
> >>>>>
> >>>>>
> >>>>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]>
> >> wrote:
> >>>>>> Hi everyone,
> >>>>>>
> >>>>>> Stephan pointed out that our naming of a generic/blackbox/opaque
> type
> >> in
> >>>>>> SQL might be not intuitive for users. As the term ANY rather
> >> describes a
> >>>>>> "super-class of all types" which is not the case in our type system.
> >> Our
> >>>>>> current ANY type stands for a type that is just a blackbox within
> SQL,
> >>>>>> serialized by some custom serializer, that can only be modified
> within
> >>>>>> UDFs.
> >>>>>>
> >>>>>> I also gathered feedback from a training instructor and native
> English
> >>>>>> speaker (David in CC) where I received the following:
> >>>>>>
> >>>>>> "The way I’m thinking about this is this: there’s a concept here
> that
> >>>>>> people have to become aware of, which is that Flink SQL is able to
> >>>>>> operate generically on opaquely typed things — and folks need to be
> >> able
> >>>>>> to connect what they see in code examples, etc. with this concept
> >> (which
> >>>>>> they may be unaware of initially).
> >>>>>> I feel like ANY misses the mark a little bit, but isn’t particularly
> >>>>>> bad. I do worry that it may cause some confusion about its purpose
> and
> >>>>>> power. I think OPAQUE would more clearly express what’s going on."
> >>>>>>
> >>>>>> Also resources like Wikipedia [1] show that this terminology is
> >> common:
> >>>>>> "a data type whose concrete data structure is not defined [...] its
> >>>>>> values can only be manipulated by calling subroutines that have
> access
> >>>>>> to the missing information"
> >>>>>>
> >>>>>> I would therefore vote for refactoring the type name because it is
> not
> >>>>>> used much yet.
> >>>>>>
> >>>>>> Implications are:
> >>>>>>
> >>>>>> - a new parser keyword "OPAQUE" and changed SQL parser
> >>>>>>
> >>>>>> - changes for logical type root, logical type visitors, and their
> >> usages
> >>>>>> What do you think?
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Timo
> >>>>>>
> >>>>>> [1] https://en.wikipedia.org/wiki/Opaque_data_type
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> --
> >>>>> Xuefu Zhang
> >>>>>
> >>>>> "In Honey We Trust!"
> >>>>
> >>
>
>

[Terry Wang]

“OPAQUE” seems a little strange to me.
+ 1 for ‘RAW’.

Best,
Terry Wang

> 2019年10月22日 09:19，Kurt Young <[hidden email]> 写道：
>
> +1 to RAW, if there's no better candidate comes up.
>
> Best,
> Kurt
>
>
> On Mon, Oct 21, 2019 at 9:25 PM Timo Walther <[hidden email]> wrote:
>
>> I would also avoid `UNKNOWN` because of the mentioned reasons.
>>
>> I'm fine with `RAW`. I will wait another day or two until I conclude the
>> discussion.
>>
>> Thanks,
>> Timo
>>
>>
>> On 21.10.19 12:23, Jark Wu wrote:
>>> I also think `UNKNOWN` is not suitable here.
>>> Because we already have `UNKNOWN` value in SQL, i.e. the three-valued
>> logic
>>> (TRUE, FALSE, UNKNOWN) of BOOLEAN type.
>>> It will confuse users here, what's the relationship between them.
>>>
>>> Best,
>>> Jark
>>>
>>> On Mon, 21 Oct 2019 at 17:53, Paul Lam <[hidden email]> wrote:
>>>
>>>> Hi,
>>>>
>>>> IMHO, `UNKNOWN` does not fully reflects the situation here, because the
>>>> types are
>>>> actually “known” to users, and users just want to leave them out of
>> Flink
>>>> type system.
>>>>
>>>> +1 for `RAW`, for it's more intuitive than `OPAQUE`.
>>>>
>>>> Best,
>>>> Paul Lam
>>>>
>>>>> 在 2019年10月21日，16:43，Kurt Young <[hidden email]> 写道：
>>>>>
>>>>> OPAQUE seems to be a little bit advanced to a lot non-english
>>>>> speakers (including me). I think Xuefu raised a good alternative:
>>>>> UNKNOWN. What do you think about it?
>>>>>
>>>>> Best,
>>>>> Kurt
>>>>>
>>>>>
>>>>> On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> I prefer OPAQUE compared to ANY because any is often the root object
>> in
>>>> an
>>>>>> object hierarchy and would indicate to users the wrong thing.
>>>>>>
>>>>>> Aljoscha
>>>>>>
>>>>>>> On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
>>>>>>>
>>>>>>> Thanks to Timo for bringing up an interesting topic.
>>>>>>>
>>>>>>> Personally, "OPAQUE" doesn't seem very intuitive with respect to
>> types.
>>>>>> (It
>>>>>>> suits pretty well to glasses, thought. :)) Anyway, could we just use
>>>>>>> "UNKNOWN", which is more explicit and true reflects its nature?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Xuefu
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]>
>>>> wrote:
>>>>>>>> Hi everyone,
>>>>>>>>
>>>>>>>> Stephan pointed out that our naming of a generic/blackbox/opaque
>> type
>>>> in
>>>>>>>> SQL might be not intuitive for users. As the term ANY rather
>>>> describes a
>>>>>>>> "super-class of all types" which is not the case in our type system.
>>>> Our
>>>>>>>> current ANY type stands for a type that is just a blackbox within
>> SQL,
>>>>>>>> serialized by some custom serializer, that can only be modified
>> within
>>>>>>>> UDFs.
>>>>>>>>
>>>>>>>> I also gathered feedback from a training instructor and native
>> English
>>>>>>>> speaker (David in CC) where I received the following:
>>>>>>>>
>>>>>>>> "The way I’m thinking about this is this: there’s a concept here
>> that
>>>>>>>> people have to become aware of, which is that Flink SQL is able to
>>>>>>>> operate generically on opaquely typed things — and folks need to be
>>>> able
>>>>>>>> to connect what they see in code examples, etc. with this concept
>>>> (which
>>>>>>>> they may be unaware of initially).
>>>>>>>> I feel like ANY misses the mark a little bit, but isn’t particularly
>>>>>>>> bad. I do worry that it may cause some confusion about its purpose
>> and
>>>>>>>> power. I think OPAQUE would more clearly express what’s going on."
>>>>>>>>
>>>>>>>> Also resources like Wikipedia [1] show that this terminology is
>>>> common:
>>>>>>>> "a data type whose concrete data structure is not defined [...] its
>>>>>>>> values can only be manipulated by calling subroutines that have
>> access
>>>>>>>> to the missing information"
>>>>>>>>
>>>>>>>> I would therefore vote for refactoring the type name because it is
>> not
>>>>>>>> used much yet.
>>>>>>>>
>>>>>>>> Implications are:
>>>>>>>>
>>>>>>>> - a new parser keyword "OPAQUE" and changed SQL parser
>>>>>>>>
>>>>>>>> - changes for logical type root, logical type visitors, and their
>>>> usages
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Timo
>>>>>>>>
>>>>>>>> [1] https://en.wikipedia.org/wiki/Opaque_data_type
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Xuefu Zhang
>>>>>>>
>>>>>>> "In Honey We Trust!"
>>>>>>
>>>>
>>
>>

[David Anderson-2]

+1 for RAW.

I agree that this is clearer than OPAQUE (which I initially proposed).

On Mon, Oct 21, 2019 at 10:33 AM Jark Wu <[hidden email]> wrote:

>
> +1 to rename ANY.
>
> I don't have strong opinion on the new name. I think "OPAQUE" is fine, because it is introduced in IBM Informix and Oracle.
> In Informix, it says [1]:
>
> "An opaque data type is fully encapsulated; the database server does not know about the internal format of an opaque data type.
> Therefore, the database server cannot make assumptions about how to access a column having an opaque data type.
> The database developer defines a data structure that holds the opaque-type information and support functions
> that tell the database server how to access this data structure."
>
> So, I think "opaque" is fine here.
>
> Another option is "RAW" which is introduced by Oracle [2] represents data that is not to be interpreted by system .
>
> Best,
> Jark
>
> [1]: https://www.ibm.com/support/knowledgecenter/en/SSGU8G_11.70.0/com.ibm.esqlc.doc/ids_esqlc_0357.htm
> [2]: https://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT613
>
> On Mon, 21 Oct 2019 at 15:49, Aljoscha Krettek <[hidden email]> wrote:
>>
>> I prefer OPAQUE compared to ANY because any is often the root object in an object hierarchy and would indicate to users the wrong thing.
>>
>> Aljoscha
>>
>> > On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
>> >
>> > Thanks to Timo for bringing up an interesting topic.
>> >
>> > Personally, "OPAQUE" doesn't seem very intuitive with respect to types. (It
>> > suits pretty well to glasses, thought. :)) Anyway, could we just use
>> > "UNKNOWN", which is more explicit and true reflects its nature?
>> >
>> > Thanks,
>> > Xuefu
>> >
>> >
>> > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]> wrote:
>> >
>> >> Hi everyone,
>> >>
>> >> Stephan pointed out that our naming of a generic/blackbox/opaque type in
>> >> SQL might be not intuitive for users. As the term ANY rather describes a
>> >> "super-class of all types" which is not the case in our type system. Our
>> >> current ANY type stands for a type that is just a blackbox within SQL,
>> >> serialized by some custom serializer, that can only be modified within
>> >> UDFs.
>> >>
>> >> I also gathered feedback from a training instructor and native English
>> >> speaker (David in CC) where I received the following:
>> >>
>> >> "The way I’m thinking about this is this: there’s a concept here that
>> >> people have to become aware of, which is that Flink SQL is able to
>> >> operate generically on opaquely typed things — and folks need to be able
>> >> to connect what they see in code examples, etc. with this concept (which
>> >> they may be unaware of initially).
>> >> I feel like ANY misses the mark a little bit, but isn’t particularly
>> >> bad. I do worry that it may cause some confusion about its purpose and
>> >> power. I think OPAQUE would more clearly express what’s going on."
>> >>
>> >> Also resources like Wikipedia [1] show that this terminology is common:
>> >>
>> >> "a data type whose concrete data structure is not defined [...] its
>> >> values can only be manipulated by calling subroutines that have access
>> >> to the missing information"
>> >>
>> >> I would therefore vote for refactoring the type name because it is not
>> >> used much yet.
>> >>
>> >> Implications are:
>> >>
>> >> - a new parser keyword "OPAQUE" and changed SQL parser
>> >>
>> >> - changes for logical type root, logical type visitors, and their usages
>> >>
>> >> What do you think?
>> >>
>> >> Thanks,
>> >>
>> >> Timo
>> >>
>> >> [1] https://en.wikipedia.org/wiki/Opaque_data_type
>> >>
>> >>
>> >>
>> >
>> > --
>> > Xuefu Zhang
>> >
>> > "In Honey We Trust!"
>>

[DONG, Weike]

+1 for RAW type, which is easy to guess its actual meaning, even for a new
user.

Best,
Weike


On Tue, Oct 22, 2019 at 5:31 PM David Anderson <[hidden email]> wrote:

> +1 for RAW.
>
> I agree that this is clearer than OPAQUE (which I initially proposed).
>
> On Mon, Oct 21, 2019 at 10:33 AM Jark Wu <[hidden email]> wrote:
> >
> > +1 to rename ANY.
> >
> > I don't have strong opinion on the new name. I think "OPAQUE" is fine,
> because it is introduced in IBM Informix and Oracle.
> > In Informix, it says [1]:
> >
> > "An opaque data type is fully encapsulated; the database server does not
> know about the internal format of an opaque data type.
> > Therefore, the database server cannot make assumptions about how to
> access a column having an opaque data type.
> > The database developer defines a data structure that holds the
> opaque-type information and support functions
> > that tell the database server how to access this data structure."
> >
> > So, I think "opaque" is fine here.
> >
> > Another option is "RAW" which is introduced by Oracle [2] represents
> data that is not to be interpreted by system .
> >
> > Best,
> > Jark
> >
> > [1]:
> https://www.ibm.com/support/knowledgecenter/en/SSGU8G_11.70.0/com.ibm.esqlc.doc/ids_esqlc_0357.htm
> > [2]:
> https://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT613
> >
> > On Mon, 21 Oct 2019 at 15:49, Aljoscha Krettek <[hidden email]>
> wrote:
> >>
> >> I prefer OPAQUE compared to ANY because any is often the root object in
> an object hierarchy and would indicate to users the wrong thing.
> >>
> >> Aljoscha
> >>
> >> > On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
> >> >
> >> > Thanks to Timo for bringing up an interesting topic.
> >> >
> >> > Personally, "OPAQUE" doesn't seem very intuitive with respect to
> types. (It
> >> > suits pretty well to glasses, thought. :)) Anyway, could we just use
> >> > "UNKNOWN", which is more explicit and true reflects its nature?
> >> >
> >> > Thanks,
> >> > Xuefu
> >> >
> >> >
> >> > On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]>
> wrote:
> >> >
> >> >> Hi everyone,
> >> >>
> >> >> Stephan pointed out that our naming of a generic/blackbox/opaque
> type in
> >> >> SQL might be not intuitive for users. As the term ANY rather
> describes a
> >> >> "super-class of all types" which is not the case in our type system.
> Our
> >> >> current ANY type stands for a type that is just a blackbox within
> SQL,
> >> >> serialized by some custom serializer, that can only be modified
> within
> >> >> UDFs.
> >> >>
> >> >> I also gathered feedback from a training instructor and native
> English
> >> >> speaker (David in CC) where I received the following:
> >> >>
> >> >> "The way I’m thinking about this is this: there’s a concept here that
> >> >> people have to become aware of, which is that Flink SQL is able to
> >> >> operate generically on opaquely typed things — and folks need to be
> able
> >> >> to connect what they see in code examples, etc. with this concept
> (which
> >> >> they may be unaware of initially).
> >> >> I feel like ANY misses the mark a little bit, but isn’t particularly
> >> >> bad. I do worry that it may cause some confusion about its purpose
> and
> >> >> power. I think OPAQUE would more clearly express what’s going on."
> >> >>
> >> >> Also resources like Wikipedia [1] show that this terminology is
> common:
> >> >>
> >> >> "a data type whose concrete data structure is not defined [...] its
> >> >> values can only be manipulated by calling subroutines that have
> access
> >> >> to the missing information"
> >> >>
> >> >> I would therefore vote for refactoring the type name because it is
> not
> >> >> used much yet.
> >> >>
> >> >> Implications are:
> >> >>
> >> >> - a new parser keyword "OPAQUE" and changed SQL parser
> >> >>
> >> >> - changes for logical type root, logical type visitors, and their
> usages
> >> >>
> >> >> What do you think?
> >> >>
> >> >> Thanks,
> >> >>
> >> >> Timo
> >> >>
> >> >> [1] https://en.wikipedia.org/wiki/Opaque_data_type
> >> >>
> >> >>
> >> >>
> >> >
> >> > --
> >> > Xuefu Zhang
> >> >
> >> > "In Honey We Trust!"
> >>
>

[Timo Walther-2]

Thanks everyone for the feedback!

We have a clear favorite which is the RAW type. I will make sure that
this change is applied to the Flink code base.

Thanks again,
Timo


On 22.10.19 04:07, Terry Wang wrote:

> “OPAQUE” seems a little strange to me.
> + 1 for ‘RAW’.
>
> Best,
> Terry Wang
>
>
>
>> 2019年10月22日 09:19，Kurt Young <[hidden email]> 写道：
>>
>> +1 to RAW, if there's no better candidate comes up.
>>
>> Best,
>> Kurt
>>
>>
>> On Mon, Oct 21, 2019 at 9:25 PM Timo Walther <[hidden email]> wrote:
>>
>>> I would also avoid `UNKNOWN` because of the mentioned reasons.
>>>
>>> I'm fine with `RAW`. I will wait another day or two until I conclude the
>>> discussion.
>>>
>>> Thanks,
>>> Timo
>>>
>>>
>>> On 21.10.19 12:23, Jark Wu wrote:
>>>> I also think `UNKNOWN` is not suitable here.
>>>> Because we already have `UNKNOWN` value in SQL, i.e. the three-valued
>>> logic
>>>> (TRUE, FALSE, UNKNOWN) of BOOLEAN type.
>>>> It will confuse users here, what's the relationship between them.
>>>>
>>>> Best,
>>>> Jark
>>>>
>>>> On Mon, 21 Oct 2019 at 17:53, Paul Lam <[hidden email]> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> IMHO, `UNKNOWN` does not fully reflects the situation here, because the
>>>>> types are
>>>>> actually “known” to users, and users just want to leave them out of
>>> Flink
>>>>> type system.
>>>>>
>>>>> +1 for `RAW`, for it's more intuitive than `OPAQUE`.
>>>>>
>>>>> Best,
>>>>> Paul Lam
>>>>>
>>>>>> 在 2019年10月21日，16:43，Kurt Young <[hidden email]> 写道：
>>>>>>
>>>>>> OPAQUE seems to be a little bit advanced to a lot non-english
>>>>>> speakers (including me). I think Xuefu raised a good alternative:
>>>>>> UNKNOWN. What do you think about it?
>>>>>>
>>>>>> Best,
>>>>>> Kurt
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 21, 2019 at 3:49 PM Aljoscha Krettek <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> I prefer OPAQUE compared to ANY because any is often the root object
>>> in
>>>>> an
>>>>>>> object hierarchy and would indicate to users the wrong thing.
>>>>>>>
>>>>>>> Aljoscha
>>>>>>>
>>>>>>>> On 18. Oct 2019, at 18:41, Xuefu Z <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> Thanks to Timo for bringing up an interesting topic.
>>>>>>>>
>>>>>>>> Personally, "OPAQUE" doesn't seem very intuitive with respect to
>>> types.
>>>>>>> (It
>>>>>>>> suits pretty well to glasses, thought. :)) Anyway, could we just use
>>>>>>>> "UNKNOWN", which is more explicit and true reflects its nature?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Xuefu
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Oct 18, 2019 at 7:51 AM Timo Walther <[hidden email]>
>>>>> wrote:
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>> Stephan pointed out that our naming of a generic/blackbox/opaque
>>> type
>>>>> in
>>>>>>>>> SQL might be not intuitive for users. As the term ANY rather
>>>>> describes a
>>>>>>>>> "super-class of all types" which is not the case in our type system.
>>>>> Our
>>>>>>>>> current ANY type stands for a type that is just a blackbox within
>>> SQL,
>>>>>>>>> serialized by some custom serializer, that can only be modified
>>> within
>>>>>>>>> UDFs.
>>>>>>>>>
>>>>>>>>> I also gathered feedback from a training instructor and native
>>> English
>>>>>>>>> speaker (David in CC) where I received the following:
>>>>>>>>>
>>>>>>>>> "The way I’m thinking about this is this: there’s a concept here
>>> that
>>>>>>>>> people have to become aware of, which is that Flink SQL is able to
>>>>>>>>> operate generically on opaquely typed things — and folks need to be
>>>>> able
>>>>>>>>> to connect what they see in code examples, etc. with this concept
>>>>> (which
>>>>>>>>> they may be unaware of initially).
>>>>>>>>> I feel like ANY misses the mark a little bit, but isn’t particularly
>>>>>>>>> bad. I do worry that it may cause some confusion about its purpose
>>> and
>>>>>>>>> power. I think OPAQUE would more clearly express what’s going on."
>>>>>>>>>
>>>>>>>>> Also resources like Wikipedia [1] show that this terminology is
>>>>> common:
>>>>>>>>> "a data type whose concrete data structure is not defined [...] its
>>>>>>>>> values can only be manipulated by calling subroutines that have
>>> access
>>>>>>>>> to the missing information"
>>>>>>>>>
>>>>>>>>> I would therefore vote for refactoring the type name because it is
>>> not
>>>>>>>>> used much yet.
>>>>>>>>>
>>>>>>>>> Implications are:
>>>>>>>>>
>>>>>>>>> - a new parser keyword "OPAQUE" and changed SQL parser
>>>>>>>>>
>>>>>>>>> - changes for logical type root, logical type visitors, and their
>>>>> usages
>>>>>>>>> What do you think?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Timo
>>>>>>>>>
>>>>>>>>> [1] https://en.wikipedia.org/wiki/Opaque_data_type
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Xuefu Zhang
>>>>>>>>
>>>>>>>> "In Honey We Trust!"
>>>>>>>
>>>>>
>>>
>>>