(DEPRECATED) Apache Flink Mailing List archive.

blink planner issue

Classic

List

Threaded

10 messages Options

kaka chen

blink planner issue

Hi All,

We are trying to switch to blink table planner in HBase connector, which
found the following error:

Running org.apache.flink.addons.hbase.HBaseSinkITCase

Formatting using clusterid: testClusterID

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 28.042 sec
<<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase

testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase) Time elapsed:
2.431 sec <<< ERROR!

org.apache.flink.table.api.TableException: Result field 'family1' does not
match requested type. Requested: Row(col1: Integer); Actual: Row(EXPR$0:
Integer)

at
org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)

The original flink table planner executed successfully.

Thanks,

Kaka Chen

kaka chen

Re: blink planner issue

I am looking this issue, the related codes which throw errors are:

SinkCodeGenerator:: validateFieldType()

...

// Tuple/Case class/Row type requested
case tt: TupleTypeInfoBase[_] =>
fieldTypes.zipWithIndex foreach {
case (fieldTypeInfo: GenericTypeInfo[_], i) =>
val requestedTypeInfo = tt.getTypeAt(i)
if (!requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]]) {
throw new TableException(
s"Result field '${fieldNames(i)}' does not match requested
type. " +
s"Requested: $requestedTypeInfo; Actual:
$fieldTypeInfo")
}
case (fieldTypeInfo, i) =>
val requestedTypeInfo = tt.getTypeAt(i)
validateFieldType(requestedTypeInfo)
* if (fromTypeInfoToLogicalType(fieldTypeInfo) !=*
* fromTypeInfoToLogicalType(requestedTypeInfo) &&*
* !requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]])
{*
* val fieldNames = tt.getFieldNames*
* throw new TableException(s"Result field '${fieldNames(i)}'
does not match requested" +*
* s" type. Requested: $requestedTypeInfo; Actual:
$fieldTypeInfo")*
* }*
}
...

Thanks,
Frank

kaka chen <[hidden email]> 于2019年7月16日周二下午5:23写道：

> Hi All,
>
>
> We are trying to switch to blink table planner in HBase connector, which
> found the following error:
>
>
> Running org.apache.flink.addons.hbase.HBaseSinkITCase
>
> Formatting using clusterid: testClusterID
>
> Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 28.042 sec
> <<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase
>
> testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase) Time
> elapsed: 2.431 sec <<< ERROR!
>
> org.apache.flink.table.api.TableException: Result field 'family1' does not
> match requested type. Requested: Row(col1: Integer); Actual: Row(EXPR$0:
> Integer)
>
> at
> org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)
>
>
> The original flink table planner executed successfully.
>
>
> Thanks,
>
> Kaka Chen
>

Jark Wu-2

Re: blink planner issue

Hi Kaka,

Thanks for reporting this. We didn't cover integrate tests for connectors
yet because of FLINK-13276. We will cover that after FLINK-13276 is fixed.

The problem you raised might because we misused `LogicalType.equals` which
will checks field names as well.
I have created an issue (FLINK-13290) to track this problem.

Best,
Jark

On Tue, 16 Jul 2019 at 17:35, kaka chen <[hidden email]> wrote:

> I am looking this issue, the related codes which throw errors are:
>
> SinkCodeGenerator:: validateFieldType()
>
> ...
>
> // Tuple/Case class/Row type requested
> case tt: TupleTypeInfoBase[_] =>
> fieldTypes.zipWithIndex foreach {
> case (fieldTypeInfo: GenericTypeInfo[_], i) =>
> val requestedTypeInfo = tt.getTypeAt(i)
> if (!requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]]) {
> throw new TableException(
> s"Result field '${fieldNames(i)}' does not match requested
> type. " +
> s"Requested: $requestedTypeInfo; Actual:
> $fieldTypeInfo")
> }
> case (fieldTypeInfo, i) =>
> val requestedTypeInfo = tt.getTypeAt(i)
> validateFieldType(requestedTypeInfo)
> * if (fromTypeInfoToLogicalType(fieldTypeInfo) !=*
> * fromTypeInfoToLogicalType(requestedTypeInfo) &&*
> * !requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]])
> {*
> * val fieldNames = tt.getFieldNames*
> * throw new TableException(s"Result field '${fieldNames(i)}'
> does not match requested" +*
> * s" type. Requested: $requestedTypeInfo; Actual:
> $fieldTypeInfo")*
> * }*
> }
> ...
>
> Thanks,
> Frank
>
> kaka chen <[hidden email]> 于2019年7月16日周二下午5:23写道：
>
> > Hi All,
> >
> >
> > We are trying to switch to blink table planner in HBase connector, which
> > found the following error:
> >
> >
> > Running org.apache.flink.addons.hbase.HBaseSinkITCase
> >
> > Formatting using clusterid: testClusterID
> >
> > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 28.042
> sec
> > <<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase
> >
> > testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase) Time
> > elapsed: 2.431 sec <<< ERROR!
> >
> > org.apache.flink.table.api.TableException: Result field 'family1' does
> not
> > match requested type. Requested: Row(col1: Integer); Actual: Row(EXPR$0:
> > Integer)
> >
> > at
> >
> org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)
> >
> >
> > The original flink table planner executed successfully.
> >
> >
> > Thanks,
> >
> > Kaka Chen
> >
>

kaka chen

Re: blink planner issue

Hi Jark,

Thanks!

Thanks,
Kaka Chen

Jark Wu <[hidden email]> 于2019年7月16日周二下午10:30写道：

> Hi Kaka,
>
> Thanks for reporting this. We didn't cover integrate tests for connectors
> yet because of FLINK-13276. We will cover that after FLINK-13276 is fixed.
>
> The problem you raised might because we misused `LogicalType.equals` which
> will checks field names as well.
> I have created an issue (FLINK-13290) to track this problem.
>
> Best,
> Jark
>
>
> On Tue, 16 Jul 2019 at 17:35, kaka chen <[hidden email]> wrote:
>
> > I am looking this issue, the related codes which throw errors are:
> >
> > SinkCodeGenerator:: validateFieldType()
> >
> > ...
> >
> > // Tuple/Case class/Row type requested
> > case tt: TupleTypeInfoBase[_] =>
> > fieldTypes.zipWithIndex foreach {
> > case (fieldTypeInfo: GenericTypeInfo[_], i) =>
> > val requestedTypeInfo = tt.getTypeAt(i)
> > if
> (!requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]]) {
> > throw new TableException(
> > s"Result field '${fieldNames(i)}' does not match
> requested
> > type. " +
> > s"Requested: $requestedTypeInfo; Actual:
> > $fieldTypeInfo")
> > }
> > case (fieldTypeInfo, i) =>
> > val requestedTypeInfo = tt.getTypeAt(i)
> > validateFieldType(requestedTypeInfo)
> > * if (fromTypeInfoToLogicalType(fieldTypeInfo) !=*
> > * fromTypeInfoToLogicalType(requestedTypeInfo) &&*
> > *
> !requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]])
> > {*
> > * val fieldNames = tt.getFieldNames*
> > * throw new TableException(s"Result field '${fieldNames(i)}'
> > does not match requested" +*
> > * s" type. Requested: $requestedTypeInfo; Actual:
> > $fieldTypeInfo")*
> > * }*
> > }
> > ...
> >
> > Thanks,
> > Frank
> >
> > kaka chen <[hidden email]> 于2019年7月16日周二下午5:23写道：
> >
> > > Hi All,
> > >
> > >
> > > We are trying to switch to blink table planner in HBase connector,
> which
> > > found the following error:
> > >
> > >
> > > Running org.apache.flink.addons.hbase.HBaseSinkITCase
> > >
> > > Formatting using clusterid: testClusterID
> > >
> > > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 28.042
> > sec
> > > <<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase
> > >
> > > testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase) Time
> > > elapsed: 2.431 sec <<< ERROR!
> > >
> > > org.apache.flink.table.api.TableException: Result field 'family1' does
> > not
> > > match requested type. Requested: Row(col1: Integer); Actual:
> Row(EXPR$0:
> > > Integer)
> > >
> > > at
> > >
> >
> org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)
> > >
> > >
> > > The original flink table planner executed successfully.
> > >
> > >
> > > Thanks,
> > >
> > > Kaka Chen
> > >
> >
>

Caizhi Weng

Re: blink planner issue

Hi Kaka and Jark,

On a side note, `RowTypeInfo` only compares field types in its `equals`
method. I think our new logical row type shouldn't break this behavior.

kaka chen <[hidden email]> 于2019年7月16日周二下午10:53写道：

> Hi Jark,
>
> Thanks!
>
> Thanks,
> Kaka Chen
>
> Jark Wu <[hidden email]> 于2019年7月16日周二下午10:30写道：
>
> > Hi Kaka,
> >
> > Thanks for reporting this. We didn't cover integrate tests for connectors
> > yet because of FLINK-13276. We will cover that after FLINK-13276 is
> fixed.
> >
> > The problem you raised might because we misused `LogicalType.equals`
> which
> > will checks field names as well.
> > I have created an issue (FLINK-13290) to track this problem.
> >
> > Best,
> > Jark
> >
> >
> > On Tue, 16 Jul 2019 at 17:35, kaka chen <[hidden email]> wrote:
> >
> > > I am looking this issue, the related codes which throw errors are:
> > >
> > > SinkCodeGenerator:: validateFieldType()
> > >
> > > ...
> > >
> > > // Tuple/Case class/Row type requested
> > > case tt: TupleTypeInfoBase[_] =>
> > > fieldTypes.zipWithIndex foreach {
> > > case (fieldTypeInfo: GenericTypeInfo[_], i) =>
> > > val requestedTypeInfo = tt.getTypeAt(i)
> > > if
> > (!requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]]) {
> > > throw new TableException(
> > > s"Result field '${fieldNames(i)}' does not match
> > requested
> > > type. " +
> > > s"Requested: $requestedTypeInfo; Actual:
> > > $fieldTypeInfo")
> > > }
> > > case (fieldTypeInfo, i) =>
> > > val requestedTypeInfo = tt.getTypeAt(i)
> > > validateFieldType(requestedTypeInfo)
> > > * if (fromTypeInfoToLogicalType(fieldTypeInfo) !=*
> > > * fromTypeInfoToLogicalType(requestedTypeInfo) &&*
> > > *
> > !requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]])
> > > {*
> > > * val fieldNames = tt.getFieldNames*
> > > * throw new TableException(s"Result field
> '${fieldNames(i)}'
> > > does not match requested" +*
> > > * s" type. Requested: $requestedTypeInfo; Actual:
> > > $fieldTypeInfo")*
> > > * }*
> > > }
> > > ...
> > >
> > > Thanks,
> > > Frank
> > >
> > > kaka chen <[hidden email]> 于2019年7月16日周二下午5:23写道：
> > >
> > > > Hi All,
> > > >
> > > >
> > > > We are trying to switch to blink table planner in HBase connector,
> > which
> > > > found the following error:
> > > >
> > > >
> > > > Running org.apache.flink.addons.hbase.HBaseSinkITCase
> > > >
> > > > Formatting using clusterid: testClusterID
> > > >
> > > > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 28.042
> > > sec
> > > > <<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase
> > > >
> > > > testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase) Time
> > > > elapsed: 2.431 sec <<< ERROR!
> > > >
> > > > org.apache.flink.table.api.TableException: Result field 'family1'
> does
> > > not
> > > > match requested type. Requested: Row(col1: Integer); Actual:
> > Row(EXPR$0:
> > > > Integer)
> > > >
> > > > at
> > > >
> > >
> >
> org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)
> > > >
> > > >
> > > > The original flink table planner executed successfully.
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Kaka Chen
> > > >
> > >
> >
>

kaka chen

Re: blink planner issue

Hi Caizhi and Mark,

I think you are correct, from the quick view for source code, it should
only compares field types in the equals method.
Currently some composite logical row type has compared name and
description, such as RowType and StructuredType.

Thanks,
Kaka Chen

Caizhi Weng <[hidden email]> 于2019年7月16日周二下午11:16写道：

> Hi Kaka and Jark,
>
> On a side note, `RowTypeInfo` only compares field types in its `equals`
> method. I think our new logical row type shouldn't break this behavior.
>
> kaka chen <[hidden email]> 于2019年7月16日周二下午10:53写道：
>
> > Hi Jark,
> >
> > Thanks!
> >
> > Thanks,
> > Kaka Chen
> >
> > Jark Wu <[hidden email]> 于2019年7月16日周二下午10:30写道：
> >
> > > Hi Kaka,
> > >
> > > Thanks for reporting this. We didn't cover integrate tests for
> connectors
> > > yet because of FLINK-13276. We will cover that after FLINK-13276 is
> > fixed.
> > >
> > > The problem you raised might because we misused `LogicalType.equals`
> > which
> > > will checks field names as well.
> > > I have created an issue (FLINK-13290) to track this problem.
> > >
> > > Best,
> > > Jark
> > >
> > >
> > > On Tue, 16 Jul 2019 at 17:35, kaka chen <[hidden email]> wrote:
> > >
> > > > I am looking this issue, the related codes which throw errors
> are:
> > > >
> > > > SinkCodeGenerator:: validateFieldType()
> > > >
> > > > ...
> > > >
> > > > // Tuple/Case class/Row type requested
> > > > case tt: TupleTypeInfoBase[_] =>
> > > > fieldTypes.zipWithIndex foreach {
> > > > case (fieldTypeInfo: GenericTypeInfo[_], i) =>
> > > > val requestedTypeInfo = tt.getTypeAt(i)
> > > > if
> > > (!requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]]) {
> > > > throw new TableException(
> > > > s"Result field '${fieldNames(i)}' does not match
> > > requested
> > > > type. " +
> > > > s"Requested: $requestedTypeInfo; Actual:
> > > > $fieldTypeInfo")
> > > > }
> > > > case (fieldTypeInfo, i) =>
> > > > val requestedTypeInfo = tt.getTypeAt(i)
> > > > validateFieldType(requestedTypeInfo)
> > > > * if (fromTypeInfoToLogicalType(fieldTypeInfo) !=*
> > > > * fromTypeInfoToLogicalType(requestedTypeInfo) &&*
> > > > *
> > > !requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]])
> > > > {*
> > > > * val fieldNames = tt.getFieldNames*
> > > > * throw new TableException(s"Result field
> > '${fieldNames(i)}'
> > > > does not match requested" +*
> > > > * s" type. Requested: $requestedTypeInfo; Actual:
> > > > $fieldTypeInfo")*
> > > > * }*
> > > > }
> > > > ...
> > > >
> > > > Thanks,
> > > > Frank
> > > >
> > > > kaka chen <[hidden email]> 于2019年7月16日周二下午5:23写道：
> > > >
> > > > > Hi All,
> > > > >
> > > > >
> > > > > We are trying to switch to blink table planner in HBase connector,
> > > which
> > > > > found the following error:
> > > > >
> > > > >
> > > > > Running org.apache.flink.addons.hbase.HBaseSinkITCase
> > > > >
> > > > > Formatting using clusterid: testClusterID
> > > > >
> > > > > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > 28.042
> > > > sec
> > > > > <<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase
> > > > >
> > > > > testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase) Time
> > > > > elapsed: 2.431 sec <<< ERROR!
> > > > >
> > > > > org.apache.flink.table.api.TableException: Result field 'family1'
> > does
> > > > not
> > > > > match requested type. Requested: Row(col1: Integer); Actual:
> > > Row(EXPR$0:
> > > > > Integer)
> > > > >
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)
> > > > >
> > > > >
> > > > > The original flink table planner executed successfully.
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Kaka Chen
> > > > >
> > > >
> > >
> >
>

kaka chen

Re: blink planner issue

In reply to this post by Caizhi Weng

Hi Caizhi and Jark,

I think you are correct, from the quick view for source code, it should
only compares field types in the equals method.
Currently some composite logical row type has compared name and
description, such as RowType and StructuredType.

Thanks,
Kaka Chen

Caizhi Weng <[hidden email]> 于2019年7月16日周二下午11:16写道：

Jark Wu-2

Re: blink planner issue

Hi Caizhi and Kaka,

Actually, equals-with-field-names and equals-without-field-names are both
needed in Flink SQL.
It's not correct to ignore field names comparison simply in RowType#equals.
We have encountered this problems before because RowTypeInfo doesn't
compare field names (see FLINK-12848).

Best,
Jark

On Tue, 16 Jul 2019 at 23:38, kaka chen <[hidden email]> wrote:

> Hi Caizhi and Jark,
>
> I think you are correct, from the quick view for source code, it should
> only compares field types in the equals method.
> Currently some composite logical row type has compared name and
> description, such as RowType and StructuredType.
>
> Thanks,
> Kaka Chen
>
> Caizhi Weng <[hidden email]> 于2019年7月16日周二下午11:16写道：
>
> > Hi Kaka and Jark,
> >
> > On a side note, `RowTypeInfo` only compares field types in its `equals`
> > method. I think our new logical row type shouldn't break this behavior.
> >
> > kaka chen <[hidden email]> 于2019年7月16日周二下午10:53写道：
> >
> > > Hi Jark,
> > >
> > > Thanks!
> > >
> > > Thanks,
> > > Kaka Chen
> > >
> > > Jark Wu <[hidden email]> 于2019年7月16日周二下午10:30写道：
> > >
> > > > Hi Kaka,
> > > >
> > > > Thanks for reporting this. We didn't cover integrate tests for
> > connectors
> > > > yet because of FLINK-13276. We will cover that after FLINK-13276 is
> > > fixed.
> > > >
> > > > The problem you raised might because we misused `LogicalType.equals`
> > > which
> > > > will checks field names as well.
> > > > I have created an issue (FLINK-13290) to track this problem.
> > > >
> > > > Best,
> > > > Jark
> > > >
> > > >
> > > > On Tue, 16 Jul 2019 at 17:35, kaka chen <[hidden email]>
> wrote:
> > > >
> > > > > I am looking this issue, the related codes which throw errors
> > are:
> > > > >
> > > > > SinkCodeGenerator:: validateFieldType()
> > > > >
> > > > > ...
> > > > >
> > > > > // Tuple/Case class/Row type requested
> > > > > case tt: TupleTypeInfoBase[_] =>
> > > > > fieldTypes.zipWithIndex foreach {
> > > > > case (fieldTypeInfo: GenericTypeInfo[_], i) =>
> > > > > val requestedTypeInfo = tt.getTypeAt(i)
> > > > > if
> > > > (!requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]]) {
> > > > > throw new TableException(
> > > > > s"Result field '${fieldNames(i)}' does not match
> > > > requested
> > > > > type. " +
> > > > > s"Requested: $requestedTypeInfo; Actual:
> > > > > $fieldTypeInfo")
> > > > > }
> > > > > case (fieldTypeInfo, i) =>
> > > > > val requestedTypeInfo = tt.getTypeAt(i)
> > > > > validateFieldType(requestedTypeInfo)
> > > > > * if (fromTypeInfoToLogicalType(fieldTypeInfo) !=*
> > > > > * fromTypeInfoToLogicalType(requestedTypeInfo) &&*
> > > > > *
> > > > !requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]])
> > > > > {*
> > > > > * val fieldNames = tt.getFieldNames*
> > > > > * throw new TableException(s"Result field
> > > '${fieldNames(i)}'
> > > > > does not match requested" +*
> > > > > * s" type. Requested: $requestedTypeInfo; Actual:
> > > > > $fieldTypeInfo")*
> > > > > * }*
> > > > > }
> > > > > ...
> > > > >
> > > > > Thanks,
> > > > > Frank
> > > > >
> > > > > kaka chen <[hidden email]> 于2019年7月16日周二下午5:23写道：
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > >
> > > > > > We are trying to switch to blink table planner in HBase
> connector,
> > > > which
> > > > > > found the following error:
> > > > > >
> > > > > >
> > > > > > Running org.apache.flink.addons.hbase.HBaseSinkITCase
> > > > > >
> > > > > > Formatting using clusterid: testClusterID
> > > > > >
> > > > > > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > > 28.042
> > > > > sec
> > > > > > <<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase
> > > > > >
> > > > > > testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase)
> Time
> > > > > > elapsed: 2.431 sec <<< ERROR!
> > > > > >
> > > > > > org.apache.flink.table.api.TableException: Result field 'family1'
> > > does
> > > > > not
> > > > > > match requested type. Requested: Row(col1: Integer); Actual:
> > > > Row(EXPR$0:
> > > > > > Integer)
> > > > > >
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)
> > > > > >
> > > > > >
> > > > > > The original flink table planner executed successfully.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Kaka Chen
> > > > > >
> > > > >
> > > >
> > >
> >
>

JingsongLee-2

Re: blink planner issue

In reply to this post by kaka chen

Hi caizhi and kaka and jark:
I think we should keep current equals in RowType and StructuredType.
@Timo WaltherWhat do you think? Do you think we can use asSerializableString?

Best, JingsongLee

------------------------------------------------------------------
From:kaka chen <[hidden email]>
Send Time:2019年7月16日(星期二) 23:45
To:dev <[hidden email]>
Subject:Re: blink planner issue

Hi Caizhi and Jark,

I think you are correct, from the quick view for source code, it should
only compares field types in the equals method.
Currently some composite logical row type has compared name and
description, such as RowType and StructuredType.

Thanks,
Kaka Chen

Caizhi Weng <[hidden email]> 于2019年7月16日周二下午11:16写道：

kaka chen

Re: blink planner issue

Hi Jark, Jingsong, Timo:

I read the FLINK-12848, the root cause of this issue should be 'cache the
RowTypeInfo in seenTypes'. In my opinion, RowType should be compared with
type. BTW, I don't understand why it need cache the composite type. Could
you please explain the following comments, thanks.

// advanced types require specific RelDataType
// for storing the original TypeInformation
seenTypes.getOrElseUpdate((typeInfo, isNullable),
createAdvancedType(typeInfo, isNullable))

Thanks,
Kaka

JingsongLee <[hidden email]> 于2019年7月17日周三上午10:14写道：

> Hi caizhi and kaka and jark:
> I think we should keep current equals in RowType and StructuredType.
> @Timo WaltherWhat do you think? Do you think we can use
> asSerializableString?
>
> Best, JingsongLee
>
>
> ------------------------------------------------------------------
> From:kaka chen <[hidden email]>
> Send Time:2019年7月16日(星期二) 23:45
> To:dev <[hidden email]>
> Subject:Re: blink planner issue
>
> Hi Caizhi and Jark,
>
> I think you are correct, from the quick view for source code, it should
> only compares field types in the equals method.
> Currently some composite logical row type has compared name and
> description, such as RowType and StructuredType.
>
> Thanks,
> Kaka Chen
>
> Caizhi Weng <[hidden email]> 于2019年7月16日周二下午11:16写道：
>
> > Hi Kaka and Jark,
> >
> > On a side note, `RowTypeInfo` only compares field types in its `equals`
> > method. I think our new logical row type shouldn't break this behavior.
> >
> > kaka chen <[hidden email]> 于2019年7月16日周二下午10:53写道：
> >
> > > Hi Jark,
> > >
> > > Thanks!
> > >
> > > Thanks,
> > > Kaka Chen
> > >
> > > Jark Wu <[hidden email]> 于2019年7月16日周二下午10:30写道：
> > >
> > > > Hi Kaka,
> > > >
> > > > Thanks for reporting this. We didn't cover integrate tests for
> > connectors
> > > > yet because of FLINK-13276. We will cover that after FLINK-13276 is
> > > fixed.
> > > >
> > > > The problem you raised might because we misused `LogicalType.equals`
> > > which
> > > > will checks field names as well.
> > > > I have created an issue (FLINK-13290) to track this problem.
> > > >
> > > > Best,
> > > > Jark
> > > >
> > > >
> > > > On Tue, 16 Jul 2019 at 17:35, kaka chen <[hidden email]>
> wrote:
> > > >
> > > > > I am looking this issue, the related codes which throw errors
> > are:
> > > > >
> > > > > SinkCodeGenerator:: validateFieldType()
> > > > >
> > > > > ...
> > > > >
> > > > > // Tuple/Case class/Row type requested
> > > > > case tt: TupleTypeInfoBase[_] =>
> > > > > fieldTypes.zipWithIndex foreach {
> > > > > case (fieldTypeInfo: GenericTypeInfo[_], i) =>
> > > > > val requestedTypeInfo = tt.getTypeAt(i)
> > > > > if
> > > > (!requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]]) {
> > > > > throw new TableException(
> > > > > s"Result field '${fieldNames(i)}' does not match
> > > > requested
> > > > > type. " +
> > > > > s"Requested: $requestedTypeInfo; Actual:
> > > > > $fieldTypeInfo")
> > > > > }
> > > > > case (fieldTypeInfo, i) =>
> > > > > val requestedTypeInfo = tt.getTypeAt(i)
> > > > > validateFieldType(requestedTypeInfo)
> > > > > * if (fromTypeInfoToLogicalType(fieldTypeInfo) !=*
> > > > > * fromTypeInfoToLogicalType(requestedTypeInfo) &&*
> > > > > *
> > > > !requestedTypeInfo.isInstanceOf[GenericTypeInfo[Object]])
> > > > > {*
> > > > > * val fieldNames = tt.getFieldNames*
> > > > > * throw new TableException(s"Result field
> > > '${fieldNames(i)}'
> > > > > does not match requested" +*
> > > > > * s" type. Requested: $requestedTypeInfo; Actual:
> > > > > $fieldTypeInfo")*
> > > > > * }*
> > > > > }
> > > > > ...
> > > > >
> > > > > Thanks,
> > > > > Frank
> > > > >
> > > > > kaka chen <[hidden email]> 于2019年7月16日周二下午5:23写道：
> > > > >
> > > > > > Hi All,
> > > > > >
> > > > > >
> > > > > > We are trying to switch to blink table planner in HBase
> connector,
> > > > which
> > > > > > found the following error:
> > > > > >
> > > > > >
> > > > > > Running org.apache.flink.addons.hbase.HBaseSinkITCase
> > > > > >
> > > > > > Formatting using clusterid: testClusterID
> > > > > >
> > > > > > Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > > 28.042
> > > > > sec
> > > > > > <<< FAILURE! - in org.apache.flink.addons.hbase.HBaseSinkITCase
> > > > > >
> > > > > > testTableSink(org.apache.flink.addons.hbase.HBaseSinkITCase)
> Time
> > > > > > elapsed: 2.431 sec <<< ERROR!
> > > > > >
> > > > > > org.apache.flink.table.api.TableException: Result field 'family1'
> > > does
> > > > > not
> > > > > > match requested type. Requested: Row(col1: Integer); Actual:
> > > > Row(EXPR$0:
> > > > > > Integer)
> > > > > >
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.flink.addons.hbase.HBaseSinkITCase.testTableSink(HBaseSinkITCase.java:140)
> > > > > >
> > > > > >
> > > > > > The original flink table planner executed successfully.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Kaka Chen
> > > > > >
> > > > >
> > > >
> > >
> >
>