Zhu Zhu created FLINK-14870:
-------------------------------
Summary: Introducing non-nullable assumption for task slot sharing group
Key: FLINK-14870
URL:
https://issues.apache.org/jira/browse/FLINK-14870 Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 1.10.0
Reporter: Zhu Zhu
In runtime, a null slot sharing group means the vertex should be run in an individual slot which is not shared with other vertices.
However, at the moment, there is no way to set the slot sharing group of a vertex to null in production. This is because {{StreamGraphGenerator}} would always assign a non-null slot sharing group to an operator({{StreamNode}}), and that would be the group of the generated {{JobVertex}}. (In the case of {{DataSet}}, all vertices are in the "default" group.)
Moreover, currently users are already able to force an operator to run in an individual slot by specify a different slot sharing group key to that operator.
So looks that supporting null slot sharing groups in runtime does not make much sense. And it's making things more complex, with codes like {{@Nullable}} annotations for slot sharing group, task required resources calculations, and single task slot allocation.
To make runtime code base and future development simpler, I'd propose to introduce the assumption that task slot sharing group should not be null.
The detailed proposal is:
1. In {{StreamingJobGraphGenerator}}, we check to ensure the slot sharing group key of a {{StreamNode}} is non-null, so that we always set a non-null {{SlotSharingGroup}} to a JobVertex
2. Remove all the {{@Nullable}} annotations for {{SlotSharingGroup}} and {{SlotSharingGroupID}} in several classes
3. Always use the slot sharing group resources as the physical slot resources for a task
4. Remove {{SchedulerImp#allocateSingleSlot}} and the related code paths
cc [~chesnay] [~gjy] [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
