Andrey Zagrebin created FLINK-18751:
---------------------------------------
Summary: Implement SlotSharingExecutionSlotAllocator
Key: FLINK-18751
URL:
https://issues.apache.org/jira/browse/FLINK-18751 Project: Flink
Issue Type: Sub-task
Components: Runtime / Coordination
Reporter: Andrey Zagrebin
Assignee: Andrey Zagrebin
SlotSharingExecutionSlotAllocator maintains a SharedSlot for each ExecutionSlotSharingGroup. SlotSharingExecutionSlotAllocator allocates physical slots for SharedSlot(s) and then allocates logical slots from it for scheduled tasks. In this way, the slot sharing hints can be respected in the ExecutionSlotAllocator. And we no longer need to rely on the SlotProvider to do the slot sharing matching. Co-location constraints will be respected since co-located subtasks will be in the same ExecutionSlotSharingGroup.
The physical slot will be lazily allocated for a SharedSlot, upon any hosted subtask asking for the SharedSlot. Each subsequent sharing subtask allocates a logical slot from the SharedSlot. The SharedSlot/physical slot can be released only if all the requested logical slots are released or canceled.
h4. Slot Allocation Process
When SlotSharingExecutionSlotAllocator receives a set of tasks to allocate slots for, it should do the following:
# Map the tasks to ExecutionSlotSharingGroup(s)
# Check which ExecutionSlotSharingGroup(s) _already have_ SharedSlot(s)
# For all involved ExecutionSlotSharingGroup(s) _which do not have a SharedSlot_ yet.
# Create a SlotProfile future by MergingSharedSlotProfileRetriever and then
# Allocate a physical slot from the PhysicalSlotProvider
# Create SharedSlot based on the returned physical slot futures
# If physical slot future fails, remove the SharedSlot
# Allocate logical slot futures for the tasks from all corresponding SharedSlot(s).
# Generates SlotExecutionVertexAssignment(s) based on the logical slot futures and returns the results.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
