lining created FLINK-14431:
------------------------------
Summary: Update TaskManager's memory information to match its memory composition
Key: FLINK-14431
URL:
https://issues.apache.org/jira/browse/FLINK-14431 Project: Flink
Issue Type: Improvement
Components: Runtime / REST, Runtime / Task, Runtime / Web Frontend
Reporter: lining
h3. Motivation
There are several shortcomings of current (Flink 1.9) Flink TaskManager memory information show in rest api.
h4. (1) There isn't enough information for manage memory.
* First Manage Memory segment's total and avaliable in metrics.
* Second what's the type and segment size of it. Because if user want to calculate Task Heap, it need to know manage memory.
h4. (2) The information from HardwareDescription and TaskManagerMetricsInfo is difficult to match the memory composition of the TaskManager.
* Accoding to TaskManagerMetricsInfo's memorySegmentsTotal(ps: Network Segment total size), user couldn't get Network memory.
* What's the meaning of HardwareDescription.sizeOfJvmHeap.
* User couldn't get something about TaskHeap.
* From this information, user couldn't know to change which TaskManager's configuration need change.
h3. Proposed Changes
h4. Add manage memory metric
* Add registerMemoryManagerMetrics in MemoryManager:
* Register it in TaskManagerServices.createMemoryManager.
h4. Get TaskManager Resource Config from rest api
* Because of the resource configuration in each TaskManager may be different.
* Add TaskManagerResourceConfiguration In TaskManagerServicesConfiguration:
* In TaskManagerRunner.startTaskManager add TaskManagerResourceConfiguration to taskManagerConfiguration.
* Add TaskManagerResourceConfiguration in WorkerRegistration, so rest api could get it through ResourceManager.requestTaskManagerInfo .
h4. Add TaskManageResourceInfo which match the memory composition of the TaskManager
* cpuAllocated depend on flip-49 （TaskExecutorResourceSpec）
* cpuUsage = （metric Status.JVM.CPU.Load[from|
https://docs.oracle.com/javase/7/docs/jre/api/management/extension/com/sun/management/OperatingSystemMXBean.html#getProcessCpuLoad()]） * cpuAllocated
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
