[GitHub] [flink-web] MarkSfik commented on a change in pull request #328: Add blog post: "Memory Management improvements with Apache Flink 1.10"

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] [flink-web] MarkSfik commented on a change in pull request #328: Add blog post: "Memory Management improvements with Apache Flink 1.10"

GitBox

MarkSfik commented on a change in pull request #328:
URL: https://github.com/apache/flink-web/pull/328#discussion_r411336465



##########
File path: _posts/2020-04-17-memory-management-improvements-flink-1.10.md
##########
@@ -0,0 +1,87 @@
+---
+layout: post
+title: "Memory Management improvements with Apache Flink 1.10"
+date: 2020-04-17T12:00:00.000Z
+authors:
+- andrey:
+  name: "Andrey Zagrebin"
+categories: news
+excerpt: This post discusses the recent changes to the memory model of the task managers and configuration options for your Flink applications in Flink 1.10.
+---
+
+Apache Flink 1.10 comes with significant changes to the memory model of the task managers and configuration options for your Flink applications. These recently-introduced changes make Flink more adaptable to all kinds of deployment environments (e.g. Kubernetes, Yarn, Mesos), providing strict control over its memory consumption. In this post, we describe Flink’s memory model, as it stands in Flink 1.10, how to set up and manage memory consumption of your Flink applications and the recent changes the community implemented in the latest Apache Flink release.
+
+## Introduction to Flink’s memory model
+
+Having a clear understanding of Apache Flink’s memory model allows you to manage resources for the various workloads more efficiently. The following diagram illustrates the main memory components in Flink:
+
+<center>
+<img src="{{ site.baseurl }}/img/blog/2020-04-17-memory-management-improvements-flink-1.10/total-process-memory.svg" width="400px" alt="Figure 1: Rule definition"/>
+<br/>
+<i><small>Flink: Total Process Memory</small></i>
+</center>
+<br/>
+
+The task manager process is a JVM process. On a high level, its memory consists of the *JVM Heap* and *Off-Heap* memory. These types of memory are consumed by Flink directly or by JVM for its specific purposes (i.e. metaspace etc). There are two major memory consumers within Flink: the user code of job operator tasks and the framework itself consuming memory for internal data structures, network buffers etc.

Review comment:
       ```suggestion
   The Task Manager process is a JVM process. On a high level, its memory consists of the *JVM Heap* and *Off-Heap* memory. These types of memory are consumed by Flink directly or by JVM for its specific purposes (i.e. metaspace etc). There are two major memory consumers within Flink: the user code of job operator tasks and the framework itself consuming memory for internal data structures, network buffers etc.
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]