Kafka Producers View
These displays provide detailed data for all producers or for a particular producer. Clicking Kafka Producers from the left/navigation menu opens the Kafka Producers Table display, which provides a view of all of your producers and their associated metric data. The options available under Kafka Producers are:
• | Producers Heatmap: Opens the Kafka Producers Heatmap display, which allows you to view all producers and their associated metrics in a particular cluster. |
• | Single Producer Summary: Opens the Kafka Single Producer Summary display, which contains current and historical metrics, as well as trend data, for a single producer. |
• | Single Producer JVM Runtime Summary: Opens the Kafka Single Producer JVM Runtime Summary display, which contains current and historical JVM runtime metrics, as well as trend data, for a single producer. |
Kafka Producers Table
The table in this display provides a view of all of your producers and their associated metric data including connection, alert level, alert count, cluster name, client ID, and the current value of each gathered metric. Each row in the table contains data for a particular producer. Click a column header to sort column data in ascending or descending order. Double-click on a table row to drill-down to the Kafka Single Producer Summary display and view metrics for that particular producer. Toggle between the commonly accessed displays by clicking the drop down list on the display title.
Note: Fields/columns with an asterisk (*) at the end of the field/column definition contain data that is provided by the selected cluster. Refer to KAFKA documentation for more information regarding these fields.
Filter By: |
||||
|
Cluster |
Select the cluster for which you want to view data. |
||
Producers |
The number of producers found on the selected cluster, and that are listed in the All Producers Table. |
|||
Table: |
||||
|
Cluster |
The name of the cluster.* |
||
|
Producer |
The name of the producer. |
||
|
Alert Level |
The current alert severity.
|
||
|
Alert Count |
The total number of alerts for the host. |
||
|
Client ID |
The ID of the producer.* |
||
|
Batch Size Average |
The average batch size sent by the producer.* |
||
|
Batch Size Max |
The maximum number of messages that can be added to a batch before being sent to the event handler.* |
||
|
Buffer Available Bytes |
The number of available bytes in the buffer.* |
||
|
Buffer Exhausted Rate |
The average per-second number of record sends that are dropped due to buffer exhaustion.* |
||
|
Buffer Total Bytes |
The total number of bytes allowed in the buffer.* |
||
|
Buffer Pool Wait Ratio |
The fraction of time an appender waits for space allocation.* |
||
|
Compression Avg Rate |
The average compression rate of record batches.* |
||
|
Connection Close Rate |
The rate of connections being closed.* |
||
|
Connections |
The number of active connections.* |
||
|
Connection Creation Rate |
The rate of connections being created.* |
||
|
Producer In Bytes/s |
The average number of incoming bytes per second.* |
||
|
IO Ratio |
The rate of input/output operations.* |
||
|
IO Time NS Avg |
The average length of time the I/O thread spent waiting for a socket (in nanoseconds).* |
||
|
IO Wait Ratio |
The percent of time the producer was performing I/O operations while the CPU was idle.* |
||
|
IO Wait Time ms Avg |
The average length of time the I/O thread spent waiting for a socket (in milliseconds).* |
||
|
Metadata Age |
The age (in seconds) of the current producer metadata being used.* |
||
|
Network IO Rate |
The rate of input/output network operations.* |
||
|
Producer Out Bytes/s |
The average number of outgoing bytes per second.* |
||
|
Produce Throttle Time Avg |
The avg time (in milliseconds) a request was throttled by a broker.* |
||
|
Produce Throttle Time Max |
The maximum time (in milliseconds) a request was throttled by a broker.* |
||
|
Record Errors/s |
The average per-second number of record sends that resulted in errors for a topic.* |
||
|
Avg Record Queue Time ms |
The average time (in milliseconds) record batches spent in the record accumulator.* |
||
|
Max Record Queue Time ms |
The maximum time (in milliseconds) record batches spent in the record accumulator.* |
||
|
Record Retry Rate |
The average per-second number of retried record sends. |
||
|
Producer Sent Records/s |
The average number of records sent (per second) for a topic.* |
||
|
Record Size Avg |
The average record size.* |
||
|
Record Size Max |
The maximum record size.* |
||
|
Records per Request Avg |
The average number of records per request.* |
||
|
Request Latency Avg |
The average request latency (in milliseconds).* |
||
|
Request Latency Max |
The maximum request latency (in milliseconds).* |
||
|
Request Rate |
The average number of requests sent per second.* |
||
|
Request Size Avg |
The average request size.* |
||
|
Request Size Max |
The maximum request size.* |
||
|
Requests In Flight |
The current number of in-flight requests awaiting a response.* |
||
|
Response Rate |
The average number of responses received per second.* |
||
|
Select Rate |
The number of times the I/O layer checked for new I/O operations to perform per second.* |
||
|
Waiting Threads |
The number of user threads blocked waiting for buffer memory to enqueue their records.* |
||
|
Jmx Connection |
The JMX connection string.* |
||
|
Kafka Version |
The current version of Apache Kafka installed.* |
||
|
Connected |
Denotes whether or not the producer is connected. |
||
|
Expired |
When checked, performance data in the row has not been received within the time specified (in seconds) in the Expire Time field in the RTView Configuration Application > (KAFKAMON-LOCAL/Project Name) > Solution Package Configuration > Apache Kafka > DATA STORAGE > Duration > Expire Time property. The RTView Configuration Application > (KAFKAMON-LOCAL/Project Name) > Solution Package Configuration > Apache Kafka > DATA Storage > Duration > Delete Time property allows you to define the amount of time (in seconds) in which the row will be removed from the table if there is no response. For example, if Expire Time was set to 120 and Delete Time was set to 3600, then the Expired check box would be checked after 120 seconds and the row would be removed from the table after 3600 seconds. |
||
|
Time Stamp |
The date and time the row data was last updated. |
Kafka Producers Heatmap
Clicking Producers Heatmap in the left/navigation menu opens the Kafka Producers Heatmap, which provides an easy-to-view interface that allows you to quickly identify the current status of each of your producers for each available metric. You can view the producers in the heatmap based on the following metrics: the current alert severity, the current alert count, the incoming/outgoing byte rate, the IO wait time, the request latency, and the request/response rates. By default, this display shows the heatmap based on the Alert Severity metric.
Each rectangle in the heatmap represents a producer. The rectangle color indicates the most critical alert state associated with the producer. Choose a cluster from the drop-down menu to view all producers for that cluster. Mouse over a rectangle to see additional metrics.
Drill-down and investigate a producer by clicking a rectangle in the heatmap to view details in the Kafka Single Producer Summary display.
Filter By |
||||
|
Cluster |
Select the cluster for which you want to view data. |
||
Fields and Data |
||||
|
Producers |
The number of producers found on the cluster and displayed in the heatmap. |
||
|
Show Cluster |
Select this check box to display the names of the clusters at the top of each rectangle in the heatmap. |
||
Heatmap |
||||
|
Log Scale |
Select this check box to enable a logarithmic scale. Use Log Scale to see usage correlations for data with a wide range of values. For example, if a minority of your data is on a scale of tens, and a majority of your data is on a scale of thousands, the minority of your data is typically not visible in non-log scale graphs. Log Scale makes data on both scales visible by applying logarithmic values rather than actual values to the data. |
||
|
Auto Scale |
Select to enable auto-scaling. When auto-scaling is activated, the color gradient bar's maximum range displays the highest value. Note: Some metrics auto-scale automatically, even when Auto Scale is not selected. |
||
|
Metric |
Select the metric driving the heatmap display. The default is Alert Severity. Each Metric has a color gradient bar that maps values to colors. Mouse-over any rectangle to display the current values of the metrics for the producer. Click on a rectangle to drill-down to the associated Kafka Single Producer Summary display for a detailed view of metrics for that particular producer. |
||
|
|
Alert Severity |
The current alert severity. Values range from 0 - 2, as indicated in the color gradient
|
|
|
|
Alert Count |
The total number of critical and warning unacknowledged alerts in the adapters. The color gradient |
|
|
|
Incoming Bytes/s |
The rate of incoming bytes (per second). The color gradient When Auto Scale is checked, the numeric values in the color gradient bar show the range of the data being displayed rather than the default values. The middle value changes accordingly to indicate the color of the middle value of the range. |
|
|
|
Outgoing Bytes/s |
The rate of outgoing bytes (per second). The color gradient bar When Auto Scale is checked, the numeric values in the color gradient bar show the range of the data being displayed rather than the default values. The middle value changes accordingly to indicate the color of the middle value of the range. |
|
|
|
Avg IO Wait Time Avg ms |
The average length of time the IO thread spent waiting for a socket (in milliseconds). The color gradient When Auto Scale is checked, the numeric values in the color gradient bar show the range of the data being displayed rather than the default values. The middle value changes accordingly to indicate the color of the middle value of the range. |
|
|
|
Avg Request Latency |
The average amount of time between when a producer is called and when the producer receives a response from the broker. The color gradient When Auto Scale is checked, the numeric values in the color gradient bar show the range of the data being displayed rather than the default values. The middle value changes accordingly to indicate the color of the middle value of the range. |
|
|
|
Requests/s |
The average number of requests sent per second. The color gradient When Auto Scale is checked, the numeric values in the color gradient bar show the range of the data being displayed rather than the default values. The middle value changes accordingly to indicate the color of the middle value of the range. |
|
|
|
Responses/s |
The average number of responses received (per second). The color gradient When Auto Scale is checked, the numeric values in the color gradient bar show the range of the data being displayed rather than the default values. The middle value changes accordingly to indicate the color of the middle value of the range. |
Kafka Single Producer Summary
Clicking Single Producer Summary in the left/navigation menu opens the Kafka Single Producer Summary display, which provides a view of the current and historical metrics for a single producer.
Clicking on the information boxes at the top of the display takes you to the Kafka Producers Table display, where you can view additional producers data.
There are two options in the trend graph: Performance and JVM. In the Performance option on the trend graph, you can view trend data for the requests rate, responses rate, maximum and average request latency, outgoing kilobyte rate, and average IO wait time over a selected time range. In the JVM option on the trend graph, you can view trend data for JVM CPU percentage, maximum memory in megabytes, committed memory in megabytes, used memory in megabytes, and number of live threads over a selected time range.
Clicking the Critical/Warning link at the bottom of the display opens the Alerts Table by Component display.
Note: Fields/columns with an asterisk (*) at the end of the field/column definition contain data that is provided by the selected cluster. Refer to KAFKA documentation for more information regarding these fields.
Filter By: |
||||
|
Cluster |
Select the cluster for which you want to show data in the display. |
||
|
Producer |
Select the producer for which you want to show data in the display. |
||
Fields and Data |
||||
|
Incoming/Outgoing KB/s |
The rate of incoming and outgoing bytes (kilobytes per second).* |
||
|
Requests/Responses/s |
The average number of requests sent per second, and the average number of responses received per second.* |
||
|
Network IO Rate |
The rate of input/output network operations.* |
||
|
Avg/Max Request Latency ms |
The average request latency (in milliseconds), and the maximum request latency (in milliseconds).* |
||
|
Avg/Max Batch Size KB |
The average batch size sent by the producer, and the maximum number of messages that can be added to a batch before being sent to the event handler.* |
||
|
Avg/Max Record Queue Time ms |
The average time (in milliseconds) record batches spent in the record accumulator, and the maximum time (in milliseconds) record batches spent in the record accumulator.* |
||
Trend Graphs |
Performance Requests/s -- traces the number of requests per second. Responses/s -- traces the number of responses per second. Max Request Latency ms -- traces the maximum request latency (in milliseconds). Avg Request Latency ms -- traces the average request latency (in milliseconds). Outgoing KBs/s -- traces the rate of outgoing bytes (kilobytes per second). Avg IO Wait ms -- traces the average length of time the I/O thread spent waiting for a socket (in milliseconds). JVM JVM CPU % -- traces the CPU being used by the JVM. Max Mem MB -- traces the maximum amount of available heap. Committed Mem MB -- traces the amount of committed heap memory. Used Mem MB -- traces the highest amount of heap used. Live Threads -- traces the number of live threads. |
|||
|
|
Log Scale |
Select to enable a logarithmic scale. Use Log Scale to see usage correlations for data with a wide range of values. For example, if a minority of your data is on a scale of tens, and a majority of your data is on a scale of thousands, the minority of your data is typically not visible in non-log scale graphs. Log Scale makes data on both scales visible by applying logarithmic values rather than actual values to the data. |
|
|
|
Time Settings |
Select a time range from the drop down menu varying from 5 Minutes to Last 7 Days. By default, the time range end point is the current time.
To change the time range, deselect the now toggle, which displays some additional date fields. You can click the left and right arrow buttons to decrease the end time by one time period (the time selected in the Time range drop down) per click, or you can choose the date and time from the associated calendar and clock icons. You can also enter the date and time in the text field using the following format: MMM dd, YYYY HH:MM:ss. For example, Aug 21, 2018 12:24 PM. Click the now toggle to reset the time range end point to the current time.
|
|
Client ID |
The ID of the client. |
|||
Peak Threads |
The highest number of threads running at one time during the current polling period.* |
|||
Critical/Warning |
The number of critical and warning alerts. |
|||
Total Memory Available After GC MB |
The amount of memory available after garbage collection takes place. |
|||
Last Update |
The date and time of the last data update. |
Kafka Single Producer JVM Runtime Summary
Clicking Single Producer JVM Runtime Summary in the left/navigation menu opens the Kafka Single Producer JVM Runtime Summary display, which provides a view of JVM runtime statistics and trend data for the selected producer.
Clicking on the information boxes at the top of the display takes you to the Kafka Producers Table display, where you can view additional producers data.
The Performance Trends trend graph provides trend data for the CPU percentage, maximum memory in megabytes, committed memory in megabytes, and the used memory in megabytes over a selected time range.
Clicking the Critical/Warning link at the bottom of the display opens the Alerts Table by Component display.
Note: Fields/columns with an asterisk (*) at the end of the field/column definition contain data that is provided by the selected cluster. Refer to KAFKA documentation for more information regarding these fields.
Filter By: |
||||
|
Cluster |
Select the cluster for which you want to show data in the display. |
||
|
Producer |
Select the producer for which you want to show data in the display. |
||
Fields and Data |
||||
|
CPU % |
The percentage of CPU being used by producer.* |
||
|
Used Memory % |
The percentage of memory used by the producer. |
||
|
Committed Mem MB |
The amount of committed heap memory, in megabytes, by the producer. |
||
|
Max Memory MB |
The maximum amount of heap memory, in megabytes, defined for the producer. |
||
|
Threads |
The number of threads running on the producer. |
||
|
Peak Threads |
The peak number of threads running on the producer. |
||
Performance Trends Graph |
Traces the following: CPU % -- traces the percentage of CPU being used by the JVM. Max Mem MB -- traces the maximum amount of available heap, in megabytes. Committed Mem MB -- traces the amount of committed heap, in megabytes. Used Mem MB -- traces the amount of heap used, in megabytes. |
|||
JMX Connection |
The name of the JMX connection.* |
|||
OS Version |
The version number of the operating systems.* |
|||
Process Name |
The name of the process.* |
|||
Start Time |
The date and time when the producer was started.* |
|||
Critical/Warning |
The number of critical and warning alerts. |
|||
Architecture |
The type of processor being used.* |
|||
JDK |
The JDK version number.* |
|||
Uptime |
The amount of time the producer has been up and running.* |
|||
Operating System |
The operating system installed on the producer.* |
|||
Last Update |
The date and time of the last data update. |