Kafka Topics View

These displays allow you to view metrics for all Kafka topics on a particular topic in heatmap/table format, view the performance metrics for a single topic on a particular broker, view the metrics for all topics on a particular cluster, and view metrics for a particular topic on a particular cluster. Clicking Kafka Topics from the left/navigation menu opens the Kafka Topics Table display, which shows a tabular view of all clusters and their associated metrics. The options available under Kafka Topics are:

•

Kafka Topics Heatmap: Opens the Kafka Topics Heatmap display, which allows you to view performance metrics for all servers on a particular cluster.

•

Single Topic Summary: Opens the Kafka Single Topic Summary display, which allows you to view current metrics and trend data for a single topic.

•

Single Topic Partition Summary: Opens the Kafka Single Topic Partition Summary display, which allows you to view current metrics and trend data for a single topic partition.

•

Topics Activity by Cluster: Opens the Kafka Topics Activity by Cluster display, which allows you to view performance metrics for all topics on a particular cluster.

•

Brokers Activity by Topic: Opens the Kafka Brokers Activity by Topic display, which allows you to view performance metrics for all brokers for a particular topic.

Kafka Topics Table

The table in this display provides a view of all of your topics for a particular cluster and their associated metric data. Each row in the table contains data for a particular topic. Click a column header to sort column data in ascending or descending order. Double-click on a table row to drill-down to the Kafka Single Topic Summary display and view metrics for that particular topic. Toggle between the commonly accessed displays by clicking the drop down list on the display title.

Filter By
	Cluster	Select the cluster for which you want to view data.
Topics		The total number of topics listed in the table.
Rate		Select the option for which you want to view data: MeanRate -- Select this option to view the average rate for each metric for the topics in the display. One Minute-- Select this option to view the rate of incoming messages (per second) averaged over a one minute period for each metric for the topics in the display. Five Minute -- Select this option to view the rate of incoming messages (per second) averaged over a five minute period for each metric for the topics in the display. Fifteen Minute -- Select this option to view the rate of incoming messages (per second) averaged over a fifteen minute period for each metric for the topics in the display.
Table:
	Cluster	Lists the name of the cluster.
	Broker	The name of the broker.
	Topic	Lists the name of the topic.
	In Bytes/s	The rate of incoming bytes.
	Out Bytes/s	The rate of outgoing bytes.
	Rejected Bytes/s	The rate of rejected bytes.
	Failed Fetched Requests/s	The rate of failed fetch requests.
	Failed Produce Requests/s	The rate of failed produce requests.
	Fetched Msg Conversions/s	The rate of fetched message conversions.
	In Msgs/s	The rate of incoming messages.
	Produced Msgs Conversions/s	The rate of produced message conversions.
	Total Fetch Requests/s	The rate of total fetch requests.
	Total Produce Requests/s	The rate of total produce requests.

Kafka Topics Heatmap

Clicking Kafka Topics Heatmap in the left/navigation menu opens the Kafka Topics Heatmap, which provides an easy-to-view interface that allows you to quickly identify the current status of each of your topics for each available metric. You can view the topics in the heatmap based on the following metrics: the rate of incoming messages, the rate of incoming bytes, the rate of outgoing bytes, the rate of rejected bytes, the rate of total fetch requests, the rate of failed fetch requests, and the rate of total produce requests. By default, this display shows the heatmap based on the In Msgs/s metric.

Each rectangle in the heatmap represents a topic. The rectangle color indicates the most critical alert state associated with the topic. Choose a cluster from the drop-down menu to view all topics for that cluster. Choose a different metric to display from the Metric drop-down menu. Use the Show Broker check-box to include or exclude labels in the heatmap. Mouse over a rectangle to see additional metrics.

Drill-down and investigate a topic by clicking a rectangle in the heatmap to view details in the Kafka Single Topic Summary display.

Fields and Data:
	Cluster	Select the cluster for which you want to view data.
Count		Lists the number of topics displayed in the heatmap.
Rate		Select the option for which you want to view data: MeanRate -- Select this option to view the average rate for each metric for the topics in the display. One Minute-- Select this option to view the rate of incoming messages (per second) averaged over a one minute period for each metric for the topics in the display. Five Minute -- Select this option to view the rate of incoming messages (per second) averaged over a five minute period for each metric for the topics in the display. Fifteen Minute -- Select this option to view the rate of incoming messages (per second) averaged over a fifteen minute period for each metric for the topics in the display.
Show Broker		Select this check box to display the names of the Brokers at the top of each rectangle in the heatmap.
Heatmap
	Log Scale	Select this check box to enable a logarithmic scale. Use Log Scale to see usage correlations for data with a wide range of values. For example, if a minority of your data is on a scale of tens, and a majority of your data is on a scale of thousands, the minority of your data is typically not visible in non-log scale graphs. Log Scale makes data on both scales visible by applying logarithmic values rather than actual values to the data.
	Auto Scale	Select to enable auto-scaling. When auto-scaling is activated, the color gradient bar's maximum range displays the highest value. Note: Some metrics auto-scale automatically, even when Auto Scale is not selected.
	Metric	Select the metric driving the heatmap display. The default is In Msgs/s. Each Metric has a color gradient bar that maps values to colors. Mouse-over any rectangle to display the current values of the metrics for the topic. Click on a rectangle to drill-down to the associated Kafka Single Topic Summary display for a detailed view of metrics for that particular topic.
		In Msgs/s	The rate of incoming messages (per second). The color gradient bar, populated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum number of incoming messages per second. The middle value in the gradient bar indicates the middle value of the range.
		In Bytes/s	The rate of incoming bytes (per second). The color gradient bar, populated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum number of incoming bytes per second. The middle value in the gradient bar indicates the middle value of the range.
		Out Bytes/s	The rate of outgoing bytes (per second). The color gradient bar, populated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum number of outgoing bytes per second. The middle value in the gradient bar indicates the middle value of the range.
		Rejected Bytes/s	The rate of bytes being rejected (per second). The color gradient bar shows the range of the value/color mapping. ated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum number of bytes rejected per second. The middle value in the gradient bar indicates the middle value of the range.
		Total Fetch Requests/s	The rate of fetch requests (per second). The color gradient bar, populated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum number of fetch requests per second. The middle value in the gradient bar indicates the middle value of the range.
		Failed Fetch Requests/s	The rate of failed fetch requests (per second). The color gradient bar, populated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum number of failed fetch requests per second. The middle value in the gradient bar indicates the middle value of the range.
		Total Produce Requests/s	The rate of total producer requests (per second). The color gradient bar, populated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum number of produce requests per second. The middle value in the gradient bar indicates the middle value of the range.

Kafka Single Topic Summary

Clicking Single Topic Summary in the left/navigation menu opens the Kafka Single Topic Summary display, which provides a view of the current metrics and trend data for a single topic. Clicking on the information boxes at the top of the display takes you to the Kafka Topics Table display, where you can view additional topic data.

There are three options in the trend graph: Throughput, Produce Requests, and Fetch Requests. In the Throughput option on the trend graph, you can view trend data for incoming message rate, incoming byte rate, outgoing byte rate, and rejected byte rate over a selected time range. In the Produce Requests option on the trend graph, you can view trend data for the total produce requests rate, failed produce requests rate, and lag rate over a selected time range. In the Fetch Requests option on the trend graph, you can view trend data for total fetch requests rate, failed fetch requests rate, and lag rate over a selected time range.

Clicking the Critical/Warning link at the bottom of the display opens the Alerts Table by Component display.

Note: Fields/columns with an asterisk (*) at the end of the field/column definition contain data that is provided by the selected cluster. Refer to KAFKA documentation for more information regarding these fields.

Filter By: The display might include these filtering options:
	Cluster	Select the cluster for which you want to see data.
	Broker	Select the broker for which you want to see data.
	Topic	Select the topic for which you want to see data.
Rate		Select the option for which you want to view data: MeanRate -- Select this option to view the average rate for each metric for the topics in the display. One Minute-- Select this option to view the rate of incoming messages (per second) averaged over a one minute period for each metric for the topics in the display. Five Minute -- Select this option to view the rate of incoming messages (per second) averaged over a five minute period for each metric for the topics in the display. Fifteen Minute -- Select this option to view the rate of incoming messages (per second) averaged over a fifteen minute period for each metric for the topics in the display.
Inbound Msgs/s		The rate of incoming messages, per second.
Inbound/Outbound KB/s		The rate of inbound and outbound kilobytes, per second.
Lag Rate		The rate of change in the amount of lag.
Total Produce Requests/s		The rate of total produce requests.
Total Fetch Requests/s		The rate of total fetch requests.
Failed Fetch/Produce Requests/s		The rate of failed fetch requests, and the rate of produce requests.
Trend Graphs		Throughput In Msgs/s -- traces the selected rate of incoming messages. In Bytes/s -- traces the selected rate of incoming bytes. Out Bytes/s -- traces the selected rate of outgoing bytes. Rejected Bytes/s -- traces the selected rate of rejected bytes. Produce Requests Total Produce Requests/s -- traces the selected rate of total produce requests. Failed Produce Requests/s -- traces the selected rate of failed produce requests. Lag Rate -- traces the rate of change in the amount of lag. Fetch Requests Total Fetch Requests/s-- traces the selected rate of total fetch requests. Failed Fetch Requests/s -- traces the selected rate of failed fetch requests. Lag Rate -- traces the rate of change in the amount of lag.
		Log Scale	Select to enable a logarithmic scale. Use Log Scale to see usage correlations for data with a wide range of values. For example, if a minority of your data is on a scale of tens, and a majority of your data is on a scale of thousands, the minority of your data is typically not visible in non-log scale graphs. Log Scale makes data on both scales visible by applying logarithmic values rather than actual values to the data.
		Time Settings	Select a time range from the drop down menu varying from 5 Minutes to Last 7 Days. By default, the time range end point is the current time. To change the time range, deselect the now toggle, which displays some additional date fields. You can click the left and right arrow buttons to decrease the end time by one time period (the time selected in the Time range drop down) per click, or you can choose the date and time from the associated calendar and clock icons. You can also enter the date and time in the text field using the following format: MMM dd, YYYY HH:MM:ss. For example, Aug 21, 2018 12:24 PM. Click the now toggle to reset the time range end point to the current time.
Last Update		The date and time of the last data update.
Critical/Warning		The total number of critical and warning alerts.
Expired		When checked, performance data in the row has not been received within the time specified (in seconds) in the Expire Time field in the RTView Configuration Application > (KAFKAMON-LOCAL/Project Name) > Solution Package Configuration > Apache Kafka > DATA STORAGE > Duration > Expire Time property. The RTView Configuration Application > (KAFKAMON-LOCAL/Project Name) > Solution Package Configuration > Apache Kafka > DATA Storage > Duration > Delete Time property allows you to define the amount of time (in seconds) in which the row will be removed from the table if there is no response. For example, if Expire Time was set to 120 and Delete Time was set to 3600, then the Expired check box would be checked after 120 seconds and the row would be removed from the table after 3600 seconds.

Kafka Single Topic Partition Summary

Clicking Single Topic Partition Summary in the left/navigation menu opens the Kafka Single Topic Partition Summary display, which provides a view of the lag per partition, where the partitions are identified by the broker hosting the partition and the consumer reading the partition for a single topic, and also a list (table) of all topic partitions and their associated brokers and metrics.

Filter By: The display might include these filtering options:
	Cluster	Select the cluster for which you want to see data.
	Topic	Select the topic for which you want to see data.
Lag Per Broker:: Consumer		Lag per partition, where the partitions are identified by the broker hosting the partition and the consumer reading the partition.
Topic Partitions Per Broker::Consumer Table
	Broker:: Consumer	The name of the broker/consumer.
	Partition	The name of the partition.
	Leader	When checked, signifies that the broker is a leader on the partition.
	Log Size	The current number of messages in the log.*
	Log Start Offset	The offset of the first message written to the log.
	Current Offset	The offset of the message currently being consumed.*
	Log End Offset	The offset of the last message written to the log.*
	Lag	The difference between the current consumer position in the partition and the end of the log.*
	Lag Delta	The difference in the amount of lag from the previous polling period to the current polling period.*
	Lag Rate	The rate of change in the amount of lag.*
	Current Delta	The difference between the current consumer position in the partition from the previous polling period to the current polling period.*
	Current Rate	The rate of change of the current consumer position.*
	Log End Delta	The difference between the offset of the last message in the partition from the previous polling period to the current polling period.*
	Log End Rate	The rate of change of the last message offset.*

Kafka Topics Activity by Cluster

Clicking Topics Activity by Cluster in the left/navigation menu opens the Kafka Topics Activity by Cluster display, which provides a view of the activity metrics on all topics for a particular cluster. You can view the metrics based on the mean rate, a 1 minute average rate, a 5 minute average rate, or a 15 minute average rate.

Select the metric from the Sort By drop down by which you want to sort the data in the display. When using this option with the Sort Descending toggle, the topics will be sorted in ascending or descending order using the option you select from this drop down. For example, if you select In Msgs/s from this drop down and select the Sort Descending toggle, the topics listed in the display will be sorted so that the topic with the most In Msgs/s will be listed at the top followed by the topic with the next most In Msgs/s, and so on.

Select the option from the Topics Shown drop down to specify the number of topics you want to view.

Filter By:
	Cluster	Select the cluster for which you want to see data.
	Topic	Select the topic for which you want to see data.
Fields and Data
	Rate	Select the option for which you want to view data: Mean Rate -- Select this option to view the average rate for each metric for the topics in the display. 1 Minute Avg -- Select this option to view the rate of incoming messages (per second) averaged over a one minute period for each metric for the topics in the display. 5 Minute Avg -- Select this option to view the rate of incoming messages (per second) averaged over a five minute period for each metric for the topics in the display. 15 Minute Avg -- Select this option to view the rate of incoming messages (per second) averaged over a fifteen minute period for each metric for the topics in the display.
Sort By		Select the metric by which you want to sort the data in the display. When using this option with the Sort Descending toggle, the topics will be sorted in ascending or descending order using the option you select from this drop down. For example, if you select In KB/s from this drop down and select the Sort Descending toggle, the topics listed in the display will be sorted so that the topic with the most In KB/s will be listed at the top followed by the topic with the next most In KB/s, and so on.
Descending		When toggled on, the topics listed in the display are sorted in descending order based on the selected metric in the Sort By drop down. When toggled off, the topics are listed in ascending order.
Topics Shown		Select the number of topics that you want to view in the trend graphs below.
Topics		The number of topics found in the cluster.
Graphs
	Consumer Lag	The difference between the current consumer position in the partition and the end of the log.*
	In KB/s	The number of incoming kilobytes per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of incoming kilobytes per second for 1 minute.
	Out KB/s	The number of outgoing kilobytes per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of outgoing kilobytes per second for 1 minute.
	Total Fetch Reqs/s	The total number of fetch requests per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of fetch requests per second for 1 minute.
	Total Produce Reqs/s	The total number of produce requests per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of producer requests per second for 1 minute.

Kafka Brokers Activity by Topic

Clicking Brokers Activity by Topic in the left/navigation menu opens the Kafka Brokers Activity by Topic display, which provides a view of the activity metrics on all brokers for a particular topic. You can view the metrics based on the mean rate, a 1 minute average rate, a 5 minute average rate, or a 15 minute average rate.

Select the metric from the Sort By drop down by which you want to sort the data in the display. When using this option with the Sort Descending toggle, the brokers will be sorted in ascending or descending order using the option you select from this drop down. For example, if you select In Msgs/s from this drop down and select the Sort Descending toggle, the brokers listed in the display will be sorted so that the broker with the most In Msgs/s will be listed at the top followed by the broker with the next most In Msgs/s, and so on.

Filter By:
	Cluster	Select the cluster for which you want to see data.
	Topic	Select the topic for which you want to see data.
	Rate	Select the option for which you want to view data: Mean Rate -- Select this option to view the average rate for each metric for the brokers in the display. 1 Minute Avg -- Select this option to view the rate of incoming messages (per second) averaged over a one minute period for each metric for the brokers in the display. 5 Minute Avg -- Select this option to view the rate of incoming messages (per second) averaged over a five minute period for each metric for the brokers in the display. 15 Minute Avg -- Select this option to view the rate of incoming messages (per second) averaged over a fifteen minute period for each metric for the brokers in the display.
Sort By		Select the metric by which you want to sort the data in the display. When using this option with the Sort Descending toggle, the brokers will be sorted in ascending or descending order using the option you select from this drop down. For example, if you select In Msgs/s from this drop down and select the Sort Descending toggle, the brokers listed in the display will be sorted so that the broker with the most In Msgs/s will be listed at the top followed by the broker with the next most In Msgs/s, and so on.
Sort Descending		When toggled on, the brokers listed in the display are sorted in descending order based on the selected metric in the Sort By drop down. When toggled off, the brokers are listed in ascending order.
Brokers		The number of brokers found in the cluster with the associated topic.
Graphs
	In Msgs/s	The number of incoming messages per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of incoming messages per second for 1 minute.
	In KB/s	The number of incoming kilobytes per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of incoming kilobytes per second for 1 minute.
	Out KB/s	The number of outgoing kilobytes per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of outgoing kilobytes per second for 1 minute.
	Total Fetch Reqs/s	The total number of fetch requests per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of fetch requests per second for 1 minute.
	Total Produce Reqs/s	The total number of produce requests per second. For example, if you select 1 Minute Avg from the drop down list, the average rate of producer requests per second for 1 minute.

RTView© Enterprise User's Guide Version 6.4