Proxy Services

Proxy Services displays present detailed proxy server performance metrics for the cluster. Use the Proxy Services displays to quickly identify overloaded proxy services and locate the extend client connection causing the issue.

Proxy Services performance metrics include: CPU%, Requests, Request Average Duration, Request Pending Count, Task Backlog and Active Threads.

Proxy / Extend Overview

Heatmap shows performance utilization and a trend graph shows the total connections and total bytes transferred for all proxy services for the selected host or hosts.

Proxy / Extend Connections

Table shows proxy services data, including connections, CPU usage and bytes sent and received, and a trend graph displays messages and bytes sent and received for the selected remote endpoint.

The table data is the result of joins of metric from the following Coherence MBeans: Service and ConnectionManager. For details on attributes of these MBeans go to: http://download.oracle.com/otn_hosted_doc/coherence/350/com/tangosol/net/management/Registry.html.

Filter By: The display might include these filtering options:
	Cluster:	Select a cluster from the drop-down menu.
	Proxy Services	Location	A unique identifier for each node. It is defined as: member_name.machine.rack.site.
		HostIP	The IP address of the host where the proxy service resides.
		Alert Level	The maximum level of alerts in the row: Red indicates that one or more exceeded their ALARM LEVEL threshold. Yellow indicates that one or more exceeded their WARNING LEVEL threshold. Green indicates that none have exceeded their alert thresholds.
		Alert Count	The number of alerts in the row.
		Running	Indicates that the proxy service is running when selected.
		Connections	The number of extend clients connected to the selected host or hosts.
		CPU%	The average percent CPU utilization for each proxy service in the cluster.
		Bytes Sent	The number of bytes sent by the proxy service since the proxy service joined the cluster.
		Delta	The number of bytes sent by the proxy service since the last data sample.
		Backlog	The size (in kilobytes) of the backlog queue.
		Bytes Rcvd	The number of bytes received by the proxy service since the proxy service joined the cluster.
		Delta	The number of bytes received by the proxy service since the last data sample.
		MsgsSent	The number of messages sent by the proxy service since the proxy service joined the cluster.
		Delta	The number of messages sent by the proxy service since the last data sample.
		Backlog	The size of the backlog queue that holds messages scheduled to be sent by one of the proxy service pool threads.
		Msgs Rcvd	The number of messages received by the proxy service since the proxy service joined the cluster.
		Delta	The number of messages received by the proxy service since the last data sample.
		Tasks	The number of tasks performed by the proxy service since the last time the statistics were reset.
		RequestAverageDuration	The average duration (in milliseconds) of an individual synchronous request issued by the proxy service since the last time the statistics were reset.
		RequestMaxDuration	Maximum duration (in milliseconds) of an individual proxy service request since the last time the statistics were reset.
		RequestTotalCount	The number of requests issued and received by the proxy service.
		TaskAverageDuration	The average duration (in milliseconds) of an individual task execution.
		TaskBacklog	The size of the backlog queue that holds tasks scheduled to be executed by one of the proxy service pool threads.
		TaskCount	The number of tasks performed by the proxy service since the last time the statistics were reset.
		TaskHungCount	The total number of currently executing hung tasks.
		TaskHungDuration	The longest currently executing hung task duration in milliseconds.
		TaskHungTaskId	The id of the of the longest currently executing hung task.
		TaskHungThresholdMillis	The duration (in milliseconds) that a proxy service task can execute before it is considered hung. Note that a posted task that has not yet started is never considered as hung.
		TaskMaxBacklog	The maximum size of the proxy service backlog queue since the last time the statistics were reset.
		TaskTimeoutCount	The total number of timed-out proxy service tasks since the last time the statistics were reset.
		RequestPendingCount	The number of pending proxy service requests.
		RequestPendingDuration	The average duration (in milliseconds) that an individual proxy service request waits before being executed.
		RequestTimeoutCount	The total number of timed-out proxy service requests since the last time the statistics were reset.
		RequestTimeoutMillis	The duration (in milliseconds) for a proxy service request to reach the specified timeout threshold.
		TaskTimeoutMillis	The default timeout value (in milliseconds) for tasks that can be timed-out but do not explicitly specify the task execution timeout value.
		IncomingBufferPoolSize	The number of buffers in the incoming pool.
		ThreadAbandonedCount	The number of abandoned threads from the proxy service thread pool. A thread is abandoned and replaced with a new thread if it executes a task for a period of time longer than execution timeout and all attempts to interrupt it fail.
		ThreadCount	The number of threads in the proxy service thread pool.
		ThreadIdleCount	The number of currently idle threads in the proxy service thread pool.
		AverageActiveThreadCount	The average number of proxy service active threads, not currently idle, since the last time the statistics were reset.
		ThreadAverageActiveCount	The average number of active (not idle) threads in the service thread pool since the last time the statistics were reset.
		AverageTaskDuration	The average duration (in milliseconds) to perform a proxy service task since the last time the statistics were reset.
		MaximumBacklog	The maximum size of the backlog queue since the last time the statistics were reset.
		Throughput	The amount of data (in kilobytes) that is transferred by the proxy service.
		ThroughputInbound	The amount of data (in kilobytes) that is transferred from clients to the proxy service.
		ThroughputOutbound	The amount of data (in kilobytes) that is transferred from the proxy service to clients.
		IncomingBufferPoolCapacity	The size (in kilobytes) of the proxy service incoming buffer pool.
		OutgoingBufferPoolCapacity	The size (in kilobytes) of the proxy service outgoing buffer pool.
		OutgoingBufferPoolSize	The number of buffers in the proxy service outgoing pool.
		nodeld	The unique identifier for the proxy service.
		RefreshTime	The timestamp when this model was last retrieved from a corresponding node. For local servers it is the local time.
		HostName	The name of the host where the proxy service resides.
		MemberName	A specified, unique name of the host where the proxy service resides.
		SeniorMemberId	The proxy service senior member id. If the proxy service is not running, it is -1.
		Rate	The number of errors accumulated per second.
	Execution	Min	The shortest execution time of any process instance, in milliseconds.
		Max	The longest execution time of any process instance, in milliseconds.
		Average	The average execution time for all completed process instances, in milliseconds.
		Current	The amount of time accumulated this update cycle.
		Rate	The amount of time accumulated per second.
	Elapsed	Min	The shortest elapsed time of any process instance, in milliseconds.
		Max	The longest elapsed time of any process instance, in milliseconds.
		Average	The average elapsed time for all completed process instances, in milliseconds.
		Current	The amount of elapsed time accumulated this update cycle.
		Rate	The amount of elapsed time accumulated per second.
	Selected Proxy	This field is populated by the selection made in the Proxy Services table.
	Selected Endpoint	This field is populated by the selection made in the Remote Endpoint table.
Trend Graphs Select a host from the Proxy Services table and a connection from the Remote Endpoint table. This table is populated by the selection made in the Proxy Services table. Alert Level shows the maximum level of alerts in row: Red indicates that one or more exceeded their ALARM LEVEL threshold. Yellow indicates that one or more exceeded their WARNING LEVEL threshold. Green indicates that none have exceeded their alert thresholds. Msgs Received: Traces the number of messages received by the selected proxy service from the remote endpoint. Msgs Sent: Traces the number of bytes received by the selected proxy service from the remote endpoint. Bytes Received: Traces the rate at which the application is accumulating process execution time, in milliseconds per second. Bytes Sent: Traces the number of executed activities per second. All Activities Exec Time/sec: Traces the number of bytes sent by the selected proxy service to the remote endpoint.
	Time Range	Select a time range from the drop down menu varying from 2 Minutes to Last 7 Days, or display All Data. To specify a time range, click Calendar . By default, the time range end point is the current time. To change the time range end point, click Calendar and select a date and time from the calendar or enter the date and time in the text field using the following format: MMM dd, YYYY HH:MM. For example, Aug 21, 2011 12:24 PM. Use the navigation arrows to move forward or backward one time period. NOTE: The time period is determined by your selection from the Time Range drop-down menu. Click Restore to Now to reset the time range end point to the current time.

Proxy / Extend Detail

Table shows data for proxy services and extend client connection data, including remote endpoint, time stamp, connect time and outgoing byte backlog.

Cluster	Select a cluster from the drop-down menu.
Proxy Services
	Location	A unique identifier for each node. It is defined as: member_name.machine.rack.site.
	HostIP	The IP address of the host where the proxy service resides.
	Running	Indicates that the proxy service is running when selected.
	Connections	The number of extend clients connected to the selected host or hosts.
	CPU%	The average percent CPU utilization for each proxy service in the cluster.
	Bytes Sent	The number of bytes sent by the proxy service since the proxy service joined the cluster.
	Delta	The number of bytes sent by the proxy service since the last data sample.
	Backlog	The size (in kilobytes) of the backlog queue.
	Bytes Rcvd	The number of bytes received by the proxy service since the proxy service joined the cluster.
	Delta	The number of bytes received by the proxy service since the last data sample.
	MsgsSent	The number of messages sent by the proxy service since the proxy service joined the cluster.
	Delta	The number of messages sent by the proxy service since the last data sample.
	Backlog	The size of the backlog queue that holds messages scheduled to be sent by one of the proxy service pool threads.
	Msgs Rcvd	The number of messages received by the proxy service since the proxy service joined the cluster.
	Delta	The number of messages received by the proxy service since the last data sample.
	Tasks	The number of tasks performed by the proxy service since the last time the statistics were reset.
	RequestAverageDuration	The average duration (in milliseconds) of an individual synchronous request issued by the proxy service since the last time the statistics were reset.
	RequestMaxDuration	Maximum duration (in milliseconds) of an individual proxy service request since the last time the statistics were reset.
	RequestPendingCount	The number of pending proxy service requests.
	RequestPendingDuration	The average duration (in milliseconds) that an individual proxy service request waits before being executed.
	RequestTimeoutCount	The total number of timed-out proxy service requests since the last time the statistics were reset.
	RequestTimeoutMillis	The duration (in milliseconds) for a proxy service request to reach the specified timeout threshold.
	RequestTotalCount	The number of requests issued and received by the proxy service.
	TaskAverageDuration	The average duration (in milliseconds) of an individual task execution.
	TaskBacklog	The size of the backlog queue that holds tasks scheduled to be executed by one of the proxy service pool threads.
	TaskCount	The number of tasks performed by the proxy service since the last time the statistics were reset.
	TaskHungCount	The total number of currently executing hung tasks.
	TaskHungDuration	The longest currently executing hung task duration in milliseconds.
	TaskHungTaskId	The id of the of the longest currently executing hung task.
	TaskHungThresholdMillis	The duration (in milliseconds) that a proxy service task can execute before it is considered hung. Note that a posted task that has not yet started is never considered as hung.
	TaskMaxBacklog	The maximum size of the proxy service backlog queue since the last time the statistics were reset.
	TaskTimeoutCount	The total number of timed-out proxy service tasks since the last time the statistics were reset.
	TaskTimeoutMillis	The default timeout value (in milliseconds) for tasks that can be timed-out but do not explicitly specify the task execution timeout value.
	IncomingBufferPoolSize	The number of buffers in the incoming pool.
	ThreadAbandonedCount	The number of abandoned threads from the proxy service thread pool. A thread is abandoned and replaced with a new thread if it executes a task for a period of time longer than execution timeout and all attempts to interrupt it fail.
	ThreadCount	The number of threads in the proxy service thread pool.
	ThreadIdleCount	The number of currently idle threads in the proxy service thread pool.
	AverageActiveThreadCount	The average number of proxy service active threads, not currently idle, since the last time the statistics were reset.
	ThreadAverageActiveCount	The average number of active (not idle) threads in the service thread pool since the last time the statistics were reset.
	AverageTaskDuration	The average duration (in milliseconds) to perform a proxy service task since the last time the statistics were reset.
	MaximumBacklog	The maximum size of the backlog queue since the last time the statistics were reset.
	Throughput	The amount of data (in kilobytes) that is transferred by the proxy service.
	ThroughputInbound	The amount of data (in kilobytes) that is transferred from clients to the proxy service.
	ThroughputOutbound	The amount of data (in kilobytes) that is transferred from the proxy service to clients.
	IncomingBufferPoolCapacity	The size (in kilobytes) of the proxy service incoming buffer pool.
	OutgoingBufferPoolCapacity	The size (in kilobytes) of the proxy service outgoing buffer pool.
	OutgoingBufferPoolSize	The number of buffers in the proxy service outgoing pool.
	nodeld	The unique identifier for the proxy service.
	RefreshTime	The timestamp when this model was last retrieved from a corresponding node. For local servers it is the local time.
	HostName	The name of the host where the proxy service resides.
	MemberName	A specified, unique name of the host where the proxy service resides.
	SeniorMemberId	The proxy service senior member id. If the proxy service is not running, it is -1.
Extend Client Connections	Select a row from the Proxy Services table to populate client data in the table.
	Location	A unique identifier for each node. It is defined as: member_name.machine.rack.site.
	RemoteEndpoint	The IP address of the client.
	Timestamp	The date and time (in cluster time) that this client joined the proxy service.
	Connect Time (mins)	The duration (in minutes) the client has been connected to the proxy service.
	OutgoingByteBacklog	The size of the backlog queue (in bytes) that holds outgoing bytes scheduled to be executed by one of the proxy service pool threads for the client.
	OutgoingMessageBacklog	The number of messages in the backlog queue that holds outgoing messages scheduled to be sent to the client by one of the proxy service pool threads.
	TotalBytesReceived	The number of bytes received from the client by the proxy service since the client connected to the proxy service.
	Delta	The number of bytes received from the client by the proxy service since the last data sample.
	TotalBytesSent	The number of bytes sent to the client by the proxy service since the client connected to the proxy service.
	Delta	The number of bytes sent to the client by the proxy service since the last data sample.
	TotalMessagesReceived	The number of messages received from the client by the proxy service since the client connected to the proxy service.
	Delta	The number of messages received from the client by the proxy service since the last data sample.
	TotalMessagesSent	The number of messages sent to the client by the proxy service since the client connected to the proxy service.
	Delta	The number of messages sent to the client by the proxy service since the last data sample.
	UUID	The unique identifier for the extend client application.
	nodeld	The unique identifier for the proxy service the extend client is connected to.

Proxy Nodes History

Heatmap shows performance utilization, over time, for all proxy service nodes in the selected cluster. Use this display to assess performance, over time, for all proxy service nodes in a cluster. Analyze load distribution, check for bottlenecks and quickly identify proxy service nodes with high usage.

Extend Connections History

Heatmap shows performance utilization, over time, for all extend connections in the selected cluster.


Cluster:	Select a cluster for which to display data in the heatmap.
Hosts	Click to open display that shows GC Duty Cycle for all the nodes in a cluster.
Metric:	Select a metric for which to display data in the heatmap.
	Alert Severity	The maximum level of alerts in the heatmap rectangle. Values range from 0 - 2, as indicated in the color gradient bar, where 2 is the highest Alert Severity:
	Alert Count	The total number of critical and warning alerts in the heatmap rectangle. The color gradient bar, populated by the current heatmap, shows the value/color mapping. The numerical values in the gradient bar range from 0 to the maximum count of alerts in the heatmap. The middle value in the gradient bar indicates the average alert count.
	Current Bytes Sent	Total number of bytes sent by the selected proxy in the time range specified.
	Current Bytes Received	Total number of bytes received by the selected proxy in the time range specified.
	Proxy CPU%	The average percent CPU utilization for the selected proxy.
	Bytes Backlog	The number of pending bytes in the Extend outgoing queue.
	Proxy Bytes Backlog	The number of pending bytes in the Proxy outgoing queue.
Range	Select a time range from the drop down menu varying from 2 Minutes to Last 7 Days, or display All Data. To specify a time range, click Calendar . By default, the time range end point is the current time. To change the time range end point, click Calendar and select a date and time from the calendar or enter the date and time in the text field using the following format: MMM dd, YYYY HH:MM. For example, Aug 21, 2011 12:24 PM. Use the navigation arrows to move forward or backward one time period. NOTE: The time period is determined by your selection from the Time Range drop-down menu. Click Restore to Now to reset the time range end point to the current time.
Total Connections	The number of extend clients connected to the selected proxy.

Cluster	Select a cluster from the drop-down menu.
Metric	Select a metric from the drop-down menu.
	Request Pending	The number of pending requests issued by the node.
	Connections	Total number of connection for the node.
	Refresh Milliseconds	The amount of time, in milliseconds, since the last data sample.
	Delta Bytes Sent	Total number of bytes sent by the node since the last data sample.
	Delta Bytes Rcvd	Total number of bytes received by the node since the last data sample.
	Bytes Sent Per Second	Total bytes sent, per second, by the node.
Range	Select a time range from the drop down menu varying from 2 Minutes to Last 7 Days, or display All Data. To specify a time range, click Calendar . By default, the time range end point is the current time. To change the time range end point, click Calendar and select a date and time from the calendar or enter the date and time in the text field using the following format: MMM dd, YYYY HH:MM. For example, Aug 21, 2011 12:24 PM. Use the navigation arrows to move forward or backward one time period. NOTE: The time period is determined by your selection from the Time Range drop-down menu. Click Restore to Now to reset the time range end point to the current time.
Enable MouseOver	Select this option to make details visible upon mouseover.
Log Scale	Enable to use a logarithmic scale for the Y axis. Use Log Scale to see usage correlations for data with a wide range of values. For example, if a minority of your data is on a scale of tens, and a majority of your data is on a scale of thousands, the minority of your data is typically not visible in non-log scale graphs. Log Scale makes data on both scales visible by applying logarithmic values rather than actual values to the data.
Proxy Nodes Heatmap	Use the heatmap to view utilization trends for all Process and Storage nodes, over time, and quickly identify heavy usage, indicated by a dark color (by default, dark green). Look for a consistently dark horizontal line, which typically indicates constant high utilization. If this level of utilization is unexpected, consider performing a lower level analysis by viewing node details in the Node Summary display. Use the mouseover tool-tip to see the node hostname and data for the selected metric.

Cluster	Select a cluster from the drop-down menu.
Metric	Select a metric from the drop-down menu.
	Delta Bytes Sent	Total number of bytes sent by the node since the last data sample.
	Delta Bytes Rcvd	Total number of bytes received by the node since the last data sample.
	Delta Messages Sent	Total number of messages sent by the node since the last data sample.
	Delta Messages Rcvd	Total number of messages received by the node since the last data sample.
	Bytes Sent per Second	Total bytes sent, per second, by the node.
	Bytes per Second	Total bytes received, per second, by the node.
	Msgs Sent per Second	Total messages sent, per second, by the node.
	Msgs Rcvd per Second	Total messages received, per second, by the node.
Range	Select a time range from the drop down menu varying from 2 Minutes to Last 7 Days, or display All Data. To specify a time range, click Calendar . By default, the time range end point is the current time. To change the time range end point, click Calendar and select a date and time from the calendar or enter the date and time in the text field using the following format: MMM dd, YYYY HH:MM. For example, Aug 21, 2011 12:24 PM. Use the navigation arrows to move forward or backward one time period. NOTE: The time period is determined by your selection from the Time Range drop-down menu. Click Restore to Now to reset the time range end point to the current time.
Enable MouseOver	Select this option to make details visible upon mouseover.
Log Scale	Enable to use a logarithmic scale for the Y axis. Use Log Scale to see usage correlations for data with a wide range of values. For example, if a minority of your data is on a scale of tens, and a majority of your data is on a scale of thousands, the minority of your data is typically not visible in non-log scale graphs. Log Scale makes data on both scales visible by applying logarithmic values rather than actual values to the data.
Proxy Nodes Heatmap	Use the heatmap to view utilization trends for all Process and Storage nodes, over time, and quickly identify heavy usage, indicated by a dark color (by default, dark green). Look for a consistently dark horizontal line, which typically indicates constant high utilization. If this level of utilization is unexpected, consider performing a lower level analysis by viewing node details in the Single Node - Summary display. Use the mouseover tool-tip to see the node hostname and data for the selected metric.