Microsoft BizTalk Server 2010 Host Throttling Performance Counters

Version 1

    BizTalk Server 2010 Host Throttling Performance Counters

    This template assesses the status and overall of performance of host throttling of default Microsoft BizTalk Server 2010 installation.

    Prerequisites: RPC access to the target server.

    Credentials: Windows Administrator on the target server.

    For more information about BizTalk, see:
    http://msdn.microsoft.com/en-us/library/aa559591(v=bts.70).aspx;
    http://msdn.microsoft.com/en-us/library/aa559893(v=bts.20).aspx.

    Note: For monitoring a SharePoint server, you can also use “Internet Information Service (IIS) Services and Counters”, “Windows Server 2003-2008 Services and Counters” and “SQL Server 2005-2008 Performance” templates.

    Monitored Components

    Note: The BizTalk:Message Agent performance counters are provided for the explicit purpose of analyzing the throttling behavior of a host and therefore will not capture data unless the specified host is actively processing documents. This behavior is by design to prevent consuming system threads with performance monitor when throttling activities are not occurring.

    Note: BizTalk Server uses in-build host throttling thresholds. Information how to change default Host Throttling settings and thresholds can be found here:
    http://msdn.microsoft.com/en-us/library/aa559628(v=bts.20).aspx.

    Active instance count

    This component monitor returns number of service instances active in memory. For the orchestration engine, a service instance refers to each running instance of an orchestration schedule. For the End Point Manager, a service instance may either correspond to a single stateless message, or to a collection of stateful messages.

    Note: Stateful instances are those that maintain certain state information about the messages associated with the instance. Messages belonging to a stateful instance are co-related in some form or the other. For example an ordered send port that maintains information about the ordering are considered stateful instances. Most messaging scenarios involve stateless instances where messages are processed completely independent of each other. Each such stateless instance corresponds to a single message within the EPM.

    Database size

    This component monitor returns number of messages in the database queues that this process has published. This value is measured by the number of items in the queue tables for all hosts and the number of items in the spool and tracking tables. If a process is publishing to multiple queues, this counter reflects the weighted average of all the queues.

    Note: If the host is restarted, statistics held in memory are lost. Since there is some overhead involved, BizTalk Server will resume gathering statistics only when there are at least 100 publishes with 5% of the total publishes within the restarted host process.

    Database session

    This component monitor returns number of concurrent MessageBox database connections being used.

    High database session

    This component monitor returns a flag indicating whether the number of currently opened database sessions exceeds the threshold.

    Possible values:
    0 – Normal
    1 – Database session count exceeds threshold.

    Threshold value is controlled by the “Database connection per CPU” value in the BizTalk Host Throttling settings. “Database connection per CPU” is the maximum number of concurrent database sessions (per CPU) allowed before throttling begins.

    High database size

    This component monitor returns a flag indicating whether the destination queue depth of all message box databases exceeds the threshold. If the host publishes to multiple queues, the weighted average across all destination queues is computed (based on recent publish statistics) and compared with the threshold.

    Possible values:
    0 – Normal;
    1 – Database size has grown beyond threshold.

    By default the host message count in database throttling threshold is set to a value of 50,000, which will trigger a throttling condition under the following circumstances:

    -       The total number of messages published by the host instance to the work, state, and suspended queues of the subscribing hosts exceeds 50,000.

    -       The number of messages in the spool table or the tracking table exceeds 500,000 messages.

    If this occurs, then consider a course of action that will reduce the number of messages in the database. For example, ensure the SQL Server jobs in BizTalk Server are running without error and use the Group Hub page in the BizTalk Server Administration console to determine whether message build up is caused by large numbers of suspended messages.

    High in-process message count

    This component monitor returns a flag indicating whether the number of in-process messages exceeds the threshold.

    Possible values:
    0 – Normal;
    1 – In-process message count exceeds limit.

    If throttling is occurring, consider adjusting the “In-Process messages per CPU” setting. This parameter only affects outbound message throttling. Enter a value of 0 in the “In-Process messages per CPU” setting to disable throttling based on the number of in-process messages per CPU. The default value for the “In-Process messages per CPU” setting is 1,000. Note that modifying this value can also have an impact on low latency of messages and/or the efficiency of BizTalk resources.

    High message delivery rate

    This component monitor returns a flag indicating whether the message delivery rate is higher than the message processing rate.

    Possible values:
    0 – Normal;
    1 – Message delivery rate exceeds the message processing rate.

    High message delivery rates can be caused by high processing complexity, slow outbound adapters, or a momentary shortage of system resources.

    High message publishing rate

    This component monitor returns a flag indicating whether the message publishing request rate is higher than the message publishing completion rate.

    Possible values:
    0 – Normal;
    1 – Publishing request rate exceeds completion rate.

    If this occurs, then the database cannot keep up with the publishing rate of messages to the BizTalk MessageBox database.

    High process memory

    This component monitor returns a flag indicating whether the process memory consumption exceeds the threshold.

    Possible values:
    0 – Normal;
    1 – Process memory exceeds threshold.

    The BizTalk Process Memory usage throttling threshold setting is the percentage of memory used compared to the sum of the working set size and total available virtual memory for the process if a value from 1 through 100 is entered. When a percentage value is specified the process memory threshold is recalculated at regular intervals.

    High system memory

    This component monitor returns a flag indicating whether the system-wide physical memory consumption exceeds the threshold.

    Possible values:
    0 – Normal;
    1 – System memory exceeds threshold.

    The BizTalk Physical Memory usage throttling threshold setting is the percentage of memory consumption compared to the total amount of available physical memory if a value from 1 through 100 is entered. This setting can also be the total amount of available physical memory in megabytes if a value greater than 100 is entered. The default value is 0.

    High thread count

    This component monitor returns a flag indicating whether the thread count exceeds the threshold.

    Possible values:

    0 – Normal;
    1 – Thread count exceeds threshold.

    “Threads Per CPU” is the total number of threads in the host process including threads used by adapters. If this threshold is exceeded, BizTalk Server will try to reduce the size of the EPM thread pool and message agent thread pool. Thread based throttling should be enabled in scenarios where high load can lead to the creation of a large number of threads. This parameter affects both inbound and outbound throttling. Thread based throttling is disabled by default.

    Message delivery incoming rate

    This component monitor returns number of messages per second that are being delivered to the Orchestration engine or the Messaging engine in the given sample interval.

    High message delivery rates can be caused by high processing complexity, slow outbound adapters, or a momentary shortage of system resources.

    Message delivery outgoing rate

    This component monitor returns number of messages per second that are being processed by the Orchestration engine or the Messaging engine in the given sample interval.

    High message delivery rates can be caused by high processing complexity, slow outbound adapters, or a momentary shortage of system resources.

    Message delivery throttling state

    This component monitor returns a flag indicating whether the system is throttling message delivery (affecting XLANG message processing and outbound transports).

    Possible values:
    0: Not throttling;
    1: Throttling due to imbalanced message delivery rate (input rate exceeds output rate);
    3: Throttling due to high in-process message count;
    4: Throttling due to process memory pressure;
    5: Throttling due to system memory pressure;
    9: Throttling due to high thread count;
    10: Throttling due to user override on delivery.

    Message delivery delay (ms)

    This component monitor returns the current delay in ms imposed on each message delivery batch (applicable if the message delivery is being throttled).

    In regards to throttling, a delay is applied in the publishing or processing of the message, depending on whether the message is inbound or outbound. The delay period is proportional to the severity of the throttling condition. Higher severity throttling conditions will initiate a longer throttling period than lower severity throttling conditions. This delay period is adjusted up and down within certain ranges by the throttling mechanism as conditions change. Long message delivery delays may indicate heavy throttling due to high load. This counter should be less than 5000 ms.

    Message delivery throttling state duration

    This component monitor returns seconds since the system entered this state. If the host is throttling, how long it has been throttling; if it is not throttling, how long since throttling was applied.

    Message publishing incoming rate

    This component monitor returns number of messages per second that are being sent to the database for publishing in the given sample interval.

    Message publishing outgoing rate

    This component monitor returns number of messages per second that are actually published in the database in the given sample interval.

    Message publishing throttling state

    This component monitor returns a flag indicating whether the system is throttling message publishing (affecting XLANG message processing and inbound transports).

    Possible values:
    0: Not throttling;
    2: Throttling due to imbalanced message publishing rate (input rate exceeds output rate);
    4: Throttling due to process memory pressure;
    5: Throttling due to system memory pressure;
    6: Throttling due to database growth;
    8: Throttling due to high session count;
    9: Throttling due to high thread count;
    11: Throttling due to user override on publishing.

    Message publishing delay (ms)

    This component monitor returns the current delay in ms imposed on each message publishing batch (applicable if the message publishing is being throttled and if the batch is not exempted from throttling).

    In regards to throttling, a delay is applied in the publishing or processing of the message, depending on whether the message is inbound or outbound. The delay period is proportional to the severity of the throttling condition. Higher severity throttling conditions will initiate a longer throttling period than lower severity throttling conditions. This delay period is adjusted up and down within certain ranges by the throttling mechanism as conditions change. Long message delivery delays may indicate heavy throttling due to high load. This counter should be less than 5000 ms.

    Message publishing throttling state duration

    This component monitor returns seconds since the system entered this state. If the host is throttling, how long it has been throttling; if it is not throttling, how long since throttling was applied.

    Process memory usage (MB)

    This component monitor returns the process memory consumption in MB. This is the maximum of the process’s working set size and the total space allocated for the page file for the process.

    BizTalk process memory throttling can occur if the batch to be published has steep memory requirements, or if too many threads are processing messages.

    Thread count

    This component monitor returns number of threads being used within the process.

    Portions of this document were originally created by and are excerpted from the following sources:

    Microsoft Corporation, “MSDN Library,” Copyright © 2012 Microsoft Corporation. 
    All rights reserved. Available at
    http://msdn.microsoft.com/en-us/library/aa578302%28v=bts.20%29.aspx