This template assesses the status and overall performance of a Hadoop installation using Java Management Extensions (JMX) technology.
If the node to be monitored is remote relative to Server & Application Monitor installation, then the high ports must be open on this node as connection is negotiated on a random high port number every time polling is performed.
Elevated privileges account credentials must be provided for the JMX monitor.
Note: Many monitors are disabled by default.
Component Monitors:
Metrics system: Total dropped publishes
Total number of dropped publishes.
Hadoop::NameNode::MetricsSystem::Stats::DroppedPubAll
DroppedPubAll
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Active sinks
Current number of active sinks.
Hadoop::NameNode::MetricsSystem::Stats::NumActiveSinks
NumActiveSinks
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Active sources
Current number of active metrics sources, not including Metrics system metrics.
Hadoop::NameNode::MetricsSystem::Stats::NumActiveSources
NumActiveSources
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Total sinks
Total number of sinks.
Hadoop::NameNode::MetricsSystem::Stats::NumAllSinks
NumAllSinks
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Total metrics sources
Total number of metrics sources, not including Metrics system metrics.
Hadoop::NameNode::MetricsSystem::Stats::NumAllSources
NumAllSources
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Average stats' publication time [ms]
Average time in milliseconds to publish stats to a sink.
Hadoop::NameNode::MetricsSystem::Stats::PublishAvgTime
PublishAvgTime
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Total operations for stats' publication
Total number of operations to publish stats to a sink.
Hadoop::NameNode::MetricsSystem::Stats::PublishNumOps
PublishNumOps
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Average stats' snapshot time [ms]
Average time in milliseconds to snapshot stats from a metrics source, including Metrics system.
Hadoop::NameNode::MetricsSystem::Stats::SnapshotAvgTime
SnapshotAvgTime
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
Metrics system: Total operations for stats' snapshot
Total number of operations to snapshot stats from a metrics source, including Metrics system.
Hadoop::NameNode::MetricsSystem::Stats::SnapshotNumOps
SnapshotNumOps
Hadoop:service=NameNode,name=MetricsSystem,sub=Stats
FS Namesystem: Block capacity
Current number of block capacity.
Hadoop::NameNode::FSNamesystem::BlockCapacity
BlockCapacity
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Allocated blocks
Current number of allocated blocks in the system.
Hadoop::NameNode::FSNamesystem::BlocksTotal
BlocksTotal
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Remaining capacity [B]
Current remaining capacity in bytes.
Hadoop::NameNode::FSNamesystem::CapacityRemaining
CapacityRemaining
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Remaining capacity [GB]
Current remaining capacity in GB.
Hadoop::NameNode::FSNamesystem::CapacityRemainingGB
CapacityRemainingGB
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Total DataNodes' capacity [B]
Current raw capacity of DataNodes in bytes.
Hadoop::NameNode::FSNamesystem::CapacityTotal
CapacityTotal
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Total DataNodes' capacity [GB]
Current raw capacity of DataNodes in GB.
Hadoop::NameNode::FSNamesystem::CapacityTotalGB
CapacityTotalGB
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Used DataNodes' capacity [B]
Current used capacity across all DataNodes in bytes.
Hadoop::NameNode::FSNamesystem::CapacityUsed
CapacityUsed
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Used DataNodes' capacity [GB]
Current used capacity across all DataNodes in gigabytes.
Hadoop::NameNode::FSNamesystem::CapacityUsedGB
CapacityUsedGB
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Used DataNodes' space (non-DFS) [GB]
Current space used by DataNodes for non DFS purposes in gigabytes.
Hadoop::NameNode::FSNamesystem::CapacityUsedNonDFS
CapacityUsedNonDFS
Hadoop:service=NameNode,name=FSNamesystem
${Statistic}/1000/1000
FS Namesystem: Current blocks with corrupt replicas
Current number of blocks with corrupt replicas.
Hadoop::NameNode::FSNamesystem::CorruptBlocks
CorruptBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current excess blocks
Current number of excess blocks.
Hadoop::NameNode::FSNamesystem::ExcessBlocks
ExcessBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Total expired heartbeats
Total number of expired heartbeats.
Hadoop::NameNode::FSNamesystem::ExpiredHeartbeats
ExpiredHeartbeats
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current files and directories
Current number of files and directories.
Hadoop::NameNode::FSNamesystem::FilesTotal
FilesTotal
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current files and directories (equiv)
Current number of files and directories (same as FilesTotal).
Hadoop::NameNode::FSNamesystem::TotalFiles
TotalFiles
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Time since last checkpoint [ms]
Time in milliseconds since epoch of last checkpoint.
Hadoop::NameNode::FSNamesystem::LastCheckpointTime
LastCheckpointTime
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Last written transaction ID
Last transaction ID written to the edit log.
Hadoop::NameNode::FSNamesystem::LastWrittenTransactionId
LastWrittenTransactionId
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Time since last edit log load [ms]
Time in milliseconds since the last time standby NameNode load edit log. In active NameNode, set to 0.
This counter is only for DFS High Availability.
MillisSinceLastLoadedEdits
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current missing blocks
Current number of missing blocks.
Hadoop::NameNode::FSNamesystem::MissingBlocks
MissingBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current missing replication-1 blocks
Current number of missing blocks with replication factor 1.
Hadoop::NameNode::FSNamesystem::MissingReplOneBlocks
MissingReplOneBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current pending block-related messages
Current number of pending block-related messages for later processing in the standby NameNode. This counter is only for DFS High Availability.
Hadoop::NameNode::FSNamesystem::PendingDataNodeMessageCount
PendingDataNodeMessageCount
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current pending deletion blocks
Current number of blocks pending deletion.
Hadoop::NameNode::FSNamesystem::PendingDeletionBlocks
PendingDeletionBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current pending replication blocks
Current number of blocks pending to be replicated.
Hadoop::NameNode::FSNamesystem::PendingReplicationBlocks
PendingReplicationBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current postponed replication blocks
Current number of blocks postponed to replicate. This counter is only for DFS High Availability.
Hadoop::NameNode::FSNamesystem::PostponedMisreplicatedBlocks
PostponedMisreplicatedBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current scheduled for replication blocks
Current number of blocks scheduled for replications.
Hadoop::NameNode::FSNamesystem::ScheduledReplicationBlocks
ScheduledReplicationBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current snapshots
Current number of snapshots.
Hadoop::NameNode::FSNamesystem::Snapshots
Snapshots
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current snapshottable directories
Current number of snapshottable directories.
Hadoop::NameNode::FSNamesystem::SnapshottableDirectories
SnapshottableDirectories
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current stale DataNodes
Current number of DataNodes marked stale due to delayed heartbeat.
Hadoop::NameNode::FSNamesystem::StaleDataNodes
StaleDataNodes
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current connections
Current number of connections.
Hadoop::NameNode::FSNamesystem::TotalLoad
TotalLoad
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Total transactions since last checkpoint
Total number of transactions since last checkpoint.
Hadoop::NameNode::FSNamesystem::TransactionsSinceLastCheckpoint
TransactionsSinceLastCheckpoint
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Total transactions since last edit log roll
Total number of transactions since last edit log roll.
Hadoop::NameNode::FSNamesystem::TransactionsSinceLastLogRoll
TransactionsSinceLastLogRoll
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem: Current under replicated blocks
Current number of blocks under replicated.
Hadoop::NameNode::FSNamesystem::UnderReplicatedBlocks
UnderReplicatedBlocks
Hadoop:service=NameNode,name=FSNamesystem
FS Namesystem state: Current files and directories
Current number of files and directories.
Hadoop::NameNode::FSNamesystemState::FilesTotal
FilesTotal
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Block deletion start time
Block deletion start time.
Hadoop::NameNode::FSNamesystemState::BlockDeletionStartTime
BlockDeletionStartTime
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Current allocated blocks
Current number of allocated blocks in the system.
Hadoop::NameNode::FSNamesystemState::BlocksTotal
BlocksTotal
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Remaining capacity [B]
Current remaining capacity in bytes.
Hadoop::NameNode::FSNamesystemState::CapacityRemaining
CapacityRemaining
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Current DataNodes' capacity [B]
Current raw capacity of DataNodes in bytes.
Hadoop::NameNode::FSNamesystemState::CapacityTotal
CapacityTotal
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Used DataNodes' capacity [B]
Current used capacity across all DataNodes in bytes.
Hadoop::NameNode::FSNamesystemState::CapacityUsed
CapacityUsed
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Estimated capacity lost [B]
Estimate of capacity lost in bytes.
Hadoop::NameNode::FSNamesystemState::EstimatedCapacityLostTotal
EstimatedCapacityLostTotal
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Maximum number of objects
Maximum number of objects.
Hadoop::NameNode::FSNamesystemState::MaxObjects
MaxObjects
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Dead DataNodes
Current number of dead DataNodes.
Hadoop::NameNode::FSNamesystemState::NumDeadDataNodes
NumDeadDataNodes
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Decommissioned dead DataNodes
Current number of decommissioned dead DataNodes.
Hadoop::NameNode::FSNamesystemState::NumDecomDeadDataNodes
NumDecomDeadDataNodes
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Decommissioned live DataNodes
Current number of decommissioned live DataNodes.
Hadoop::NameNode::FSNamesystemState::NumDecomLiveDataNodes
NumDecomLiveDataNodes
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Decommissioning DataNodes
Number of decommissioning DataNodes.
Hadoop::NameNode::FSNamesystemState::NumDecommissioningDataNodes
NumDecommissioningDataNodes
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Live DataNodes
Number of live DataNodes.
Hadoop::NameNode::FSNamesystemState::NumLiveDataNodes
NumLiveDataNodes
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Stale DataNodes
Number of stale DataNodes.
Hadoop::NameNode::FSNamesystemState::NumStaleDataNodes
NumStaleDataNodes
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Stale storages
Number of stale storages.
Hadoop::NameNode::FSNamesystemState::NumStaleStorages
NumStaleStorages
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Current pending deletion blocks
Current number of blocks pending deletion.
Hadoop::NameNode::FSNamesystemState::PendingDeletionBlocks
PendingDeletionBlocks
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Current pending replication blocks
Current number of blocks pending to be replicated.
Hadoop::NameNode::FSNamesystemState::PendingReplicationBlocks
PendingReplicationBlocks
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Current scheduled for replication blocks
Current number of blocks scheduled for replications.
Hadoop::NameNode::FSNamesystemState::ScheduledReplicationBlocks
ScheduledReplicationBlocks
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Current connections
Current number of connections.
Hadoop::NameNode::FSNamesystemState::TotalLoad
TotalLoad
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Current under replicated blocks
Current number of blocks under replicated.
Hadoop::NameNode::FSNamesystemState::UnderReplicatedBlocks
UnderReplicatedBlocks
Hadoop:service=NameNode,name=FSNamesystemState
FS Namesystem state: Total volume failures
Total number of volume failures occurred.
Hadoop::NameNode::FSNamesystemState::VolumeFailuresTotal
VolumeFailuresTotal
Hadoop:service=NameNode,name=FSNamesystemState
NameNode JVM metrics: Total GC count
Total GC count.
Hadoop::NameNode::JvmMetrics::GcCount
GcCount
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC count (copy)
Total GC count (copy).
Hadoop::NameNode::JvmMetrics::GcCountCopy
GcCountCopy
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC count (MarkSweep, Compact)
Total GC count with MarkSweep and Compact options.
Hadoop::NameNode::JvmMetrics::GcCountMarkSweepCompact
GcCountMarkSweepCompact
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC info thresholds' exceeds
Number of times that the GC info threshold is exceeded.
Hadoop::NameNode::JvmMetrics::GcNumInfoThresholdExceeded
GcNumInfoThresholdExceeded
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC warn thresholds' exceeds
Number of times that the GC warn threshold is exceeded.
Hadoop::NameNode::JvmMetrics::GcNumWarnThresholdExceeded
GcNumWarnThresholdExceeded
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC time [ms]
Total GC time in milliseconds.
Hadoop::NameNode::JvmMetrics:: GcTimeMillis
GcTimeMillis
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC time (copy) [ms]
Total GC time in milliseconds (copy).
Hadoop::NameNode::JvmMetrics:: GcTimeMillisCopy
GcTimeMillisCopy
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC time (MarkSweep, Compact) [ms]
Total GC time in milliseconds with MarkSweep and Compact options.
Hadoop::NameNode::JvmMetrics:: GcTimeMillisMarkSweepCompact
GcTimeMillisMarkSweepCompact
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total GC extra sleep time [ms]
Total GC extra sleep time in milliseconds.
Hadoop::NameNode::JvmMetrics::GcTotalExtraSleepTime
GcTotalExtraSleepTime
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total ERROR logs
Total number of ERROR logs.
Hadoop::NameNode::JvmMetrics::LogError
LogError
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total FATAL logs
Total number of FATAL logs.
Hadoop::NameNode::JvmMetrics::LogFatal
LogFatal
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total INFO logs
Total number of INFO logs.
Hadoop::NameNode::JvmMetrics::LogInfo
LogInfo
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Total WARN logs
Total number of WARN logs.
Hadoop::NameNode::JvmMetrics::LogWarn
LogWarn
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current heap memory committed [MB]
Current heap memory committed in megabytes.
Hadoop::NameNode::JvmMetrics::MemHeapCommittedM
MemHeapCommittedM
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Max heap memory size [MB]
Max heap memory size in megabytes.
Hadoop::NameNode::JvmMetrics::MemHeapMaxM
MemHeapMaxM
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current heap memory used [MB]
Current heap memory used in megabytes.
Hadoop::NameNode::JvmMetrics:: MemHeapUsedM
MemHeapUsedM
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Max memory size [MB]
Max memory size in megabytes.
Hadoop::NameNode::JvmMetrics::MemMaxM
MemMaxM
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current non-heap memory committed [MB]
Current non-heap memory committed in megabytes.
Hadoop::NameNode::JvmMetrics::MemNonHeapCommittedM
MemNonHeapCommittedM
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Max non-heap memory size [MB]
Max non-heap memory size in megabytes.
Hadoop::NameNode::JvmMetrics::MemNonHeapMaxM
MemNonHeapMaxM
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current non-heap memory used [MB]
Current non-heap memory used in megabytes.
Hadoop::NameNode::JvmMetrics::MemNonHeapUsedM
MemNonHeapUsedM
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current BLOCKED threads
Current number of BLOCKED threads.
Hadoop::NameNode::JvmMetrics::ThreadsBlocked
ThreadsBlocked
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current NEW threads
Current number of NEW threads.
Hadoop::NameNode::JvmMetrics::ThreadsNew
ThreadsNew
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current RUNNABLE threads
Current number of RUNNABLE threads.
Hadoop::NameNode::JvmMetrics::ThreadsRunnable
ThreadsRunnable
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current TERMINATED threads
Current number of TERMINATED threads.
Hadoop::NameNode::JvmMetrics::ThreadsTerminated
ThreadsTerminated
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current TIMED_WAITING threads
Current number of TIMED_WAITING threads.
Hadoop::NameNode::JvmMetrics::ThreadsTimedWaiting
ThreadsTimedWaiting
Hadoop:service=NameNode,name=JvmMetrics
NameNode JVM metrics: Current WAITING threads
Current number of WAITING threads.
Hadoop::NameNode::JvmMetrics::ThreadsWaiting
ThreadsWaiting
Hadoop:service=NameNode,name=JvmMetrics
NameNode activity: Successful addBlock operations
Total number of addBlock operations succeeded.
Hadoop::NameNode::NameNodeActivity::AddBlockOps
AddBlockOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Successful allowSnapshot operations
Total number of allowSnapshot operations.
Hadoop::NameNode::NameNodeActivity::AllowSnapshotOps
AllowSnapshotOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Successful blockReceivedAndDeleted operations
Total number of blockReceivedAndDeleted operations.
Hadoop::NameNode::NameNodeActivity::BlockReceivedAndDeletedOps
BlockReceivedAndDeletedOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Average blockReport time [ms]
Average time of processing block reports in milliseconds.
Hadoop::NameNode::NameNodeActivity::BlockReportAvgTime
BlockReportAvgTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total blockReport calls
Total number of processing block reports from DataNode.
Hadoop::NameNode::NameNodeActivity::BlockReportNumOps
BlockReportNumOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Average processing cache reports time [ms]
Average time of processing cache reports in milliseconds.
Hadoop::NameNode::NameNodeActivity::CacheReportAvgTime
CacheReportAvgTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total processing cache reports
Total number of processing cache reports from DataNode.
Hadoop::NameNode::NameNodeActivity::CacheReportNumOps
CacheReportNumOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total files created
Total number of files created.
Hadoop::NameNode::NameNodeActivity::CreateFileOps
CreateFileOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total createSnapshot operations
Total number of createSnapshot operations.
Hadoop::NameNode::NameNodeActivity::CreateSnapshotOps
CreateSnapshotOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total createSymlink operations
Total number of createSymlink operations.
Hadoop::NameNode::NameNodeActivity::CreateSymlinkOps
CreateSymlinkOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total delete operations
Total number of delete operations.
Hadoop::NameNode::NameNodeActivity::DeleteFileOps
DeleteFileOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total deleteSnapshot operations
Total number of deleteSnapshot operations.
Hadoop::NameNode::NameNodeActivity::DeleteSnapshotOps
DeleteSnapshotOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total disallowSnapshot operations
Total number of disallowSnapshot operations.
Hadoop::NameNode::NameNodeActivity::DisallowSnapshotOps
DisallowSnapshotOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total getFileInfo and getLinkFileInfo operations
Total number of getFileInfo and getLinkFileInfo operations.
Hadoop::NameNode::NameNodeActivity::FileInfoOps
FileInfoOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total files appended
Total number of files appended.
Hadoop::NameNode::NameNodeActivity::FilesAppended
FilesAppended
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total files and directories created
Total number of files and directories created by create or mkdir operations.
Hadoop::NameNode::NameNodeActivity::FilesCreated
FilesCreated
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total files and directories deleted
Total number of files and directories deleted by delete or rename operations.
Hadoop::NameNode::NameNodeActivity::FilesDeleted
FilesDeleted
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total files and directories listed
Total number of files and directories listed by directory listing operations.
Hadoop::NameNode::NameNodeActivity::FilesInGetListingOps
FilesInGetListingOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total rename operations
Total number of rename operations. This is not the number of files/dirs renamed.
Hadoop::NameNode::NameNodeActivity::FilesRenamed
FilesRenamed
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total truncate operations
Total number of truncate operations.
Hadoop::NameNode::NameNodeActivity::FilesTruncated
FilesTruncated
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Time loading FS Image at startup [ms]
Time loading FS Image at startup in milliseconds.
Hadoop::NameNode::NameNodeActivity::FsImageLoadTime
FsImageLoadTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total getAdditionalDatanode operations
Total number of getAdditionalDatanode operations.
Hadoop::NameNode::NameNodeActivity::GetAdditionalDatanodeOps
GetAdditionalDatanodeOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total getBlockLocations operations
Total number of getBlockLocations operations.
Hadoop::NameNode::NameNodeActivity::GetBlockLocations
GetBlockLocations
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Average edits download time [ms]
Average edits download time in milliseconds.
Hadoop::NameNode::NameNodeActivity::GetEditAvgTime
GetEditAvgTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total edits downloads
Total number of edits downloads from SecondaryNameNode.
Hadoop::NameNode::NameNodeActivity::GetEditNumOps
GetEditNumOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Average FS Image download time [ms]
Average FS Image download time in milliseconds.
Hadoop::NameNode::NameNodeActivity::GetImageAvgTime
GetImageAvgTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total FS Image downloads
Total number of FS Image downloads from SecondaryNameNode.
Hadoop::NameNode::NameNodeActivity::GetImageNumOps
GetImageNumOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total getLinkTarget operations
Total number of getLinkTarget operations.
Hadoop::NameNode::NameNodeActivity::GetLinkTargetOps
GetLinkTargetOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total directory listing operations
Total number of directory listing operations.
Hadoop::NameNode::NameNodeActivity::GetListingOps
GetListingOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total snapshottableDirectoryStatus operations
Total number of snapshottableDirectoryStatus operations.
Hadoop::NameNode::NameNodeActivity::ListSnapshottableDirOps
ListSnapshottableDirOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Average FS Image upload time [ms]
Average FS Image upload time in milliseconds.
Hadoop::NameNode::NameNodeActivity::PutImageAvgTime
PutImageAvgTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total number of FS Image uploads
Total number of FS Image uploads to SecondaryNameNode.
Hadoop::NameNode::NameNodeActivity::PutImageNumOps
PutImageNumOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total renameSnapshot operations
Total number of renameSnapshot operations.
Hadoop::NameNode::NameNodeActivity::RenameSnapshotOps
RenameSnapshotOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Safe mode time [ms]
The interval between FSNameSystem starts and the last time safemode leaves in milliseconds. Sometimes not equal to the time in SafeMode, see HDFS-5156.
Hadoop::NameNode::NameNodeActivity::SafeModeTime
SafeModeTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total getSnapshotDiffReport operations
Total number of getSnapshotDiffReport operations.
Hadoop::NameNode::NameNodeActivity::SnapshotDiffReportOps
SnapshotDiffReportOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total storageBlockReport operations
Total number of storageBlockReport operations.
Hadoop::NameNode::NameNodeActivity::StorageBlockReportOps
StorageBlockReportOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Average Journal syncs time [ms]
Average time of Journal syncs in milliseconds.
Hadoop::NameNode::NameNodeActivity::SyncsAvgTime
SyncsAvgTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total syncs
Number of sync operations.
Hadoop::NameNode::NameNodeActivity::SyncNumOps
SyncsNumOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total file operations
Total number of file operations performed.
Hadoop::NameNode::NameNodeActivity::TotalFileOps
TotalFileOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Average Journal transactions time [ms]
Average time of Journal transactions in milliseconds.
Hadoop::NameNode::NameNodeActivity::TransactionsAvgTime
TransactionsAvgTime
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total Journal transactions batched in sync
Total number of Journal transactions batched in sync.
Hadoop::NameNode::NameNodeActivity::TransactionsBatchedInSync
TransactionsBatchedInSync
Hadoop:service=NameNode,name=NameNodeActivity
NameNode activity: Total Journal transactions
Total number of Journal transactions.
Hadoop::NameNode::NameNodeActivity::TransactionsNumOps
TransactionsNumOps
Hadoop:service=NameNode,name=NameNodeActivity
NameNode RPC activity for port 9000: Length of the call queue
Current length of the call queue.
Hadoop::NameNode::RpcActivityForPort9000::CallQueueLength
CallQueueLength
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Open connections
Current number of open connections.
Hadoop::NameNode::RpcActivityForPort9000::NumOpenConnections
NumOpenConnections
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total received bytes
Total number of received bytes.
Hadoop::NameNode::RpcActivityForPort9000::ReceivedBytes
ReceivedBytes
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total authentication failures
Total number of authentication failures.
Hadoop::NameNode::RpcActivityForPort9000::RpcAuthenticationFailures
RpcAuthenticationFailures
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total authentication successes
Total number of authentication successes.
Hadoop::NameNode::RpcActivityForPort9000::RpcAuthenticationSuccesses
RpcAuthenticationSuccesses
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total authorization failures
Total number of authorization failures.
Hadoop::NameNode::RpcActivityForPort9000::RpcAuthorizationFailures
RpcAuthorizationFailures
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total authorization successes
Total number of authorization successes.
Hadoop::NameNode::RpcActivityForPort9000::RpcAuthorizationSuccesses
RpcAuthorizationSuccesses
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Average processing time [ms]
Average processing time in milliseconds.
Hadoop::NameNode::RpcActivityForPort9000::RpcProcessingTimeAvgTime
RpcProcessingTimeAvgTime
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total RPC calls
Total number of RPC calls (same as RpcQueueTimeNumOps).
Hadoop::NameNode::RpcActivityForPort9000::RpcProcessingTimeNumOps
RpcProcessingTimeNumOps
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Average queue time [ms]
Average queue time in milliseconds.
Hadoop::NameNode::RpcActivityForPort9000::RpcQueueTimeAvgTime
RpcQueueTimeAvgTime
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total RPC calls
Total number of RPC calls (same as RpcProcessingTimeNumOps).
Hadoop::NameNode::RpcActivityForPort9000::RpcQueueTimeNumOps
RpcQueueTimeNumOps
Hadoop:service=NameNode,name=RpcActivityForPort9000
NameNode RPC activity for port 9000: Total sent bytes
Total number of sent bytes.
Hadoop::NameNode::RpcActivityForPort9000::SentBytes
SentBytes
Hadoop:service=NameNode,name=RpcActivityForPort9000
RetryCache/NameNodeRetryCache: Total RetryCache cleared
Total number of RetryCache cleared.
Hadoop::NameNode::RetryCache.NameNodeRetryCache::CacheCleared
CacheCleared
Hadoop:service=NameNode,name=RetryCache.NameNodeRetryCache
RetryCache/NameNodeRetryCache: Total RetryCache hit
Total number of RetryCache hit.
Hadoop::NameNode::RetryCache.NameNodeRetryCache::CacheHit
CacheHit
Hadoop:service=NameNode,name=RetryCache.NameNodeRetryCache
RetryCache/NameNodeRetryCache: Total RetryCache updated
Total number of RetryCache updated.
Hadoop::NameNode::RetryCache.NameNodeRetryCache::CacheUpdated
CacheUpdated
Hadoop:service=NameNode,name=RetryCache.NameNodeRetryCache
RPC detailed: Average blockReport time [ms]
Average turnaround time of blockReport method in milliseconds.
Hadoop::NameNode::RpcDetailedActivityForPort9000::BlockReportAvgTime
BlockReportAvgTime
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Total blockReport method calls
Total number of the times blockReport method is called.
Hadoop::NameNode::RpcDetailedActivityForPort9000::BlockReportNumOps
BlockReportNumOps
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Average getEditLogManifest time [ms]
Average turnaround time of getEditLogManifest method in milliseconds.
Hadoop::NameNode::RpcDetailedActivityForPort9000::GetEditLogManifestAvgTime
GetEditLogManifestAvgTime
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Total getEditLogManifest method calls
Total number of the times the getEditLogManifest method is called.
Hadoop::NameNode::RpcDetailedActivityForPort9000::GetEditLogManifestNumOps
GetEditLogManifestNumOps
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Average getTransactionId time [ms]
Average turnaround time of getTransactionId method in milliseconds.
Hadoop::NameNode::RpcDetailedActivityForPort9000::GetTransactionIdAvgTime
GetTransactionIdAvgTime
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Total getTransactionId method calls
Total number of the times the getTransactionId method is called.
Hadoop::NameNode::RpcDetailedActivityForPort9000::GetTransactionIdNumOps
GetTransactionIdNumOps
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Average registerDatanode time [ms]
Average turnaround time of the registerDatanode method in milliseconds.
Hadoop::NameNode::RpcDetailedActivityForPort9000::RegisterDatanodeAvgTime
RegisterDatanodeAvgTime
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Total registerDatanode method calls
Total number of the times the registerDatanode method is called.
Hadoop::NameNode::RpcDetailedActivityForPort9000::RegisterDatanodeNumOps
RegisterDatanodeNumOps
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Average rollEditLog time [ms]
Average turnaround time of the rollEditLog method in milliseconds.
Hadoop::NameNode::RpcDetailedActivityForPort9000::RollEditLogAvgTime
RollEditLogAvgTime
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Total rollEditLog method calls
Total number of the times the rollEditLog method is called.
Hadoop::NameNode::RpcDetailedActivityForPort9000::RollEditLogNumOps
RollEditLogNumOps
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Average sendHeartbeat time [ms]
Average turnaround time of sendHeartbeat method in milliseconds.
Hadoop::NameNode::RpcDetailedActivityForPort9000::SendHeartbeatAvgTime
SendHeartbeatAvgTime
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Total sendHeartbeat method calls
Total number of the times sendHeartbeat method is called.
Hadoop::NameNode::RpcDetailedActivityForPort9000::SendHeartbeatNumOps
SendHeartbeatNumOps
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Average versionRequest time [ms]
Average turnaround time of versionRequest method in milliseconds.
Hadoop::NameNode::RpcDetailedActivityForPort9000::VersionRequestAvgTime
VersionRequestAvgTime
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
RPC detailed: Total versionRequest method calls
Total number of the times the versionRequest method is called.
Hadoop::NameNode::RpcDetailedActivityForPort9000::VersionRequestNumOps
VersionRequestNumOps
Hadoop:service=NameNode,name=RpcDetailedActivityForPort9000
Startup progress: NameNode's startup elapsed time [ms]
Total elapsed time in milliseconds.
Hadoop::NameNode::StartupProgress::ElapsedTime
ElapsedTime
Hadoop:service=NameNode,name=StartupProgress
Startup progress: LoadingEdits completed steps
Total number of steps completed in loading edits phase.
Hadoop::NameNode::StartupProgress::LoadingEditsCount
LoadingEditsCount
Hadoop:service=NameNode,name=StartupProgress
Startup progress: LoadingEdits elapsed time [ms]
Total elapsed time in loading edits phase in milliseconds.
Hadoop::NameNode::StartupProgress::LoadingEditsElapsedTime
LoadingEditsElapsedTime
Hadoop:service=NameNode,name=StartupProgress
Startup progress: LoadingEdits completion rate
Current rate completed in loading edits phase.
The max polled value is not 100 but 1.0.
LoadingEditsPercentComplete
Hadoop:service=NameNode,name=StartupProgress
${Statistic}*100
Startup progress: LoadingEdits total steps
Total number of steps in loading edits phase.
Hadoop::NameNode::StartupProgress::LoadingEditsTotal
LoadingEditsTotal
Hadoop:service=NameNode,name=StartupProgress
Startup progress: LoadingFsImage completed steps
Total number of steps completed in loading FS image phase.
Hadoop::NameNode::StartupProgress::LoadingFsImageCount
LoadingFsImageCount
Hadoop:service=NameNode,name=StartupProgress
Startup progress: LoadingFsImage elapsed time [ms]
Total elapsed time in loading FS image phase in milliseconds.
Hadoop::NameNode::StartupProgress::LoadingFsImageElapsedTime
LoadingFsImageElapsedTime
Hadoop:service=NameNode,name=StartupProgress
Startup progress: LoadingFsImage completion rate
Current rate completed in loading FS image phase.
The max polled value is not 100 but 1.0.
LoadingFsImagePercentComplete
Hadoop:service=NameNode,name=StartupProgress
${Statistic}*100
Startup progress: LoadingFsImage total steps
Total number of steps in loading FS image phase.
Hadoop::NameNode::StartupProgress::LoadingFsImageTotal
LoadingFsImageTotal
Hadoop:service=NameNode,name=StartupProgress
Startup progress: NameNode's startup completion rate
Current rate completed in NameNode startup progress.
The max polled value is not 100 but 1.0.
Hadoop::NameNode::StartupProgress::PercentComplete
PercentComplete
Hadoop:service=NameNode,name=StartupProgress
${Statistic}*100
Startup progress: SafeMode completed steps
Total number of steps completed in safe mode phase.
Hadoop::NameNode::StartupProgress::SafeModeCount
SafeModeCount
Hadoop:service=NameNode,name=StartupProgress
Startup progress: SafeMode elapsed time [ms]
Total elapsed time in safe mode phase in milliseconds.
Hadoop::NameNode::StartupProgress::SafeModeElapsedTime
SafeModeElapsedTime
Hadoop:service=NameNode,name=StartupProgress
Startup progress: SafeMode completion rate
Current rate completed in safe mode phase.
The max polled value is not 100 but 1.0.
Hadoop::NameNode::StartupProgress::SafeModePercentComplete
SafeModePercentComplete
Hadoop:service=NameNode,name=StartupProgress
${Statistic}*100
Startup progress: SafeMode total steps
Total number of steps in safe mode phase.
Hadoop::NameNode::StartupProgress::SafeModeTotal
SafeModeTotal
Hadoop:service=NameNode,name=StartupProgress
Startup progress: SavingCheckpoint completed steps
Total number of steps completed in saving checkpoint phase.
Hadoop::NameNode::StartupProgress::SavingCheckpointCount
SavingCheckpointCount
Hadoop:service=NameNode,name=StartupProgress
Startup progress: SavingCheckpoint elapsed time [ms]
Total elapsed time in saving checkpoint phase in milliseconds.
Hadoop::NameNode::StartupProgress::SavingCheckpointElapsedTime
SavingCheckpointElapsedTime
Hadoop:service=NameNode,name=StartupProgress
Startup progress: SavingCheckpoint completion rate
Current rate completed in saving checkpoint phase.
The max polled value is not 100 but 1.0.
Hadoop::NameNode::StartupProgress:: SavingCheckpointPercentComplete
SavingCheckpointPercentComplete
Hadoop:service=NameNode,name=StartupProgress
${Statistic}*100
Startup progress: SavingCheckpoint total steps
Total number of steps in saving checkpoint phase.
Hadoop::NameNode::StartupProgress::SavingCheckpointTotal
SavingCheckpointTotal
Hadoop:service=NameNode,name=StartupProgress
User and group information: Average group resolution time [ms]
Average time for group resolution in milliseconds.
Hadoop::NameNode::UgiMetrics::GetGroupsAvgTime
GetGroupsAvgTime
Hadoop:service=NameNode,name=UgiMetrics
User and group information: Total group resolutions
Total number of group resolutions (num seconds granularity). num is specified by hadoop.user.group.metrics.percentiles.intervals.
Hadoop::NameNode::UgiMetrics::GetGroupsNumOps
GetGroupsNumOps
Hadoop:service=NameNode,name=UgiMetrics
User and group information: Average failed Kerberos login time [ms]
Average time for failed kerberos logins in milliseconds.
Hadoop::NameNode::UgiMetrics::LoginFailureAvgTime
LoginFailureAvgTime
Hadoop:service=NameNode,name=UgiMetrics
User and group information: Total failed Kerberos logins
Total number of failed kerberos logins.
Hadoop::NameNode::UgiMetrics::LoginFailureNumOps
LoginFailureNumOps
Hadoop:service=NameNode,name=UgiMetrics
User and group information: Average successful Kerberos login time [ms]
Average time for successful kerberos logins in milliseconds.
Hadoop::NameNode::UgiMetrics::LoginSuccessAvgTime
LoginSuccessAvgTime
Hadoop:service=NameNode,name=UgiMetrics
User and group information: Total successful Kerberos logins
Total number of successful kerberos logins.
Hadoop::NameNode::UgiMetrics::LoginSuccessNumOps
LoginSuccessNumOps
Hadoop:service=NameNode,name=UgiMetrics
Queue metrics: Active applications
Current number of active applications.
ActiveApplications
Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Queue metrics: Total failed applications
Total number of failed applications.
Hadoop::ResourceManager::QueueMetrics::root::AppsFailed
AppsFailed
Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Queue metrics: Total killed applications
Total number of killed applications.
Hadoop::ResourceManager::QueueMetrics::root::AppsKilled
AppsKilled
Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Queue metrics: Pending applications
Current number of applications that have not yet been assigned by any containers.
Hadoop::ResourceManager::QueueMetrics::root::AppsPending
AppsPending
Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Queue metrics: Running applications
Current number of running applications.
Hadoop::ResourceManager::QueueMetrics::root::AppsRunning
AppsRunning
Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Queue metrics: Total submitted applications
Total number of submitted applications.
Hadoop::ResourceManager::QueueMetrics::root::AppsSubmitted
AppsSubmitted
Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Queue metrics: Total completed applications
Total number of completed applications.
Hadoop::ResourceManager::QueueMetrics::root::AppsCompleted
AppsCompleted
Hadoop:service=ResourceManager,name=QueueMetrics,q0=root
Cluster metrics: Active NodeManagers
Current number of active NodeManagers.
Hadoop::ResourceManager::ClusterMetrics::NumActiveNMs
NumActiveNMs
Hadoop:service=ResourceManager,name=ClusterMetrics
Cluster metrics: Lost NodeManagers
Current number of lost NodeManagers (not sending heartbeats).
Hadoop::ResourceManager::ClusterMetrics::NumLostNMs
NumLostNMs
Hadoop:service=ResourceManager,name=ClusterMetrics
Cluster metrics: Decommissioned NodeManagers
Current number of decommissioned NodeManagers.
ResourceManager::ClusterMetrics::NumDecommissionedNMs
NumDecommissionedNMs
Hadoop:service=ResourceManager,name=ClusterMetrics
Cluster metrics: Rebooted NodeManagers
Current number of rebooted NodeManagers.
Hadoop::ResourceManager::ClusterMetrics::NumRebootedNMs
NumRebootedNMs
Hadoop:service=ResourceManager,name=ClusterMetrics
Cluster metrics: Unhealthy NodeManagers
Current number of unhealthy NodeManagers.
Hadoop::ResourceManager::ClusterMetrics::NumUnhealthyNMs
NumUnhealthyNMs
Hadoop:service=ResourceManager,name=ClusterMetrics
Safemode status
Reports whether safemode is turned on.
Hadoop NameNode process
Reports whether the Hadoop's NameNode process is running.
Hadoop Secondary NameNode process
Reports whether the Hadoop's Secondary NameNode process is running.
Hadoop DataNode process
Reports whether the Hadoop's DataNode process is running.
Hadoop port monitor
Reports whether the Hadoop is listening on specified port.
Portions of this document were compiled from the information found at https://hadoop.apache.org/docs/
Last updated: 2/12/2016.