Cannot retrieve CDR files and list - CiscoCcmFtpClient job timeout

Question

Hi guys,

We're using Cisco call mgr and unified communication monitored by VNQM.

We have'nt got any data updates on solarwinds latetly...
If I try to search calls, nothing older than a month ago shows up

Solarwinds has the proper repository path to fetch CDR data and it seems the files are accessible:

FTP                              TEST OK

AXL CREDENTIALS      TEST OK

I've investigated the logfiles and here's what I found:

IpSlaM.ftp.job_[XXXX].log

------------------------------------------------------------------

ASSIGN SEARCH PATTERNS TO THREADS

------------------------------------------------------------------

THREAD 0 =>      cdr_CLUSTERX_01 02 03 04

cmr_CLUSTERX_01 02 03 04

with JobTimeout:00:10:00

THREAD 1 =>      cdr_OTHERCLUSTER_01 02 03

cmr_OTHERCLUSTER_01 02 03

2015-10-27 11:37:36,455 [STP SmartThreadPool Thread #0] INFO  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob -

FtpJobDescription:

PassiveMode: True, RetentionDate: 28/08/2015 11:37:30, DownloadFileCount: 1000, DownloadFileSize: 104857600, Directory: /backups/CDR/, SecureConnection: True, Port: 22, FtpServer: FTP_Server_IP, TimeoutPreventionPeriod: 00:00:30

CallManagerInfo:

NodeId: 221, CdrNamePattern: cdr_CLUSTERX_01, CmrNamePattern: cmr_CLUSTERX_01,

CallManagerInfo:

NodeId: 224, CdrNamePattern: cdr_CLUSTERX_04, CmrNamePattern: cmr_CLUSTERX_04,

CallManagerInfo:

NodeId: 222, CdrNamePattern: cdr_CLUSTERX_03, CmrNamePattern: cmr_CLUSTERX_03,

CallManagerInfo:

NodeId: 223, CdrNamePattern: cdr_CLUSTERX_02, CmrNamePattern: cmr_CLUSTERX_02,

2015-10-27 11:37:36,455 [STP SmartThreadPool Thread #1] INFO  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob -

FtpJobDescription:

PassiveMode: True, RetentionDate: 28/08/2015 11:37:31, DownloadFileCount: 1000, DownloadFileSize: 104857600, Directory: /backups/CDR, SecureConnection: True, Port: 22, FtpServer: FTP_Server_IP, TimeoutPreventionPeriod: 00:00:30

CallManagerInfo:

NodeId: 229, CdrNamePattern: cdr_OTHERCLUSTER_01, CmrNamePattern: cmr_OTHERCLUSTER_01

CallManagerInfo:

NodeId: 230, CdrNamePattern: cdr_OTHERCLUSTER_02, CmrNamePattern: cmr_OTHERCLUSTER_02

CallManagerInfo:

NodeId: 5401, CdrNamePattern: cdr_OTHERCLUSTER_03, CmrNamePattern: cmr_OTHERCLUSTER_03

------------------------------------------------------------------

GET SORTED FILE LIST FROM FTP SERVER

------------------------------------------------------------------

2015-10-27 11:37:36,455 [STP SmartThreadPool Thread #0] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - { ExecuteInternal entered

2015-10-27 11:37:36,455 [STP SmartThreadPool Thread #1] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - { ExecuteInternal entered

2015-10-27 11:37:36,626 [STP SmartThreadPool Thread #1] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Connecting

2015-10-27 11:37:36,626 [STP SmartThreadPool Thread #0] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Connecting

2015-10-27 11:37:37,080 [STP SmartThreadPool Thread #1] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Downloading files

2015-10-27 11:37:37,095 [STP SmartThreadPool Thread #1] INFO  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - CDRFTP - Getting sorted file list [Server:FTP_Server_IP], [Directory:/backups/CDR/]

2015-10-27 11:37:37,095 [STP SmartThreadPool Thread #1] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.FtpDLXIpSlaFtpSession - Getting files from server:[folder:[/backups/CDR/]

2015-10-27 11:37:40,001 [STP SmartThreadPool Thread #0] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Downloading files

2015-10-27 11:37:40,001 [STP SmartThreadPool Thread #0] INFO  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - CDRFTP - Getting sorted file list [Server:FTP_Server_IP], [Directory:/backups/CDR/]

2015-10-27 11:37:40,001 [STP SmartThreadPool Thread #0] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.FtpDLXIpSlaFtpSession - Getting files from server:[folder:[/backups/CDR/]

---------------------------------------------------------------

FAILED DUE TO JOB TIMEOUT => 10 minutes

2015-10-27 11:47:06,463 [10] WARN  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Finishing job due internal timeout...

2015-10-27 11:47:06,463 [26] WARN  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Finishing job due internal timeout...

2015-10-27 11:47:06,463 [26] INFO  SolarWinds.Orion.IpSla.Jobs.Ftp.FtpDLXIpSlaFtpSession - Disconnecting by request from outside

2015-10-27 11:47:06,463 [10] INFO  SolarWinds.Orion.IpSla.Jobs.Ftp.FtpDLXIpSlaFtpSession - Disconnecting by request from outside

2015-10-27 11:47:06,479 [26] WARN  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Job finished...

2015-10-27 11:47:06,479 [10] WARN  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpJob - CDRFTP - Job finished...

---------------------------------------------------------------

GET CDR & CMD FILES FAILED

NO MATCHING PATTERNS BTW THREAD ASSIGNED PATTERNS AND FILE

examples bellow:

------------------------------------------------------------------

2015-10-27 11:47:10,510 [STP SmartThreadPool Thread #1] DEBUG SolarWinds.Orion.IpSla.Jobs.Ftp.FtpDLXIpSlaFtpSession - Got files from server: [folder:/backups/CDR/]

2015-10-27 11:47:10,510 [STP SmartThreadPool Thread #1] INFO  SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - CDRFTP - Processing file list

2015-10-27 11:47:10,510 [STP SmartThreadPool Thread #1] VERBOSE SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - Checking candidate to download: [file:cdr_CLUSTERX_02_201503201630_355429]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201503201630_355429, pattern:cdr_OTHERCLUSTER_01]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201503201630_355429, pattern:cmr_OTHERCLUSTER_01]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201503201630_355429, pattern:cdr_OTHERCLUSTER_02]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201503201630_355429, pattern:cmr_OTHERCLUSTER_02]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201503201630_355429, pattern:cdr_OTHERCLUSTER_03]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201503201630_355429, pattern:cmr_OTHERCLUSTER_03]

2015-10-27 11:47:10,526 [STP SmartThreadPool Thread #1] VERBOSE SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - Checking candidate to download: [file:cdr_CLUSTERX_03_201504130656_397797]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_03_201504130656_397797, pattern:cdr_OTHERCLUSTER_01]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_03_201504130656_397797, pattern:cmr_OTHERCLUSTER_01]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_03_201504130656_397797, pattern:cdr_OTHERCLUSTER_02]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_03_201504130656_397797, pattern:cmr_OTHERCLUSTER_02]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_03_201504130656_397797, pattern:cdr_OTHERCLUSTER_03]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_03_201504130656_397797, pattern:cmr_OTHERCLUSTER_03]

2015-10-27 11:47:10,541 [STP SmartThreadPool Thread #1] VERBOSE SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - Checking candidate to download: [file:cdr_OTHERCLUSTER_03_201510211223_222392]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_03_201510211223_222392, pattern:cdr_OTHERCLUSTER_01]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_03_201510211223_222392, pattern:cmr_OTHERCLUSTER_01]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_03_201510211223_222392, pattern:cdr_OTHERCLUSTER_02]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_03_201510211223_222392, pattern:cmr_OTHERCLUSTER_02]

2015-10-27 11:47:33,370 [STP SmartThreadPool Thread #0] VERBOSE SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - Checking candidate to download: [file:cdr_OTHERCLUSTER_01_201505151938_112934]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_01_201505151938_112934, pattern:cdr_CLUSTERX_01]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_01_201505151938_112934, pattern:cmr_CLUSTERX_01]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_01_201505151938_112934, pattern:cdr_CLUSTERX_04]

File name doesn't start with expected pattern [fileName:cdr_OTHERCLUSTER_01_201505151938_112934, pattern:cmr_CLUSTERX_04]

2015-10-27 11:47:33,385 [STP SmartThreadPool Thread #0] VERBOSE SolarWinds.Orion.IpSla.Jobs.Ftp.CiscoCcmFtpClient - Checking candidate to download: [file:cdr_CLUSTERX_02_201509301348_480118]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201509301348_480118, pattern:cdr_CLUSTERX_01]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201509301348_480118, pattern:cmr_CLUSTERX_01]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201509301348_480118, pattern:cdr_CLUSTERX_04]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201509301348_480118, pattern:cmr_CLUSTERX_04]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201509301348_480118, pattern:cdr_CLUSTERX_03]

File name doesn't start with expected pattern [fileName:cdr_CLUSTERX_02_201509301348_480118, pattern:cmr_CLUSTERX_03]

Have anyone suffered from the same sudden disapearences ?

Thanks

lucousin · Accepted Answer

Hi,

Sorry for the delay but I'll still answer in case you didn't figure it out on your own.

Log files can be found under %ProgramData%  => C:\ProgramData\Solarwinds\Logs\IPSLAM

At first I thought Resetting CDR sequence numbers - SolarWinds Worldwide, LLC. Help and Support would be the solution, you also might wanna check it out.

But as a solution to my problem I had to modify VOIP batch polling settings for CCM directly into the database using the database manager on the solarwinds orion main server.

VoIP and Network Quality Manager creates multiple new sessions on the Avaya Call Manager - SolarWinds Worldwide, LLC. He…

I followed this article and made the following tweaking (highlighted entries) until all phone call historical data was recovered:

Hope this helps you...
Anyhow, it's a good tweak for everyone using VNQM, although it was quite hard to find all this info !