14 Replies Latest reply on Dec 6, 2011 11:24 AM by DanielleH

    CatTools_Client Memory Issue

    Sartuche24

      Okay I've been running into a serious problem when using CatTools. I've wrote a custom script to do IOS Upgrades and also another script to pull Cisco Device information and save it to a file so I can import it into the database. The scripts work great but one problem. The memory that the CatTools_Client.exe is using when running these scripts never goes down but keeps climbing. This looks like a memory leak to me because after it finishes the activity on a device it's never restore that memory back to the pool. For me to do my upgrades I have to run it on less than 400 devices, after it completes, kill the CatTools_Client.exe process and then I can perform another batch. I've watched the scripts that are already with Cattools and they don't experience this problem.

      So my question is what is wrong with my scripts that isn't wrong with the default scripts that are included. I have included this line at the end of my scripts.     cl.DisconnectHost which is supposedly to disconnect the client but it doesn't sound or look like it's working correctly. If a developer sees this I would be more than happy to let them look at my script so they can see if there is a problem in it.

        • Re: CatTools_Client Memory Issue

          Hi Sartuch--

          What version of CatTools are you running? Did you upgrade to 3.5 recently?

          M

          • Re: CatTools_Client Memory Issue
            Steve Welsh

            Hi Sartuche24,

            Can you attach your custom scripts and I'll see if I can spot anything obvious which may be causing the memory leak in the CatTools_Client.exe.

            Also, with regards to the memory leak; how serious is it?  Are we talking a a few bytes with each run or are we talking mega bytes?

            Regards,

            Steve 

              • Re: CatTools_Client Memory Issue
                Sartuche24

                Here is the script that is causing a problem, I only have one other script and it's based off of this so whatever is fixed in this one will correct my other script. If in fact it is a script problem. Also it is using Mega Bytes and has actually used over 1Gig of memory before but if I use the same amount of devices with an included script I don't have a problem with the memory.

                 

                [View:/cfs-file.ashx/__key/CommunityServer.Components.UserFiles/00.00.03.64.27/Device.Update.IOS.Router.FTP.zip]

                  • Re: CatTools_Client Memory Issue
                    Steve Welsh

                    Sartuche24,

                    I had a quick browse over your script and I noticed a few things.

                    The main one is that you appear to have added a lot of functions coded into the Client Activity script file; which probably ought to go into the Device script file.  Normally the Client script files simply has instructions to call functions within the device scripts (for example 'Login'; 'SendPostLoginCommands';  'EnterEnableMode'; 'SendPostEnterEnableModeCommands'; 'SendSessionTerminationCommands', etc.  I can see you are already calling these functions from within your client script.

                    However I also see functions such as 'ProcessShowVersion', 'RunCommandFile', 'VerifyFlash', etc. which are all still running commands on the device.  These really ought to be added to the device script file.

                    You'd then normally access these from within the function call to the activity within the device script, e.g:

                    sResults = Custom_Activity_Template(sAltCommand)

                    Take a look at the template files in the \templates sub-folder of CatTools and you'll see what I mean.

                     

                    I also noticed you are handling a reload.  When a reload is issued on the device, the communication between CatTools can sometimes timeout.  If not careful (i.e. if the reload command is being issued within a loop of commands to send); it could result in a infinite loop.

                    You'd need to put debug messages in your script to check for this.

                    Steve

                      • Re: CatTools_Client Memory Issue
                        Sartuche24

                        I'll modify my scripts and see what happens from there. I'll give the results after I complete some testing.

                          • Re: CatTools_Client Memory Issue
                            Sartuche24

                            I changed my scripts to pushed everything into the .custom file that is loaded with the Device Script and still the same issue. Also on my Reload Function I never do an immediate reload because of losing connectivity so I reload it in a minute which give me time to logout of the device. I have attached the scripts again so you can see the difference.[View:/cfs-file.ashx/__key/CommunityServer.Components.UserFiles/00.00.03.64.27/Custom-Script.zip]

                              • Re: CatTools_Client Memory Issue
                                Sartuche24

                                Well I have Version 3.6 and I'm still experiencing the same problem. No matter if I have the scripts using the .custom option or not I still have a memory issue. I have ORION NCM but the scripting isn't as powerful as it is in CatTools so this is why I prefer to use CatTools for my IOS Updates. Even after the activities are finished the memory stays peaked. This is only on custom scripts not the included scripts so something within the application isn't releasing the memory when it's finished. this needs to be fixed and I'm willing to supply anything to help in getting the issue resolved.

                      • Re: CatTools_Client Memory Issue
                        Sartuche24

                        issue has been resolved after working with Steve Welsh on the issue. It took some major digging but in the end the problem was using the CatTools API within my scripts. After talking to Steve this was never designed to run in an activity but on external scripts which I knew but I didn't know it would cause a problem like it did, so my advice don't use the API in your activities you run through CatTools only on external scripts. The reason I was running the API is because I wanted some information from the DB but didn't know there were included variables already holding this information. These variables aren't documented but I will list them below and also brought it to the attention of Steve who will work the internal side of trying to get the documentation updated with the variables that are available within CatTools.

                        Variables:
                        cl.CurDevHostName <-- The Current Device IP Address
                        cl.CurDevType <-- The Current Device type such as Cisco.Router.General
                        cl.CurDevModel <-- The Current Model of the Device as it is in the DB
                        cl.CurDevName <-- The name of the device in the DB, this is different from cl.DeviceName because cl.DeviceName is the hostname that the activity determines once it logs into the device.

                        I would assume anything in the DB can be used if you use the following prefix to how the column is labelled in the DB:
                        cl.CurDev{Column Name} <-- Now don't take my word on it but so far it has worked. If you need a list of Column names just perform an export and the Column Headers are what you would use.



                        Thanks again to Steve for all the help.