cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post
Level 7

Kiwi Syslog Service hanging

1st time starting a discussion.

1st time working with Kiwi Syslog.

Let me know if I'm in the wrong place.

I am very new to Syslog Servers.

I'm a Route/Switch type guy.

We are using Kiwi Syslog to get Call Manager Call Traces for troubleshooting.

This Instance of Kiwi Syslog was working fine as a Guest VMware Server on a Host Server.

We used the app Veeam to move the Kiwi Syslog VMware Guest Server to another Host.

This issue started after the copy/move of the Kiwi Syslog

No IP addresses were changed, it's on the same network as before.

It starts up, logs are being received, and then they stop.

If you try to start the service, it tells you it's already running.

At the bottom of the Kiwi Syslog Service Manager, you can see the MPH indicator has stopped.

Looking at the correct folder I can see the logs are no longer being  received.

If I stop the service and start the service it starts.

There is a script that tells it to restart every morning at 4am, and it will do this.

Below is the error event seen when it stopped last time.

pastedImage_0.png

Windows Server 2012 R2

64 -bit OS

Has anyone seen this type of issue before?

Any help would be greatly appreciated,

Mhaley

0 Kudos
36 Replies
Level 7

I've just updated the support case since it was closed , i will upload a screenshot of installed software to this post:

Case # 1174540 - Kiwi Syslog Crashing on changing setup

Dear Support department,

we've uploaded Multiple logs like more then enough. Now the case gets closed by contacting bianca, please contact S******@promax.nl

i'm almost sure that kiwi syslog breaks as soon as you install wireshark which in turn install some visual C++ resistributables i would urge you to install that in a lab environment and test it yourself i'm pretty much done with being a test subject for flawed software.

solarwinds.jpgsolarwinds.jpg

0 Kudos

Thanks jtc_osaka​ !!

Finally after 6 Months someone points us to something that seems to have fixed the issue while " support "  can only respond with :

Please provide us a contact number and contact person other than the distro e-mail. We tried to call last time, and the latest e-mail and contact was linked to Bianca. As stated before, we need fresh logs and info of your environment. Old logs are already stale. Please send us new KSS Tech Support file, screenshot of the error, crash logs if any and the NFO file of the server.

0 Kudos

I'm glad you got this resolved.  Sometimes fellow thwack members actually have knowledge that even support may not be able to piece together.  I've had my own issues with kiwi syslog server myself.  I could never get the service to stay running on windows2012R2 domain controller that was STIG'd.  I may go back and try again.  My temporary solution has been to run free version of WUG syslog server to comply with DISA security requirements.  I'm hoping to be able to go and try a newer version of kiwi syslog server in near future because I did pay for it and honestly WUG syslog server free version is terrible.

Mega props to jtc_osaka​ for providing solution.

0 Kudos
Level 7

We've basicly given up on this support case, it seems that selling a product is one but making it work is two. We now have 5 licensed softwarepackage wich down work correctly. With a new customer we are working on implementing syslog , we installad a new server 2016 and the following software :

DisplayName                                                    DisplayVersion            Publisher             InstallDate

-----------                                                    --------------            ---------             -----------

7-Zip 16.04 (x64)                                              16.04                     Igor Pavlov                     

                                                                                                                         

Mozilla Firefox 56.0.2 (x64 en-US)                             56.0.2                    Mozilla                         

Mozilla Maintenance Service                                    56.0.2                    Mozilla                         

nProbe for Windows                                             7.5.170507                ntop.org              20171101  

                                                                                                                         

VMware Tools                                                   10.0.0.3000743            VMware, Inc.          20170207  

WinCollect                                                     7.2.6.39 - 20170427163606 IBM                   20171031  

PuTTY release 0.70 (64-bit)                                    0.70.0.0                  Simon Tatham          20171031  

Microsoft Visual C++ 2015 x64 Minimum Runtime - 14.0.24215     14.0.24215                Microsoft Corporation 20171101  

Microsoft Visual C++ 2008 Redistributable - x64 9.0.30729.6161 9.0.30729.6161            Microsoft Corporation 20170207  

Google Chrome                                                  62.0.3202.75              Google, Inc.          20171031  

Microsoft Visual C++ 2013 x64 Additional Runtime - 12.0.21005  12.0.21005                Microsoft Corporation 20171101  

Microsoft Visual C++ 2013 x64 Minimum Runtime - 12.0.21005     12.0.21005                Microsoft Corporation 20171101  

Microsoft Visual C++ 2015 x64 Additional Runtime - 14.0.24215  14.0.24215                Microsoft Corporation 20171101  

I'm starting to think that the problem started when we installed wireshark 2.4.2 64bit i urge support to test this and fix this issue we now have basicly 5 licensed kiwi syslog servers that we cant work with correctly

0 Kudos

Can you provide me with your support ticket number so I can investigate?

0 Kudos
Level 12

After upgrading to version 9.6.2 my problem seems to be resolved

0 Kudos
Level 10

I'm having the same issue.  No problems running on 9.6.0 but when I upgraded to 9.6.1 last week, we had our service fail all three atempted restarts.  Very frustrating!

Error:

2017-07-28 23:51:19*** INTERNAL PROGRAM ERROR - Please contact http://www.kiwisyslog.com/support/ ***
2017-07-28 23:51:19Service Version =9.6.1.6 | Error Number: 13 | Description: Type mismatch | Module Name: Syslogd.frm | Procedure Name: ParseMessageQueue | Line Number: 420 | Date and time: 7/28/2017 11:51:19 PM

We normally see 400k messages per hour with no issues at all.  Rules hasn't changed. 

Support case was opened:

Case #1203969 - "Kiwi Syslog Service Stops With An Internal Error"

Did anyone receive a resolution for this?

If needed, I can debug the program.

0 Kudos

Hi deiberts-san,

Have you not gotten the solution from SolarWinds yet?

In my system,

KSS 9.6.1(9.6.1.6) was stopped with same Error by receiving TCP.

---------

2017-07-10 10:14:16 *** INTERNAL PROGRAM ERROR - Please contact http://www.kiwisyslog.com/support/ ***

2017-07-10 10:14:16 Service Version =9.6.1.6 | Error Number: 13 | Description: Type mismatch | Module Name: Syslogd.frm | Procedure Name: ParseMessageQueue | Line Number: 420 | Date and time: 2017/07/10 10:14:16

---------

My Support Case is  #1193443 - "KSS 9.6.1 was stopped, when it received a syslog by TCP."

I got the Buddy Drop 33351 that fixes this problem.

After applied this BD, KSS 9.6.1(9.6.1.28) can received TCP logs without error.

SolarWinds says that I can not provide this BD to another user.

Please ask SolarWinds technical support how to get this BD33351. 

Regards,

0 Kudos

In our organization, we only utilize UDP 514 as our default syslog source.  However, from what I was told from Solarwinds support that certain files are being unregistered from Windows Update. I'm seriously debating on reverting back to 9.6. 

0 Kudos
Level 7

I had the issue of the log file taking all the usable disk space with version 9.6.0

I upgraded to 9.6.1 to fix this issue, but I then the syslogd service started crashing after a couple of minutes.

I do not have the latest .Net upgrades installed.

I notice the same error messages as you guys:

Service Version =9.6.1.6 | Error Number: 13 | Description: Type mismatch | Module Name: Syslogd.frm | Procedure Name: ParseMessageQueue | Line Number: 420 |

My volume is 1293K MPH.

I easily solved this issue by installing KiwiSyslog 9.5.1.59

The problem is clearly a bug introduced in version 9.6.1

0 Kudos

Hi,

Does your KSS 9.6.1 enable "Inputs>TCP"?

KSS 9.6.1 was stopped with same Error by receiving TCP.

---------

2017-07-10 10:14:16 *** INTERNAL PROGRAM ERROR - Please contact http://www.kiwisyslog.com/support/ ***

2017-07-10 10:14:16 Service Version =9.6.1.6 | Error Number: 13 | Description: Type mismatch | Module Name: Syslogd.frm | Procedure Name: ParseMessageQueue | Line Number: 420 | Date and time: 2017/07/10 10:14:16

---------

When only "Inputs> UDP" is enable, KSS 9.6.1 is running without problem.

So, KSS 9.6.1 has a problem receiving TCP logs, I think.

0 Kudos

Yes I have one small data source doing encrypted TCP syslog. It was not causing problems until 9.6.1

This might be the root cause of the bug we're seeing.

Fun Fact I noticed as well: I don't have to activate UDP for it to be enabled in 9.6.x. In short the checkbox does nothing and it is enabled by default.

In version 9.5.1, if they are not explicitely enabled in the setup menu, the logs won't show up in display and enter KiwiSyslog.

Until a new version gets released with fixes, it's gonna be 9.5.1 for me.

Sad state of things honestly for a professionnal product.

0 Kudos
Level 15

jeffreyc how many messages are you receiving per hour?

0 Kudos

Last hour 120,000

I have 3 other servers running 9.6.1 without issue and one of them has many more messages.

This server just doesn't seem to want to run 9.6.1 but runs 9.6.0 fine

0 Kudos
Level 7

It seems that some .net updates or something screwed up the compatibility , coworker found the main issue with the setup of the Kiwi Syslog Service Manager and updates our support ticket with this information. Now we are a week into this ticket and we didn't get a response anymore.

It seems that the DLL has changed with some dependencies. This causes issue with the manager.

To solve the issue with the manager, I can confirm that copying the DLL to the program folder itself solves the messages and errors for about 98% (as in everything works, but sometimes there is a new error that pops-up because of error handling (according to .NET explanation)).

2017-06-01 11:56:59       *** INTERNAL PROGRAM ERROR - Please contact http://www.kiwisyslog.com/support/ ***

2017-06-01 11:56:59       Service Version =9.6.1.6 | Error Number: 13 | Description: Type mismatch | Module Name: Syslogd.frm | Procedure Name: ParseMessageQueue | Line Number: 420 | Date and time: 1-6-2017 11:56:59

2017-06-01 11:59:28       *** INTERNAL PROGRAM ERROR - Please contact http://www.kiwisyslog.com/support/ ***

2017-06-01 11:59:28       Service Version =9.6.1.6 | Error Number: 13 | Description: Type mismatch | Module Name: Syslogd.frm | Procedure Name: ParseMessageQueue | Line Number: 420 | Date and time: 1-6-2017 11:59:28

2017-06-01 12:05:25       *** INTERNAL PROGRAM ERROR - Please contact http://www.kiwisyslog.com/support/ ***

2017-06-01 12:05:25 Manager Version = 9.6.1.6 | Error Number: 440 | Description: Automation error | Module Name: Syslogd.frm | Procedure Name: UpdateGridDisplay | Line Number: 30 | Date and time: 1-6-2017 12:05:25

In regards to the Syslog service itself kicking the bucket, we’ve found out that the cause of the service hanging itself is UDP and TCP syslog on one and the same port (514), will kill the service. If we try the same with an installation we’ve not yet updated to this newest release, it doesn’t crash, but with the newest release it crashes.

For now it seems like we’ve reached an all new record of the service and manager running for 10 minutes without crashing

0 Kudos

mhaley​ & martijng

If you are having trouble with support, perhaps DanielleH​, jeff.stewart​​, or bkyle can assist with getting some momentum going on those tickets. Each of you may benefit from requesting an escalation. Also, it might be a good idea to post your case # here, in case Danielle or Jeff need to help push the tickets to attention.

I'm glad you're making progress, but sorry to hear support hasn't been able to fix it, yet.

Thank you,

-Will

0 Kudos

Happy to look into this if you want to provide the Support Ticket number.

Jeff

Our ticket number is > Case #1174540 'Kiwi Syslog Crashing on changing setup' has been closed

Yesterday we've send another email " So where you able to replicate the issue in your LAB environment? Since the issue with the manager still needs to be solved." 

With replacing the DLL's in the installation folder of kiwi we managed to keep the service running but the GUI keeps crashing soi've installed the old 9.5.2 version which means we run into the isso of the log files being created in the program files folder again (which was fixed in the 9.6.1 version)

The old version now also crashes the gui :

THE FOLLOWING INTERNAL PROGRAM ERROR HAS OCCURRED:

Manager Version = 9.5.2.5

Error Number: 440

Description: Automation error

Module Name: Syslogd.frm

Procedure Name: UpdateGridDisplay

Line Number: 30

Date and time: 8-6-2017 17:00:422017-06-08_17-01-43.png

0 Kudos

we've been asked :

Hi Sean,   We are unable to replicate the issue.   If a Microsoft *.dll file in SYSWOW was causing most of the error, were you able to perform a system file scan?   You can also try the following:   - Disable .NET framework 3.5 in server features. - Reboot the machine. - Then re-enable .NET framework 3.5     Regards, Ryan Rosales SolarWinds Technical Support 1500h-2400h and GMT+0800 Support helpline for 24x7 Support: http://www.solarwinds.com/company/contact.aspx

We've done that and it didnt help it is still crashing

I've installed an NEW server 2016. Installed kiwi syslog imported my settings and all seem to work. i am gettings logs and everything works just fine. I decide to deactive the licence on the old server and activate it on this one that was succesfull also. but suprise surprise AFTER activation it starts crashing also.

I'm prety amazed here i've had it running for at least 10 minutes without a single crash and AFTER activating i'm getting the following errors :

Application: Syslogd_Service.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: exception code c0000005, exception address 74E0BA87

Stack:

Faulting application name: Syslogd_Service.exe, version: 9.6.1.6, time stamp: 0x59019ce7

Faulting module name: OLEAUT32.dll, version: 6.2.14393.1198, time stamp: 0x5902888e

Exception code: 0xc0000005

Fault offset: 0x0001ba87

Faulting process id: 0x%9

Faulting application start time: 0x%10

Faulting application path: %11

Faulting module path: %12

Report Id: %13

Faulting package full name: %14

Faulting package-relative application ID: %15

0 Kudos