This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Kiwi Syslog Server HA (High Availability)

Hi Folks, I am starting to evaulate Kiwi Syslog Server and one of the main requirements will be how we provision HA (High Availability)

I have seen some posts regarding the use of LB's (Load Balancers) but these posts are pretty old and don't go into that much detail.

I'm hoping that someone can point me in thr right direction.

If we use 2 LB's in a cluster (probably Netscalers) all clients will connect to the LB VIP.

I'm "guessing at this stage" that the LB's will send all trafiic to one of 2 Kiwi Syslog servers (lets call them Kiwi A and Kiwi B where Kiwi A is the current live server)

We are resilient against the loss of a LB (as theye are operating in a HA Cluster)

If we lose Kiwi A, traffic will be redirected to Kiwi B.

Thoughts/Comments

If we lose a LB, we will probably lose syslog records - I don't think its possible to avoid this (even if we use TCP)?

If we lose Kiwi A, syslog records will be redirected to Kiwi B by the LB (again, I think we could lose some syslog records)

If we need old logfiles on Kiwi A (that isn't now available) - I guess we can't unless Kiwi A writes to a CIFS share (that Kiwi B also writes to) ???

If we don't have access to a direct CIFS share, could we use Windows DFS (so that Kiwi A is replicating to Kiwi B and vice-versa) - again, I think we will miss records.

So basically, if we lose Kiwi A, and the LB starts writing to Kiwi B, Kiwi B will have the replicated records (via DFS from Kiwi A)

Kiwi B should pretty much have almost all of the records available (would need to test this against busy input devices)

Before we go down this road and start testing, It would be great if anyone has any information/feedback/comments  they could provide.

Many Thanks.

  • Your post covers just about all the points...

    Some sort of load balancer is needed, and they should always be an HA pair. Load balancing UDP can have some special settings depending on the LB design.  Using TCP offers more LB options but can significantly reduce the number of messages that can be handled in Kiwi.

    You can set them up as a failover pair, with one active or a load balanced pair with both active and shared load. We configure ours(3 servers) as a load balanced set with all servers active. Then there are 2 other servers in remote locations that normally handle only the local logs.  In the event of a failure of more than 1 of the 3 server set the remote servers become active in the cluster to help with the load. Because of distance and traveling over WAN links we try to minimize their usage in the LB cluster.

    You could lose some messages during the transition but unless it's under heavy load it will be minimal.  This is dependent on the configuration of the load balance and how quickly it will detect a down Kiwi server.  Ours takes about 3 seconds max to transition fully. At our average load this could mean 300 or so dropped messages.

    Yes, anything written to the local servers is unavailable while that server is offline.  You can write to a network share, I personally don't recommend it.  Write locally and copy/archive to network shares.  DFS is an option for replication. You can also create a rule to forward the messages to the other server. I also don't recommend this. In high message count environments you should try to minimize forwarding rules as they are an additional load on the Kiwi server engine.  Forwarding all messages effectively doubles your message count and halves your capacity.   Your rules could also write to the other server/locations via a share.  I'd be cautious with this as you don't want those writes to potentially slow your Kiwi server processing.

    We write all logs to the local disk and they roll over at 500Mb. Then there is a Kiwi Scheduled task that runs every 10 minutes and looks for any rolled over files and moves them to shared storage for archiving.