We run a a small farm of JBoss servers - JBoss exports all sorts of information out via SNMP, so I'm trying to set up APM to access it. I have configured the node to use SNMPv1, port 1161, read-only community 'public', which is what JBoss uses by default. I've also set up two test checks by OID in a template, and assigned it to the nodes.
Functionally, it works - however only 33% of the time. About 1 in 3 checks works fine - I get a number, and the light is green. The other 2 requests simply time out (after 150 seconds). When it works, I get very good response times - about the same as an ICMP round-trip. There isn't any obvious pattern to when it does and does not work - however I've noticed that it always seems to either fail or succeed for both my tests - so either all the tests for a node work, or none of them do.
I've run tcpdumps on both the orion server and the nodes in question, and can see a response for every request.
For example - below is a subset of the a wireshark capture I ran on the Orion server - you can clearly see the responses making it back in a timely fashion (first column is time, measured in seconds since the capture started):
"40.814048","83.231.236.193","128.242.96.110","UDP","Source port: 3288 Destination port: 1161"
"40.814049","83.231.236.193","83.231.207.104","UDP","Source port: 3287 Destination port: 1161"
"40.815287","83.231.207.104","83.231.236.193","UDP","Source port: 1161 Destination port: 3287"
"40.865797","83.231.236.193","128.242.96.109","UDP","Source port: 3289 Destination port: 1161"
"40.986414","128.242.96.110","83.231.236.193","UDP","Source port: 1161 Destination port: 3288"
"41.038009","128.242.96.109","83.231.236.193","UDP","Source port: 1161 Destination port: 3289"
"42.252565","83.231.236.193","83.231.207.102","UDP","Source port: 2544 Destination port: 1161"
"42.253503","83.231.207.102","83.231.236.193","UDP","Source port: 1161 Destination port: 2544"
"42.260571","83.231.236.193","83.231.207.106","UDP","Source port: 2544 Destination port: 1161"
"42.261824","83.231.207.106","83.231.236.193","UDP","Source port: 1161 Destination port: 2544"
"42.300580","83.231.236.193","83.231.207.104","UDP","Source port: 2544 Destination port: 1161"
"42.301444","83.231.207.104","83.231.236.193","UDP","Source port: 1161 Destination port: 2544"
So - does anyone know why only a third of these responses might be making it into Orion? As you can imagine, this is a bit of a show-stopper, so I'd be very keen to find a solution!
