Is this only happening on one switch? If so, it's likely the switch or something on the network in between. Two things that would be helpful in understanding the problem:
1. Try running a continuous ping of the switch in question from the command line of your polling server (if you only have one Orion server, that's the polling server).
2. Please go to Settings/Details/Polling Engines and post a screen shot of the stats for the polling engine that handles this switch.