The tools for ESXi messages/vmkernel that are natively built in to LEM are built for receiving the data from syslog to LEM directly, since you can't install an agent on ESXi as you found. I'd recommend sending the ESXi data via syslog to LEM and see if you can get that working first. The right tools to configure are the two tools labeled ESXi (vmkernel and messages) to their default log locations, then the regular ESX hostd tool to /var/log/local4.log.
Configuring syslog on ESXi: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1016621
If you build your own tools, chances are they are conflicting with the existing LEM tools, since we aren't really set up for customer-built tools and there are some key "keywords" that are probably causing your tools to not function. If that's not it, it's probably a matter of matching the right data in the right ways - there's some internals to the way the tools match data that isn't self-explanatory.