There's another thread going on this right now. The simple answer is that your NetFlow exporter needs to be configured to see the client-side traffic, and NPM needs to be configured to monitor those interfaces.
If you go onto your switch and issue the "show ip cache flow" command (assuming it's Cisco), do you see the client side flows?
If yes, then the problem is with your NTA/NPM configuration.
If you don't see the client-side flows on the switch, then the problem is in your switch configuration or in your network design.
You'll never see the true HTTP conversation when the device uses the proxy server. You will only see client-proxy as the source/destination pairs. If you want to see http to destinations not required to pass through the proxy, then you need to bypass those destinations with proxy exclusions at the client.
If your proxy port is 80, change it so you can make a cleaner application definition in NTA for proxy.
Your proxy server should have reporting to tell you where users are going. If you can't or don't want to do it that way then you need to look at wire level (or SPAN/MIRROR ) probes that will do deep packet inspection and pull out the true destination from a higher layer of the OSI model. That solution is not cheap.
It would be nice if the proxy server vendors would build netflow records reflecting the true conversation to be forwarded to a collector.
Thanks for your reply. You mean there's no work-around as to monitor the http traffic using the NTA if the proxy is explicitly configured? For the http traffic to work, we either have to disable the proxy or use a probe appliance??
That's pretty much correct.
With proxies and "traditional" NetFlow, you have the following options:
1) NetFlow exporter outside of proxy: allows you to see proxy-server to Internet flows.
2) NetFlow exporter behind proxy: allows you to see client to proxy-server flows.
3) Transparent (aka "intercepting") proxy: allows you to see end-to-end, client to Internet flows, but in this case the proxy server doesn't terminate the client sessions, and isn't explicitly configured.
If you are using a traditional, explicitly configured proxy, a probe appliance would have to be smart enough to look inside the HTTP header and export the HTTP request itself (Websense does this when configured in transparent mode, for example). NetFlow version 9 does have the option to export packet slices, so in theory this would be possible with a sophisticated enough collector. The problem, of course, is that the HTTP request length is variable, so the developer would need to come up with some way of dealing with that issue. In any case, the Solarwinds product doesn't currently support any type of raw packet export.
If you want to see HTTP that doesn't need to go through the proxy then you need to add proxy exclusions for those destinations in the browser, java, etc..