I have an "Out of the box" application that uses port 5000. I've found that the SolarWinds Agent is also using port 5000. Searching Thwack took me to @aLTeReGo 's famous article https://thwack.solarwinds.com/t5/Product-Blog/Portocalypse-WMI-Demystified/ba-p/441106 However this problem does not seem to be WMI related. If I start this other application/service first, then start the SolarWinds Agent, this error comes up in c:\ProgramData\SolarWinds\Logs_Agent\Cortex\cortex.log file.
So the agents will utilize the localhost connections for various functionality. That error makes me think that you have bound localhost to port 5000, that you have a redirect set on it, or that you have an alteration to the host file on the server for it. As far as the port in and of itself as being a requirement, I'm not aware of any default setting requirement on any of the core Solarwinds products that need 5000, nor can I find any within the port requirements documentation:
Any chance you can check those three things above? Also, if you do a netstat -aon sans agent service running, are there any references to port 5000 being used by any of the PIDs?
When the agent is running and I run netstat -nab ( netstat -aon doesn't show the services using the port) I see this for port 5000
TCP 127.0.0.1:5000 0.0.0.0:0 LISTENING
TCP [::1]:5000 [::]:0 LISTENING
This is on every server with a Solarwinds Agent on it that I have checked in my environment. It is only a problem on one server where I have a Web Server that has been using port 5000 for a couple of years, and I only noticed (the DevOps guy told me about the conflict) after upgrading to 2020.2
I'm not sure what you mean that I have "bound localhost to port 5000." I haven't setup any redirects nor have I altered any host files. After opening this forum request I did see this in the 2020.2.1 release notes.
|557589, 558717, 566095||Cortex on Agents starts Kestrel on port 5000 only when the endpoint is switched on.|
I wish there was more information about what that means. What is Kestrel? What is it used for? What endpoint is switched on? It sound like it can be switched off, but how?
Howdy @jfaldmo ,
First off, thanks for that link. Gave me some info to start digging into the back end cases. And there was an issue with the agents where it attempted to spin up that connection which could result in blocking the agent from being started (as in your experience) or also a possible memory leak that cropped up in a rare instance.
I'm digging more into it to see if I can get an answer to your questions there (in regards to the what is it, what its used for, and how to switch it on/off). The major question I have at the moment is simply if it is referring to the Kestrel queue or if it is simply talking about the default web server implementation in ASP.NET core (which is Kestrel). Also, I am trying to see if the fix that was included in 2020.2.1 includes the situation you are in (which is that you actually have sometime utilizing that port) as opposed to just the arbitrary attempt of the agent to connect on port 5000.
In the mean time, two quick questions. First off are you on the 2020.2.1 release? And secondly, did you uninstall and reinstall the agent on the server you are having issues with?
I cannot speak for the original poster but I am personally on 2020.2.0 and so far have one server using Apache on the default 5000 port. have uninstalled and reinstalled 3 times. I was able to work with vendor support for the server that it is impacting and tested using another port for Apache Web Services. As long as Apache Web Services is on another port (say 43216), they run side by side no problem, but this appears to be a temporary fix. I have also attempted to stop web services and SolarWinds, start SolarWinds, and then web services to see if maybe an order was needed. Also tried the reverse to the same results. running a get-nettcpconnection on local port 5000 yields 3 listen results while web services and solarwind are running.
Local: "0.0.0.0" Remote: "0.0.0.0" Apache Web Sevices
Local: "::1" Remote: "::" Solarwinds
Local: 127.0.0.1 Remote: "0.0.0.0" Solarwinds
SolarWinds solutions are rooted in our deep connection to our user base in the THWACK® online community. More than 150,000 members are here to solve problems, share technology and best practices, and directly contribute to our product development process.