Scripts not working

Someone else trouble that powershell script not work anymore?

This new update destroys more than its helped.

I find in the log this entry:

[ 95|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Grpc.GrpcCertificateStore|240731| 5:41:54.220|Error ] GrpcCertificateStore.LoadCertificates(IApplicationPaths appPaths): Unable to load certificate file 'C:\ProgramData\protected-networks.com\8MAN\cfg\security\server\ARM-SERVER.Domain.de_client-cert.pfx'! CryptographicException Das System kann die angegebene Datei nicht finden.

[ 96|ARM-SERVER|pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Grpc.GrpcCertificateStore|240731| 5:41:54.223|Info ] GrpcCertificateStore.LoadCertificates(IApplicationPaths appPaths): Check the read permission of file 'C:\ProgramData\protected-networks.com\8MAN\cfg\security\server\ARM-SERVER.Domain.de_client-cert.pfx' for account 'DOMAIN\ARM-USER '.


[ 97|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Grpc.GrpcCertificateStore|240731| 5:41:54.237|Error ] GrpcCertificateStore.LoadCertificates(IApplicationPaths appPaths): Unable to load certificate file 'C:\ProgramData\protected-networks.com\8MAN\cfg\security\server\WebAPI_client-cert.pfx'! CryptographicException Das System kann die angegebene Datei nicht finden.

[ 98|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Grpc.GrpcCertificateStore|240731| 5:41:54.238|Info ] GrpcCertificateStore.LoadCertificates(IApplicationPaths appPaths): Check the read permission of file 'C:\ProgramData\protected-networks.com\8MAN\cfg\security\server\WebAPI_client-cert.pfx' for account 'DOMAIN\ARM-USER '.

"Das System kann die angegebene Datei nicht finden." means File not found

But the Files are there. The Arm User have Permission on it.

Later in the Log.

[ 501|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Grpc.GrpcClientManager |240731| 5:41:54.812|Debug ] GrpcClientManager.createChannel(Uri address): Got server certificate from grpc://ARM-SERVER:55555/

[ 502|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Grpc.GrpcClientManager |240731| 5:41:54.815|Warning ] GrpcClientManager.createChannel(Uri address): ARM Client certificate not found for grpc://ARM-SERVER:55555/, this will cause issues when communication to ARM.

[ 503|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Grpc.GrpcClientManager |240731| 5:41:54.925|Debug ] GrpcClientManager.createChannel(Uri address): New gRPC channel created for grpc://ARM-SERVER:55555/

Now the exception.

[ 504|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |SolarWinds.ARM.Remo…cSubscriptionChannel|240731| 5:41:55.061|Error ] GrpcRemotingChannelCreator.createGlobalServerChannelInternal(Int32 port, Boolean selectServerPort): System.Runtime.Remoting.RemotingException: Initialization of gRPC stream failed by status 'Status(StatusCode="Unavailable", Detail="", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1722404514.978000000","description":"Error received from peer ipv6:[fe80::7e36:18b:c79e:e332%258]:55555","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"","grpc_status":14}")'! ---> Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1722404514.978000000","description":"Error received from peer ipv6:[fe80::7e36:18b:c79e:e332%258]:55555","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"","grpc_status":14}")
--BEGIN-OF-EXCEPTION------------------------------------------------------------
System.Runtime.Remoting.RemotingException: Initialization of gRPC stream failed by status 'Status(StatusCode="Unavailable", Detail="", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1722404514.978000000","description":"Error received from peer ipv6:[fe80::7e36:18b:c79e:e332%258]:55555","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"","grpc_status":14}")'! ---> Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1722404514.978000000","description":"Error received from peer ipv6:[fe80::7e36:18b:c79e:e332%258]:55555","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"","grpc_status":14}")
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.<sendToServer>d__54.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.<initStream>d__55.MoveNext()
--- Ende der internen Ausnahmestapelüberwachung ---
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.<initStream>d__55.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.<startListening>d__52.MoveNext()
--END-OF-EXCEPTION--------------------------------------------------------------
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.<startListening>d__52.MoveNext()
--END-OF-EXCEPTION--------------------------------------------------------------

the next exception

[ 507|ARM-SERVER |pnJob |5276.1| 1|DOMAIN\ARM-USER |pn.framework.ProgramSystemServiceManager|240731| 5:41:55.062|Error ] SystemServiceManager.EnterSystemRunLevel(SystemRunLevel systemRunLevel): Error while entering runlevel Startup in service pn.jobs.JobProgram: System.Runtime.Remoting.RemotingException: Initialization of gRPC stream failed by status 'Status(StatusCode="Unavailable", Detail="", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1722404514.978000000","description":"Error received from peer ipv6:[fe80::7e36:18b:c79e:e332%258]:55555","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"","grpc_status":14}")'! ---> Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1722404514.978000000","description":"Error received from peer ipv6:[fe80::7e36:18b:c79e:e332%258]:55555","file":"..\..\..\src\core\lib\surface\call.cc","file_line":953,"grpc_message":"","grpc_status":14}")
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.<sendToServer>d__54.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.<initStream>d__55.MoveNext()
--- Ende der internen Ausnahmestapelüberwachung ---
bei SolarWinds.ARM.Remoting.GrpcSubscriptionChannel.StartListening(Object data)
bei SolarWinds.ARM.Remoting.GrpcRemotingChannelCreator.createGlobalServerChannelInternal(Int32 port, Boolean selectServerPort)
bei pn.framework.tasks.TaskProgram`5.Startup()
bei pn.framework.SystemServiceManager.EnterSystemRunLevel(SystemRunLevel systemRunLevel)

Its try ipv6? and this gRPC is the new certificate garbage. This error looks like the certificate is not available.

In the Webinterface i have the Server 2 Times.

Anyone a Idea? I open already a Ticket.

My template Work find without script. I already create a simple script but it dont work ether. I guess this certificates make it very worse.

  • Exchange Mailbox permissions change dont work ether!

    Same Error

  • My Ticket was excelated to the next tier...

  • Thanks for keeping us posted. 
    Please update, if they were able to help you.

  • Same here and no Scans. I have reinstalled Collectors and Certs 3 times and also after a reboot I have to restart the services on all Servers where the collectors are installed. Then they connect. One collector though does not work and tries to reinstall, which ends up in a wmi error. 

  • My Ticket is now by Greg:-)
    He gave me this Answer.

    The scripts are an issue with the latest release as you have found.

    I wanted to provide for you the full steps to uninstall and reinstall a Collector with the new version.

    All steps:

    1. Uninstall the ARM software from the Collector server
    2. Delete the C:\Program Files\Solarwinds\ARM folder
    3. Delete all folders in the C:\ProgramData\protected-networks.com\8MAN except for the Logs folder if you wish to save those. Normally they are not needed.
    4. You may find that you cannot delete some of the certificate files, so you can change the permissions of the files in C:\ProgramData\protected-networks.com\8MAN\cfg\security\server to 'Everyone' with full access in order to remove those few files, and you will be able to delete them.
    5. Reinstall the ARM Collector and FS Logga by doing a advanced > custom installation using the ARM Setup installation file.

    Please let me know if this works to resolve the errors you have seen.

    I Let him know that is my only Collector, primary ARM Collector. So i ask do uninstall our ARM Server or i should better do an migration.
    Maybe you guys can use it or "try" :-)

  • Sounds like much work, to reconfigure all the settings etc.
    We also only got one primary ARM Collector.

    If this helps for you, pleae give a heads-up.
    My favorized Option would be to install a patch which solves the problem. 

  • Hi Got this Information


    Ok so this is different, when cx says 'collector' we assume it is another machine that is not the ARM server.

    Ok try these steps:

    1. Stop the ARM-Service on the ARM server.
    2. Remove/delete all the files and folders from C:\ProgramData\protected-networks.com\8MAN\cfg\security
    3. You'll see the 2 older ARM certificates, and 4 newer certificates in a folder called 'server'.
    4. You may need to change the security of the new files in order to delete them, I usually choose 'everyone - full access' in order to delete them
    5. Restart the ARM service. This will recreate the certificates. The 2 older ARM certificates will not be recreated, but the 4 newer certificates will be recreated in the server folder.

    For me it not work. Now i cant Log in via Server Desktop. My Client Work after i re-register it. 

    Waiting now for the next try...

  • We had a live support session and found a temporary resolution for this issue:

    Log into the ARM server with the same account that is configured for the Exchange Scan, in this case it is the ARM service account.
    This may also need to be the account that installed the ARM software, which should also be used in the Exchange Scan configuration.

    Both the Exchange Scan and the new user scripts work when logged in this way for this environment.

    We had one small issue (error code 1346) but this was resolved by using only the NETBIOS domain name for the service account instead of the FQDN, i.e. domain\acct instead of domain.company.com\acct

  • Thanks for the information.
    Are you referring to the issue of scripts not running?
    Or is this suggested solution about the Exchange scans not working?

    I am surprised that the same user should be used for the Exchange scan as for the ARM installation.
    I always understood that separate service accounts should be used for the different ARM functions:
    documentation.solarwinds.com/.../arm_2024-3_system_requirements.htm

    And can you please clarify what "temporary resolution" means?
    Does the account need to be permanently logged in? Or is a one-time login enough?

    Edit:

    Sorry, I missed this statement:
    "Both the Exchange Scan and the new user scripts work when logged in this way for this environment."

    However, my question remains whether the solution is only valid as long as the user is logged in.
    As some versions of ARM have implemented automatic logout, this sounds like a rather cumbersome solution.