This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Automated Failed Account/Client Login 0 Minutes Setting - How To Reset?

How To Re-Set the 0 Minutes forever denied setting?

Literally within minutes after updating the server on 12/23 for one account/client are getting denied; below find both Client logs and mFTP server logs

Client log 21:30 my time 00:30 client time

12/24/2022 00:30:19 1 0 O Global *** Scripting Step Begin: [FTP_Connect_Success]
12/24/2022 00:30:19 16 0 O Global Successfully connected to FTP site mysecure.org///toHSA
12/24/2022 00:30:19 1 0 O Global *** Scripting Step End: [FTP_Connect_Success] completed.

Client log 1 hour later after the update

12/24/2022 01:30:09 4 19 O Global Error during connect: The FTP Queue Component was unable to connect to FTP Server at 'mysecure.org///toHSA.' java.net.ConnectException: A remote host refused an attempted connect operation. (Connection refused). Please verify that the server, username, password and port number are correct.
12/24/2022 01:30:09 1 19 O Global *** Queue Step End: [SFTP_Connect] failed.


The site Setting(s) are 4 times in 8 seconds, 0 Minutes blocked (forever)

Global Dashboard note: <octets removed> by me
[02] Tue 03Jan23 08:30:07 - Connection denied from IP address 63.<octets removed> (local address 192.<octets removed>, port 22)

At about 08:40, Changed 0 minutes to 1 Minutes and checking the log...
[02] Tue 03Jan23 09:30:07 - Connection denied from IP address 63.<octets removed> (local address 192.<octets removed>, port 22)


Thanks,
JeffP...


Parents
  • Is it possible that the application could not get to the Serv-U server during the update and then flooded the server with connections that previously failed (causing the IP to be blocked)? If so the suggestion of going into Domain Details > IP Access and deleting the the specific entry for "deny" for your client IP address it should allow the application to connect again.

    If you think this is a new issue with the latest update, please let us know the specific scenario so we can replicate it too.

    Does your application only connect every hour or does it connect every 1 minute and then got blocked after 1 hour?

  • The update was minutes and occurred at on a Friday at 22ish hour +8 so very unlikely... Plus there are a few issues...

    - any ssh dsa keys are automagically removed during the update
    - NETWORK SERVICE account with full rights, should be added to program and hosted folders

    So, far only noted impacted sites use JSCH and it's suggested that not having any key causes the Java clients to not continue their connection hand-shaking; and the solution is to add an RSA 2048 key, however this means other clients will need to automatically accept the key (preferred setting) or will have an error or warning or pop-up to accept & store; and making it worse for many clients for the 2 seems risky in light of not getting absolute cause->effect from SW

  • first off, I hope you're in Monday TZ, it's 11:37 pm Sunday for me, programming to get back on track after spending time on SU last week... 

    Yeah, the RSA key is a change management process, email blast to our known user email; getting to business folks at our partners is easy getting to the correct tech to notify them to trust/accept & store the new RSA key not as easy....

  • I want to look at this from a different angle. Please can you let me know what version of JSCH you have experienced this issue with?

  • So, we've strolled off topic, but no worries from me...

    At the client side there are two log files, this one that suggests checking user name & password, but the login never actually occurs
    1) ...java.net.ConnectException: A remote host refused an attempted connect operation.
    2) ...Session.connect: java.io.IOException: End of IO Stream Read. 

    This second error message mends with SW Techs saying it's RFC rfc4253#section-4.2 not sending CRLF (non-windows systems send LF as they're often self-trimming and don't need the CR)

    From the server side, is "connected" then "time out" & disconnect, as if the client & server are each waiting for the other to respond correctly.

    Clients, these are two long time clients setup in SU c.2017, so pointing a finger at the RFC seems like SW should have warned that Java clients are going to have an issue(s).
    SSH_FXP_INIT: client version 3 (JSCH-0.1.53)
    SSH_FXP_INIT: client version 3 (JSCH-0.1.54)

    The first response 1-week after applying the SR was to Roll-back but 20+other clients are not having a problem, well except for the pending task to trust/accept the new RSA key/fingerprint. But rolling back is not really a viable option after a week+ of days...


  • No worries either, happy to talk through the process.

    Are the JSCH users able to connect now that you have an RSA host key on your domain/server, or are they still having the same problem?

  • We ran a few tests with the JSCH-0.1.53 as that client is also our/a vendor by switching to the domain with the RSA, and same result; which adds more weight to saying it's the client side.

    The user was setup in the alternate domain (only a few user/clients connect here) and added a 2023 & Test RSA key/fingerprints and tested both as well as setup of same info in WinSCP which connected w/out issue using the same credentials. The 2023 RSA was also tested at the alt site/domain by one of our vendor clients not using JSCH and had no issues, connecting, download, upload and renaming a file.

  • You are correct, I've done a test app using the JSCH library 0.1.54 and it can no longer connect to Serv-U after being upgraded to 15.3.2. Having either RSA or DSA does not resolve the issue.

    Please can you raise this with Solarwinds support and update the thread?

  • I noticed a pattern w/SW of first blaming the client software, w/out knowing what the error is; namely not reviewing the logs attached to the original ticket, these really tell the story.

    02 Jan 2023 03:46 PM -0800
    Author: Jeffry Proctor [jeffry.proctor@ahf.org]
    Recipient: Customer Service
    Since the client is being logged by Serv-U we can assume that the IP is not on the Windows Server Denied List

    However, we can see in the logs that the connection was accepted on 12/24/2022 at 00:30 ET / 21:30 PT (my time), and then refused shortly after the update

    The attached includes TP Client logs of the client system connecting to our server AFH MFTP running Serv-U mFTP File Server; these logs show before & after and correspond to similar Serv-U Archive logs & DL (daily) logs.

    03 Jan 2023 08:19 AM -0800
    Author: Jeffry Proctor [jeffry.proctor@ahf.org]
    Recipient: technicalsupport@solarwinds.com
    Srinivasa,   Why are you asking for some of the logs already provided in the original ticket?   This is not a global problem but a problem for one account, did you encounter the event noted in the original ticket?     Thanks, JeffP…   From: noreply@salesforce.com on behalf of Support Team - Technical Support Address Sent: Monday, January 2, 2023 4:57:51 PM To: Jeffry Proctor Subject: Case # - 01249694 Immediately after the update to 15.3x one client is being refused connection. [ ref:_00D506e2N._5002J1dJPmM:ref ]   Hi Jeffry, Thank you for contacting SolarWinds Technical Support. My name is Srinivasa, and I will be working on this case with you, rest assured we will be providing you with the best possible resolution on the case. As per my understanding of the issue, I do understand that you are facing an issue after the update to 15.3 one client ...

    ....bla blah, bla

    09 Jan 2023 09:02 AM -0800
    Author: Support Team - Technical Support Address [technicalsupport@solarwinds.com]
    Recipient: jeffry.proctor@ahf.org
    Hello Jeff, Let me verify this to dev team and will get back to you as soon as I have it. Thanks for your patience! Regards, Angeline Camacho SolarWinds | Technical Support My Working Hours: 11:00 AM - 8:00 PM Eastern Time Monday to Friday | Online Support: Submit an Online Support Case | 24x7 Telephone Support: Find regional phone numbers Success Center | Virtual Learning | THWACK --------------- Original Message --------------- From: Jeffry Proctor [jeffry.proctor@ahf.org] Sent: 1/8/2023 12:07 PM To: technicalsupport@solarwinds.com Cc: xing.liu@ahf.org; kmukasa@ramtechinc.com; razmik.marghosian@ahf.org; christian.corado@ahf.org; natalya.adeli@ahf.org Subject: RE: Case # - 01249694 Immediately after the update to 15.3x one client is being refused connection. [ ref:_00D506e2N._5002J1dJPmM:ref ] Angeline,   Can you confirm with the developer that the implementation of RFC 4553 4.2 from 2006, has changed in this build of Serv-U mFTP File Server?   Re: Clients, the applicati ...
    08 Jan 2023 10:07 AM -0800
    Author: Jeffry Proctor [jeffry.proctor@ahf.org]
    Recipient: technicalsupport@solarwinds.com
    Angeline,   Can you confirm with the developer that the implementation of RFC 4553 4.2 from 2006, has changed in this build of Serv-U mFTP File Server?   Re: Clients, the application for both clients is not a trivial one-off client, they are Java client version 3 w/builds JSCH-0.1.54 (SAP/SF) & JSCH-0.1.54=3 (RAM Tech Inc); both clients have been setup since approximately 2017 ~5 years, so as above what exactly changed in this service release?     Thanks, Jeffry Proctor 
  • There's ticket from Jan 2... SW tech road map, 1. blame client, 2. roll-back, 3. blame client, 4. cite RFC, 5. We'll check & get back to you...

  • Do you have any way to check if the suspected CR missing the CRLF is the cause?
     RFC rfc4253#section-4.2 

    And, that doesn't let SW off the hook, if this is the cause, Fair, but they should have included this a warning w/SR Docs

  • I can confirm the same, I have a client with jsch-0.1.53 which no longer works. Also an old client with OpenSSH 4.3 no longer works. Everything else works fine. Tried all options relating to host keys, but agree at this point it does not appear to be key related. 

Reply
  • I can confirm the same, I have a client with jsch-0.1.53 which no longer works. Also an old client with OpenSSH 4.3 no longer works. Everything else works fine. Tried all options relating to host keys, but agree at this point it does not appear to be key related. 

Children