6 Replies Latest reply on Aug 5, 2013 9:55 AM by dbladorn

    Trouble sending large files to a Pure-FTPd Server

    rich.silva

      I have a connection set up to a server, all that is fine...

      I can send smaller files just fine...

       

      Larger files? It seems that the Command channel times out...

      In the header of the Welcome message is a warning of a timeout at 3 minutes of inactivity...

      So I dutifully setup a 150 second Keep Alive...

       

      What I see in the log is...

      The file (11GB+) sends fine...(Although there seems to be many reconnections during the transfer)

      I will see an OPTS MLST command send

      followed by a PWD and then a reconnection attempt...

       

      Finally I will see a "Transfered <many> bytes STATUS message... Followed by an XCRC - which seems to receive no response after maybe 30 seconds...

      Then another PWD...

      Then an ERROR> Server has unexpectedly closed the connection...

       

      And we start again....

       

      Ideas?

        • Re: Trouble sending large files to a Pure-FTPd Server
          Peter Krutý

          Hi rich.silva,

          can you please share with us the actual log file?

          Thanks, Peter

            • Re: Trouble sending large files to a Pure-FTPd Server
              rich.silva

              Here is one...

              STATUS> Connecting to "97.74.141.1" on port 21.

              STATUS> Connected to 97.74.141.1:21 from 10.0.7.68:25716.

              220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------

              220-You are user number 16 of 500 allowed.

              220-Local time is now 12:39. Server port: 21.

              220-This is a private system - No anonymous login

              220 You will be disconnected after 3 minutes of inactivity.

              COMMAND> HOST 97.74.141.1

              530 You aren't logged in

              STATUS> Connected. Logging in to the server.

              COMMAND> USER cadasysprousa

              331 User cadasysprousa OK. Password required

              COMMAND> PASS ***

              230 OK. Current restricted directory is /

              STATUS> Login successful.

              COMMAND> SYST

              215 UNIX Type: L8

              COMMAND> FEAT

              211-Extensions supported:

              EPRT

              IDLE

              MDTM

              SIZE

              MFMT

              REST STREAM

              MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;

              MLSD

              AUTH TLS

              PBSZ

              PROT

              TVFS

              ESTA

              PASV

              EPSV

              SPSV

              ESTP

              211 End.

              STATUS> Server supports resume.

              COMMAND> OPTS MLST Type;Size;Modify;UNIX.mode;UNIX.owner;UNIX.group;

              200  MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;

              COMMAND> PWD

              257 "/" is your current location

              COMMAND> PASV

              227 Entering Passive Mode (97,74,141,1,198,214)

              STATUS> Opening passive mode data connection to 97.74.141.1, port 50902.

              COMMAND> MLSD

              150 Accepted data connection

              226-Options: -a -l

              226 6 matches total

              STATUS> Transferred 782 bytes. 3.50 KB/sec.

              STATUS> Connecting to "97.74.141.1" on port 21.

              STATUS> Connected to 97.74.141.1:21 from 10.0.7.68:25745.

              220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------

              220-You are user number 16 of 500 allowed.

              220-Local time is now 12:40. Server port: 21.

              220-This is a private system - No anonymous login

              220 You will be disconnected after 3 minutes of inactivity.

              COMMAND> HOST 97.74.141.1

              530 You aren't logged in

              STATUS> Connected. Logging in to the server.

              COMMAND> USER cadasysprousa

              331 User cadasysprousa OK. Password required

              COMMAND> PASS ***

              230 OK. Current restricted directory is /

              STATUS> Login successful.

              COMMAND> SYST

              215 UNIX Type: L8

              COMMAND> FEAT

              211-Extensions supported:

              EPRT

              IDLE

              MDTM

              SIZE

              MFMT

              REST STREAM

              MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;

              MLSD

              AUTH TLS

              PBSZ

              PROT

              TVFS

              ESTA

              PASV

              EPSV

              SPSV

              ESTP

              211 End.

              STATUS> Server supports resume.

              COMMAND> OPTS MLST Type;Size;Modify;UNIX.mode;UNIX.owner;UNIX.group;

              200  MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;

              COMMAND> PWD

              257 "/" is your current location

              COMMAND> MLST /FiveStar_7.bak_Voyager

              550 Can't check for file existence

              STATUS> Sending: C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Backup\FiveStar_7.bak_Voyager -> /FiveStar_7.bak_Voyager

              COMMAND> TYPE I

              200 TYPE is now 8-bit binary

              COMMAND> PASV

              227 Entering Passive Mode (97,74,141,1,196,196)

              STATUS> Opening passive mode data connection to 97.74.141.1, port 50372.

              COMMAND> STOR FiveStar_7.bak_Voyager

              150 Accepted data connection

              COMMAND> NOOP

              200 Zzz...

              COMMAND> PWD

              257 "/" is your current location

              COMMAND> NOOP

              200 Zzz...

              COMMAND> PWD

              257 "/" is your current location

              COMMAND> NOOP

              421 Timeout (no operation for 360 seconds)

              226-File successfully transferred

              226 2194.358 seconds (measured here), 4.91 Mbytes per second

              STATUS> Transferred 11,299,543,552 bytes. 5,028.64 KB/sec.

              COMMAND> XCRC "/FiveStar_7.bak_Voyager"

              ERROR> The server has unexpectedly closed the connection.

              ERROR> Retrying failed transfer (attempt 2 out of 3)...

              STATUS> Connecting to "97.74.141.1" on port 21.

              STATUS> Connected to 97.74.141.1:21 from 10.0.7.68:26713.

              220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------

              220-You are user number 23 of 500 allowed.

              220-Local time is now 13:17. Server port: 21.

              220-This is a private system - No anonymous login

              220 You will be disconnected after 3 minutes of inactivity.

              COMMAND> HOST 97.74.141.1

              530 You aren't logged in

              STATUS> Connected. Logging in to the server.

              COMMAND> USER cadasysprousa

              331 User cadasysprousa OK. Password required

              COMMAND> PASS ***

              230 OK. Current restricted directory is /

              STATUS> Login successful.

              COMMAND> SYST

              215 UNIX Type: L8

              COMMAND> FEAT

              211-Extensions supported:

              EPRT

              IDLE

              MDTM

              SIZE

              MFMT

              REST STREAM

              MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;

              MLSD

              AUTH TLS

              PBSZ

              PROT

              TVFS

              ESTA

              PASV

              EPSV

              SPSV

              ESTP

              211 End.

              STATUS> Server supports resume.

              COMMAND> OPTS MLST Type;Size;Modify;UNIX.mode;UNIX.owner;UNIX.group;

              200  MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;

              COMMAND> PWD

              257 "/" is your current location

              COMMAND> MLST /FiveStar_7.bak_Voyager

              250-Begin

              type=file;size=11299543552;modify=20130731201710;UNIX.mode=0604;UNIX.uid=10059032;UNIX.gid=100450;unique=17g382b9b9; /FiveStar_7.bak_Voyager

              250 End.

              COMMAND> MLST /FiveStar_7 (2).bak_Voyager

              550 Can't check for file existence

              STATUS> Sending: C:\Program Files\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\Backup\FiveStar_7.bak_Voyager -> /FiveStar_7.bak_Voyager

              COMMAND> TYPE I

              200 TYPE is now 8-bit binary

              COMMAND> PASV

              227 Entering Passive Mode (97,74,141,1,197,176)

              STATUS> Opening passive mode data connection to 97.74.141.1, port 50608.

              COMMAND> STOR FiveStar_7.bak_Voyager

              150 Accepted data connection

              COMMAND> ABOR

              426 Transfer aborted

              226-Transfer aborted

              226 5.538 seconds (measured here), 4.69 Mbytes per second

              COMMAND> QUIT

              221-Goodbye. You uploaded 26583 and downloaded 0 kbytes.

              221 Logout.

                • Re: Trouble sending large files to a Pure-FTPd Server
                  rich.silva

                  The odd thing to me is that the Transfer (physical) actually completes...

                  Someplace around the XCRC things go awry and I get kicked off the server (although there are/were many reconnections).

                   

                  Then a later MLST command seems to fail as well, even though the File does (seem to) exist.. And then a retry occurs and we start again... (I have never tried letting all three retries complete to see if the file is actually left behind... That might be interesting).

                   

                  Other things I have tried...

                  Shortened the Keep Alive interval...

                  Manually Reconnected just before the transfer was to complete...

                    • Re: Trouble sending large files to a Pure-FTPd Server
                      dbladorn

                      According to the FEAT command, the server you're connecting to doesn't support the XCRC command.  You can try turning that off in FTP Voyager.  Click the "Advanced Options" button within the Site Profile manager; then select the "Connection" section and locate the "File Transfer" settings.  Change "Use Global Settings" to "No", then change "XCRC Checking" to "Disabled".  Please let me know if that helps.

                        • Re: Trouble sending large files to a Pure-FTPd Server
                          rich.silva

                          Thanx for the hint… I made the change you suggested, but still get the unexpected close from the Server…

                           

                          This time I see the “Transferred  bytes…

                          Then a MFMT command is sent…

                          Then I see the Server closed the connection error…

                           

                          I still have a bit of confusion going on with what commands I see sent…

                          When the connection is reconnected… (amongst other things) I see an MLST command that seems to have a reasonable result (a size, modify time, etc is returned)… Then immediately the same MLST command is sent (why?) and then the next return is a “550 Cant check for existence”…

                          And the Retry begins…

                           

                          Also, probably a Red Herring, but the Keep Alive isn’t doing anything useful… Even though a NOOP is sent at 120 seconds, at 360 seconds, a timeout response is received “No activity for 360 seconds”