Permlink Replies: 11 - Pages: 1 - Last Post: 20-Aug-2019 06:27 Last Post By: jsbeddow Threads: [ Previous | Next ]
jsbeddow

Posts: 6
Registered: 2-Sep-2012
Uploads not completing
Posted: 09-Aug-2019 00:22
  Click to reply to this thread Reply
I am a longtime user of JAlbum (since the Fall of 2012), never having any issue serious enough to ask for support, until now, with our recent paid upgrade to version 18.3.1 and we are also facing issues with uploads failing to complete (often hanging at the last few Kb of a total upload of 375MB). I am including the relevant log trace information, with a few areas that include my site login information now shown as "redacted". Any further details I can provide to assist in troubleshooting I will be happy to provide. BTW, ultimately I was able to upload this site by manually connecting and FTP uploading through FileZilla software.

info.cqs.remotefs.RemoteFSException: com.enterprisedt.net.ftp.FTPException: The FTP client has not yet connected to the server. The requested action cannot be performed until after a connection has been established.
at info.cqs.remotefs.ftp.FtpAdapter.put(FtpAdapter.java:374)
at info.cqs.remotefs.RemoteFSBean.put(RemoteFSBean.java:394)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at info.cqs.remotefs.RemoteFSBean$RemoteFSHandler.invoke(RemoteFSBean.java:510)
at com.sun.proxy.$Proxy15.put(Unknown Source)
at se.datadosen.jalbum.AlbumManifest.write(AlbumManifest.java:127)
at se.datadosen.jalbum.UploadBean.uploadAlbum(UploadBean.java:716)
at se.datadosen.jalbum.JPublishWizard$PublishStep$19.run(JPublishWizard.java:2113)
Caused by: com.enterprisedt.net.ftp.FTPException: The FTP client has not yet connected to the server. The requested action cannot be performed until after a connection has been established.
at com.enterprisedt.net.ftp.FTPClient.checkConnection(FTPClient.java:1081)
at com.enterprisedt.net.ftp.FTPClient.initPut(FTPClient.java:2782)
at com.enterprisedt.net.ftp.FTPClient.putData(FTPClient.java:2916)
at com.enterprisedt.net.ftp.FTPClient.putStream(FTPClient.java:2623)
at com.enterprisedt.net.ftp.FTPClient.put(FTPClient.java:2497)
at com.enterprisedt.net.ftp.FTPClient.put(FTPClient.java:2562)
at info.cqs.remotefs.ftp.FtpAdapter.put(FtpAdapter.java:368)
... 10 more

jAlbum 18.3.1 system console running Groovy (Groovy JAlbum Script Engine)

RemoteFSHandler: Caught java.net.SocketException: Connection reset by peer: socket write error when calling putFile. Reconnecting
RemoteFSHandler: addDiskSpaceListener se.datadosen.jalbum.UploadBean$$Lambda$566/0x0000000800634c40@325db692
RemoteFSHandler: setProtocol FTP (edtFTPj)
RemoteFSHandler: setProgressMonitor se.datadosen.jalbum.UploadBean@c1410b2
RemoteFSHandler: setForceUTF8 true
RemoteFSHandler: setPassiveMode true
RemoteFSHandler: setPort 21
RemoteFSHandler: connect ftp.redacted
RemoteFSHandler: getDirectory
RemoteFSHandler: setDirectory /
RemoteFSHandler: setDirectory /
RemoteFSHandler: getFiles info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: setDirectory /webspace
RemoteFSHandler: getFiles info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: setDirectory /webspace/httpdocs
RemoteFSHandler: getFiles info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: exists /ftp_anonymous/thumbs
RemoteFSHandler: exists /log/thumbs
RemoteFSHandler: exists /tmp/thumbs
RemoteFSHandler: exists /webspace/thumbs
RemoteFSHandler: setDirectory /webspace/httpdocs/ilmprovence.com
RemoteFSHandler: getFiles info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: exists /webspace/httpdocs/ilmprovence.com/thumbs
RemoteFSHandler: exists /webspace/httpdocs/marinpctech.com/thumbs
RemoteFSHandler: setProgressMonitor se.datadosen.jalbum.UploadBean@c1410b2
RemoteFSHandler: setDirectory /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio
RemoteFSHandler: getFiles info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: put java.io.ByteArrayInputStream@529c36e2, timedifftester
RemoteFSHandler: getFiles
RemoteFSHandler: removeFile timedifftester
RemoteFSHandler: getUnmonitored /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/:filetree,
RemoteFSHandler: getFiles /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio, info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: getFiles /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/res, info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: getFiles /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/res/fonts, info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: getFiles /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/res/photosphere, info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: getFiles /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/slides, info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: getFiles /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/thumbs, info.cqs.remotefs.RemoteFSBean$$Lambda$565/0x0000000800632040@2d11ab04
RemoteFSHandler: setDirectory /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio
RemoteFSHandler: removeFile /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/manifest.jmf
RemoteFSHandler: setProgressMonitor se.datadosen.jalbum.UploadBean@c1410b2
RemoteFSHandler: setProgressMonitor ThreadMain upload worker,6,main
RemoteFSHandler: putFile C:\Users\James Beddow\Documents\My Albums\Empress_Yacht_Cruise_2019_Aperio\album\res\all.min.js, /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/res/all.min.js
RemoteFSHandler: setProgressMonitor ThreadUpload worker 2,6,main
RemoteFSHandler: setProtocol FTP (edtFTPj)
RemoteFSHandler: setForceUTF8 true
RemoteFSHandler: setPassiveMode true
RemoteFSHandler: setPort 21
RemoteFSHandler: setProgressMonitor ThreadUpload worker 5,6,main
RemoteFSHandler: setProgressMonitor ThreadUpload worker 6,6,main
RemoteFSHandler: setProgressMonitor RemoteFSHandler: setProtocol FTP (edtFTPj)RemoteFSHandler: setProtocol FTP (edtFTPj)
RemoteFSHandler: setForceUTF8 true
RemoteFSHandler: setPassiveMode RemoteFSHandler: setProgressMonitor ThreadUpload worker 4,6,main
RemoteFSHandler: connect RemoteFSHandler: setProtocol FTP (edtFTPj)
ftp.d1105399-6936.site.myhosting.comRemoteFSHandler: setProgressMonitor ThreadUpload worker 7,6,main, redacted
true
RemoteFSHandler: setPort
RemoteFSHandler: setForceUTF8 ThreadUpload worker 3,6,main
true
RemoteFSHandler: setProgressMonitor ThreadUpload worker 8,6,main
21
RemoteFSHandler: connect RemoteFSHandler: setProtocol
FTP (edtFTPj)
RemoteFSHandler: setForceUTF8 true
RemoteFSHandler: setForceUTF8 RemoteFSHandler: setProtocol RemoteFSHandler: setPassiveMode true
RemoteFSHandler: setPort 21
RemoteFSHandler: connect redacted
true
RemoteFSHandler: setPassiveMode true
RemoteFSHandler: setPort RemoteFSHandler: setProtocol FTP (edtFTPj)
RemoteFSHandler: setForceUTF8 true
RemoteFSHandler: setPassiveMode true
RemoteFSHandler: setPort 21
ftp.d1105399-6936.site.myhosting.comRemoteFSHandler: setPassiveMode , redacted
RemoteFSHandler: connect RemoteFSHandler: setPort 21
FTP (edtFTPj)RemoteFSHandler: connect redacted

redacted

RemoteFSHandler: connect RemoteFSHandler: setForceUTF8 redacted
RemoteFSHandler: setPassiveMode true
RemoteFSHandler: setPort 21
RemoteFSHandler: connect redacted
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
Caught java.net.SocketException: Connection reset by peer: socket write error while uploading C:\Users\James Beddow\Documents\My Albums\Empress_Yacht_Cruise_2019_Aperio\album\res\all.min.js. Attempt 1
java.net.SocketException: Connection reset by peer: socket write error
at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
at java.base/java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.base/java.net.SocketOutputStream.write(Unknown Source)
at java.base/java.io.DataOutputStream.write(Unknown Source)
at java.base/java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.base/java.io.BufferedOutputStream.write(Unknown Source)
at com.enterprisedt.net.ftp.FTPClient.putData(FTPClient.java:2979)
at com.enterprisedt.net.ftp.FTPClient.putStream(FTPClient.java:2623)
at com.enterprisedt.net.ftp.FTPClient.put(FTPClient.java:2591)
at com.enterprisedt.net.ftp.FTPClient.put(FTPClient.java:2552)
at info.cqs.remotefs.ftp.FtpAdapter.putFile(FtpAdapter.java:355)
at info.cqs.remotefs.RemoteFSBean.putFile(RemoteFSBean.java:385)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at info.cqs.remotefs.RemoteFSBean$RemoteFSHandler.invoke(RemoteFSBean.java:526)
at com.sun.proxy.$Proxy15.putFile(Unknown Source)
at se.datadosen.jalbum.UploadWorkers$Worker.uploadFiles(UploadWorkers.java:253)
at se.datadosen.jalbum.UploadWorkers$Worker.uploadFiles(UploadWorkers.java:217)
at se.datadosen.jalbum.UploadWorkers$Worker.run(UploadWorkers.java:167)
Aug 08, 2019 8:07:48 AM se.datadosen.jalbum.AlbumManifest getBytes
CONFIG: Writing manifest
RemoteFSHandler: put java.io.ByteArrayInputStream@6a7f0aa1, /webspace/httpdocs/ilmprovence.com/Empress_Yacht_Cruise_2019_Aperio/manifest.jmf

RemoteFSHandler: isConnected
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
RemoteFSHandler: disconnect
RemoteFSHandler: Caught java.net.SocketException: Socket is closed when calling putFile. Reconnecting
JeffTucker

Posts: 6,878
Registered: 31-Jan-2006
Re: Uploads not completing
Posted: 09-Aug-2019 00:28   in response to: jsbeddow in response to: jsbeddow
  Click to reply to this thread Reply
(Post split from an unrelated thread.)

This is one of those things that falls squarely into "David" territory, and he's on vacation at the moment (either that, or he's run away from home - not sure which).

Glad that FileZilla took care of it, so at least you're not dead in the water.
jsbeddow

Posts: 6
Registered: 2-Sep-2012
Re: Uploads not completing
Posted: 09-Aug-2019 01:20   in response to: JeffTucker in response to: JeffTucker
  Click to reply to this thread Reply
Ok thanks, I will try to wait patiently until David can chime on on a cause and (more importantly) a solution. I will say that have already tried the various other FTP options listed within the JAlbum manage/upload section, as well as disabling the "use zipstreaming when available" option within Tools, Preferences, Publishing, Advanced section as suggested for a user with a similar problem. No luck fixing the issue within JAlbum at the moment, so FileZilla is the only workaround right now.
JeffTucker

Posts: 6,878
Registered: 31-Jan-2006
Re: Uploads not completing
Posted: 09-Aug-2019 01:28   in response to: jsbeddow in response to: jsbeddow
  Click to reply to this thread Reply
Glad you've already tried the other FTP options within jAlbum - that's what I was going to suggest, but it takes an ungodly amount of time when dealing with a large upload like yours. That's at least one thing off the list of culprits.
jsbeddow

Posts: 6
Registered: 2-Sep-2012
Re: Uploads not completing
Posted: 09-Aug-2019 01:30   in response to: JeffTucker in response to: JeffTucker
  Click to reply to this thread Reply
Actually, I just tried again to re-upload within JAlbum using the FTP (ftp4j) upload settings and this time it worked fine, so this may have been a transient issue with my hosting provider, or perhaps my (normally very stable) fiber internet connection (wired to this desktop). I will continue testing.....
davidekholm

Posts: 3,222
Registered: 18-Oct-2002
Re: Uploads not completing
Posted: 10-Aug-2019 14:27   in response to: jsbeddow in response to: jsbeddow
  Click to reply to this thread Reply
Please do. I'm monitoring this (just back from vacation). So far we haven't seen a case where FileZilla repeatedly works when jAlbum doesn't, but I'm fully open to the possibility that there are cases that FileZilla handles better than jAlbum's uploader (that's why we recommend people to try FileZilla as a test). If we find such cases, then I'm fully dedicated to address them.
jsbeddow

Posts: 6
Registered: 2-Sep-2012
Re: Uploads not completing
Posted: 15-Aug-2019 07:27   in response to: davidekholm in response to: davidekholm
  Click to reply to this thread Reply
Thank you David. Unfortunately I can confirm that I am still having trouble uploading "natively" through JAlbum, but with no issues when done in FileZilla. I thought the problem was solved, but it only appears to work when my connection is slightly slowed "artificially" by running through a commercial VPN connection. Let me explain...

I recently (a few months back actually) ungraded to a very fast fiber optic (1Gbps symmetrical) internet connection, and it seems like JAlbum is having trouble dealing with the speed of this connection (for example, a 375MB album takes less than 30 seconds to upload in JAlbum, but curiously much, much slower in FileZilla, perhaps 5-10 minutes or more), however it always hangs right near the end of the process in JAlbum. It may appear to be 99.9% complete, or even nominally report to be 100%, but the verification or handshake that takes place at that moment always fails, and when a directory comparison is made with FileZilla, I will always find one or two files that are incomplete and need to be re-uploaded with FileZilla. Another strange thing: after a few attempts at uploading (3-4 at most) with JAlbum, followed by the associated failure (and in most cases, crash reports that pop up many seconds later), I am effectively locked out of my FTP server for a period of time as JAlbum has "held" and not released all 10 of my available FTP upload connections. This only lasts a matter of another 5 or 10 minutes before the server reports that I can connect again.

What can I do in the way of further log reporting to assist with resolving this?
JeffTucker

Posts: 6,878
Registered: 31-Jan-2006
Re: Uploads not completing
Posted: 15-Aug-2019 12:22   in response to: jsbeddow in response to: jsbeddow
  Click to reply to this thread Reply
jsbeddow wrote:
I recently (a few months back actually) ungraded to a very fast fiber optic (1Gbps symmetrical) internet connection, and it seems like JAlbum is having trouble dealing with the speed of this connection....

Ah, that's a clue! I found that my own web host couldn't handle too many simultaneous connections, whether using FileZilla or the jAlbum uploader, and I had to reduce the number. In short, it's not jAlbum that can't handle the speed - it's your web host.

Take a look at the FileZilla settings - see Edit, Settings, Transfers, Maximum simultaneous transfers. How many is it using?

Now compare to the jAlbum settings - see Tools, Preferences, Publishing, Max simultaneous transfers. How many is it using?

In any event, try reducing the number in jAlbum, and see if your host is happier with that.
jsbeddow

Posts: 6
Registered: 2-Sep-2012
Re: Uploads not completing
Posted: 15-Aug-2019 21:47   in response to: JeffTucker in response to: JeffTucker
  Click to reply to this thread Reply
Thank you jGromit, you were right that it was a clue, but it still wasn't quite that setting. After much experimenting in both FileZilla and JAlbum, it was not a problem of maximum simultaneous connections, but rather that my hosting company seems to have a strong preference for "active" FTP connections rather than the more common "passive" default that most setups seem to advise trying first. Once that was switched over, I was able to achieve fast consistent connections in both programs, even with simultaneous connections turned up to 8. I will consider this problem solved, as I have already spent way too much time testing it. Many thanks to both you and David.
JeffTucker

Posts: 6,878
Registered: 31-Jan-2006
Re: Uploads not completing
Posted: 15-Aug-2019 22:44   in response to: jsbeddow in response to: jsbeddow
  Click to reply to this thread Reply
Ah, I hadn't even thought about that! I'm using a secure connection, which is always active - the passive setting isn't an option.

Glad you've got things working - the rest of us need to set a memory flag about this one.
davidekholm

Posts: 3,222
Registered: 18-Oct-2002
Re: Uploads not completing
Posted: 19-Aug-2019 12:38   in response to: JeffTucker in response to: JeffTucker
  Click to reply to this thread Reply
Interesting. Thanks for getting back on this. First time I hear reports that active works better than passive. In my view, these are only different ways of establishing a connection. Once established, it shouldn't really matter. Perhaps your router has a preference to "active"?
jsbeddow

Posts: 6
Registered: 2-Sep-2012
Re: Uploads not completing
Posted: 20-Aug-2019 06:27   in response to: davidekholm in response to: davidekholm
  Click to reply to this thread Reply
Yes, I suppose it could be a router related issue. On a related note however, I did notice a lack of in-depth explanation/documentation (even in the full version web help/manual for JAlbum, see here:https://jalbum.net/help/en/jAlbum/Upload) for the various FTP connection types offered in JAlbum. Did I just not find them, or are those explanations truly missing? Might be nice to see an update to the docs to clarify what those options mean, and how they might affect users in situations like this. In particular, the two (basic?) FTP options that are appended by (edtFTPj) and (FTP4j) have no further explanation in the program or elsewhere...
Legend
Forum admins
Helpful Answer
Correct Answer

Point your RSS reader here for a feed of the latest messages in all forums