ANOTHER all images rebuilt upon upload ...

davidekholm

Posts: 4,072
Registered: 18-Oct-2002

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 9 May 23, 15:00

in response to: jimbella

Reply

jimbella wrote:

Close the upload window and reattempt the upload. It should now, after some initial file comparison
(could take minutes) ONLY upload new and changed files. Does it?

I'm afraid not, attempt umpteen to upload 2.47 GB...

I take it you HAVE deleted the manifest file as well?
If so, can you make available a zip file of your album so I can see if I can reproduce the problem with it

SharpByCoop

Posts: 25
Registered: 2-Sep-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 16 May 23, 00:02

in response to: davidekholm

Reply

NEW Upload: All set, but with a hiccup:

I needed to do my monthly update. I added about 100 new files and created new albums, etc.

All set, I rebuilt it as asked.

(Note: UPDATE available. I didn't go near it. If it works don't fix it. This has been problematic in the past. Now that it is uploaded, I would update.)

It took about 3 mins to build. I have 12k+ images. 15.99GB worth!

Upload went ok, and was cruising along the timeline bar to finish in 2 mins and then it STOPPED. (See error code below.)

It booted me back to begin upload, and then I did it again, and it ran through quickly and completed OK 2nd time. Unsure what the hiccup was.

http://www.knifegallery.com/index.html

David, should I upgrade?

********
com.enterprisedt.net.ftp.ControlChannelIOException: Read timed out
at com.enterprisedt.net.ftp.FTPControlSocket.readLine(FTPControlSocket.java:1090)
at com.enterprisedt.net.ftp.FTPControlSocket.readReply(FTPControlSocket.java:1150)
at com.enterprisedt.net.ftp.FTPControlSocket.sendCommand(FTPControlSocket.java:1049)
at com.enterprisedt.net.ftp.FTPControlSocket.createDataSocketPASV(FTPControlSocket.java:876)
at com.enterprisedt.net.ftp.FTPControlSocket.createDataSocket(FTPControlSocket.java:632)
at com.enterprisedt.net.ftp.FTPClient.createDataSocket(FTPClient.java:2756)
at com.enterprisedt.net.ftp.FTPClient.initPut(FTPClient.java:2819)
at com.enterprisedt.net.ftp.FTPClient.putData(FTPClient.java:2916)
at com.enterprisedt.net.ftp.FTPClient.putStream(FTPClient.java:2623)
at com.enterprisedt.net.ftp.FTPClient.put(FTPClient.java:2497)
at com.enterprisedt.net.ftp.FTPClient.put(FTPClient.java:2562)
at info.cqs.remotefs.ftp.FtpAdapter.put(FtpAdapter.java:383)
at info.cqs.remotefs.RemoteFSBean.put(RemoteFSBean.java:404)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at info.cqs.remotefs.RemoteFSBean$RemoteFSHandler.invoke(RemoteFSBean.java:520)
at jdk.proxy2/jdk.proxy2.$Proxy11.put(Unknown Source)
at se.datadosen.jalbum.AlbumManifest.write(AlbumManifest.java:128)
at se.datadosen.jalbum.UploadBean.uploadAlbum(UploadBean.java:904)
at se.datadosen.jalbum.JPublishWizard$PublishStep$19.run(JPublishWizard.java:2063)

Edited by: SharpByCoop on 16 May 2023, 00:02

davidekholm

Posts: 4,072
Registered: 18-Oct-2002

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 16 May 23, 12:28

in response to: SharpByCoop

Reply

Seems jAlbum got stuck writing the manifest file. If this leaves us with an empty or truncated manifest file, then that may be the cause of your long uploads. IF you get these long uploads again, cancel the upload, then connect to the server and download the (assumed) existing "manifest.jmf" file from the album's root folder. Now delete the "manifest.jmf" file and retry. Now the upload should be fast after an initial file comparison operation.

Another thing you can try is to open Tools->Upload/Manage and switch "Connection type" from "ftp (edtftpj)" to "ftp (ftp4j)". This changes the client side ftp library to another brand, which may play better with you server.

Otherwise, I suspect a shaky internet connection. Reset your router for starters.

JeffTucker

Posts: 8,035
Registered: 31-Jan-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 16 May 23, 14:38

in response to: davidekholm

Reply

I would also suggest switching to SFTP, if the web host supports it. I've always found that to be more robust, and reliable.

Just be aware that SFTP requires a different port number setting, and each web host is inclined to "do its own thing." Check with your web host to get the connection settings for SFTP.

The cable modem or router might also be the culprit. If your cable modem is more than a few years old, see if you can get your provider to swap it out. If you tell them you're having uploading problems, they're often willing to just send you a new modem, rather than incurring the expense of "rolling a truck." Routers don't live forever, either.

Lots of links in the chain.

davidekholm

Posts: 4,072
Registered: 18-Oct-2002

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 17 May 23, 12:04

in response to: JeffTucker

Reply

FYI: I've been doing tests where I randomly truncate the "manifest.jmf" file in order to simulate a connection loss during transfer. "luckily" jAlbum detects this and thereby ignores the manifest file, so we're still left in a mystery as to why jAlbum sometimes uploads all files.

(the manifest.jmf file is actually a zip file and the unzipper chokes on prematurely terminated content)

SharpByCoop

Posts: 25
Registered: 2-Sep-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 27 Jun 23, 14:40

in response to: SharpByCoop

Reply

	Screen Shot 2023-06-26 at 9.59.01 AM.png (260.8 KB)
	Screen Shot 2023-06-26 at 9.59.26 AM.png (264.9 KB)

So I have left things alone for a month but I had a show so I needed to update.

I broke my cardinal rule and updated the Jalbum software to the latest before rebuild. (31.1.2)

And..... it built the Album with 150 new files quickly.

Upload started OK then...BAM! it is rebuilding everything. SO I used a file transfer program (Fetch) and deleted the Manifest file. Twice. Didn't change a thing.

So I let it run it's thing. Five upload hours later it rebuilt with the album of 150 new images.

I need to add more this week. Fingers crossed. I have no answers. Internet seems to be fine here. (Speedtest: 216 download, 10mb upload.)

davidekholm

Posts: 4,072
Registered: 18-Oct-2002

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 28 Jun 23, 17:24

in response to: SharpByCoop

Reply

SharpByCoop wrote:
So I have left things alone for a month but I had a show so I needed to update.

I broke my cardinal rule and updated the Jalbum software to the latest before rebuild. (31.1.2)

And..... it built the Album with 150 new files quickly.

Upload started OK then...BAM! it is rebuilding everything. SO I used a file transfer program (Fetch) and deleted the Manifest file. Twice. Didn't change a thing.

So I let it run it's thing. Five upload hours later it rebuilt with the album of 150 new images.

I need to add more this week. Fingers crossed. I have no answers. Internet seems to be fine here. (Speedtest: 216 download, 10mb upload.)

Super confusing and naturally a bad experience. If you can pinpoint what triggers jAlbum to re-upload all images all of a sudden, let me know!
Note: You don't have to use other upload software in order to delete the manifest file. Just hit the "Advanced" button in jAlbum's toolbar and it will reveal files as well.

If/when you encounter this problem again, abort the upload and use FileZilla (or similar tools) to compare file dates and file sizes of the local and remote files. jAlbum should only re-upload files where either the size differs or the local date > remote date.

JeffTucker

Posts: 8,035
Registered: 31-Jan-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 28 Jun 23, 17:49

in response to: davidekholm

Reply

The bigger mystery is how an uploading problem could trigger a forced remake of the album, which seems to be what's being reported here. These are unrelated things.

JeffTucker

Posts: 8,035
Registered: 31-Jan-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 28 Jun 23, 17:50

in response to: davidekholm

Reply

davidekholm wrote:
You don't have to use other upload software in order to delete the manifest file. Just hit the "Advanced" button in jAlbum's toolbar and it will reveal files as well.

To clarify, choose Tools > Upload/Manage > Connect, then:

SharpByCoop

Posts: 25
Registered: 2-Sep-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 28 Jun 23, 23:46

in response to: JeffTucker

Reply

OK. I need to upload again very soon. I'll see if I can record the screen as it does so so we may see the trigger. I'll do it late in the eve so if it HAS to run long, it can.

Did not know about that advanced tab and files. Will look into it.

I will find out file date info to see what actually IS being 'overwritten'. I think I did this before and they were earlier versions. So confusing.

Sorry about this, yet if I can figure it out it will save others. Or me....

SharpByCoop

Posts: 25
Registered: 2-Sep-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 29 Jun 23, 03:32

in response to: SharpByCoop

Reply

Screen Shot 2023-06-28 at 9.21.24 PM.png (225.7 KB)

OK, fresh upload. I even screen recorded the process.

(Spoiler: it's fine)

I deleted and then added about 180 new files and arranged them.

Built the new album in about three mins on my computer. Normal.

Upload: it reviewed and said it had 194mb of new files. That's good. Hit GO.

First upload went halfway through and then hung up on a file. Quickly it aborted time out.

Began again, and this time it claimed to have only 123mb of new files, so it recognized what had been uploaded prior before gltch?

Anyway, it did not flag for a FULL rebuild, and it went all the way through SUCCESSFULLY in about four mins. Opened to a fresh website.

David, my experience has been that when I update to a new version, it has to do the whole thing over. That's how I began two days ago. I know, this SHOULD NOT happen, but it happens often.

So I rarely update (sidestepping your hard work). Don't spoil a good thing. HA!

Lastly, the Upload button has TWO settings: 'Smart Upload' and 'Upload ALL'. The default appears to be Upload All. Is this a concern?

Jeff and Dave: Thanks for helping.

Jim

Edited by: SharpByCoop on 29 Jun 2023, 03:34

JeffTucker

Posts: 8,035
Registered: 31-Jan-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 29 Jun 23, 03:48

in response to: SharpByCoop

Reply

SharpByCoop wrote:
Lastly, the Upload button has TWO settings: 'Smart Upload' and 'Upload ALL'. The default appears to be Upload All. Is this a concern?

Where are you seeing that?

ETA: Please don't crop so much of the screenshot that we can't tell what's going on. In short, I have never seen those options on the upload button, so I'm very puzzled, indeed.

ETA: Finally found the button that has those two choices. David, is this a holdover from some earlier version? SharpByCoop, just click the big Upload button on the lower left of the main jAlbum window. You appear to be going into "managing" the upload, and I don't understand why. The regular upload gives you ample opportunity to change the upload location, confirm what you're about to do, and so on. And once you've set the proper upload location, and saved the project, it will remember it. You don't have to keep resetting it.

JeffTucker

Posts: 8,035
Registered: 31-Jan-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 29 Jun 23, 04:00

in response to: SharpByCoop

Reply

SharpByCoop wrote:
Upload: it reviewed and said it had 194mb of new files. That's good. Hit GO.

This is yet another mystery. The upload routine shows only the total size of the album. It doesn't provide any indication of how many new files there are, or of how many MB of stuff it's going to upload, until it actually starts carrying out the upload.

Are you running a very old version of jAlbum? Many of the things you're describing simply aren't present in the current version.

davidekholm

Posts: 4,072
Registered: 18-Oct-2002

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 29 Jun 23, 11:05

in response to: JeffTucker

Reply

I honestly only hit the center of the "Upload" button each time and let jAlbum decide what's best.

Reason to use "Upload all":
The decision process can be slow in case the manifest file is missing as all files needs to be compared prior to the upload. If you use "Upload all", it should skip the decision process and just upload everything as most other ftp clients do.

JeffTucker

Posts: 8,035
Registered: 31-Jan-2006

Re: ANOTHER all images rebuilt upon upload thread.
Posted: 29 Jun 23, 14:55

in response to: davidekholm

Reply

davidekholm wrote:
I honestly only hit the center of the "Upload" button each time and let jAlbum decide what's best.

Me, too. That's why I didn't even recognize the options in the OP's screenshot. I hit the little down-arrow on the button only to choose the host, since I have a jAlbum account, two domains of my own, and one subdomain that I use.

Reason to use "Upload all":
The decision process can be slow in case the manifest file is missing as all files needs to be compared prior to the upload. If you use "Upload all", it should skip the decision process and just upload everything as most other ftp clients do.

Unless most of the files are, in fact, new, the "smart" upload should almost always be faster, and by quite a bit. In FileZilla, I have never turned off the option to upload only new or changed files.

But the existence of that option suggests that the OP's problem is that somewhere, somehow, that "upload all" option is being automatically triggered by a failure of the smart upload. It shouldn't be, of course, but that's the only explanation I can think of for the "suddenly uploading everything" phenomenon.