Thread Locked This thread is locked - replies are not allowed.


This question is answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 14 - Pages: 1 - Last Post: 26 Mar 25, 09:38 Last Post By: davidekholm Threads: [ Previous | Next ]
K-Allen

Posts: 47
Registered: 4-Aug-2022
Too many connections?
Posted: 20 May 23, 04:26
 
I did some serious reorganizing of my site today and re-uploaded over a dozen albums. After the 8th one was uploaded (I was doing them manually to reflect their new location on my server) I got a message from my host that said I had too many active connections when I tried the 9th.

I had to exit jAlbum and restart it a couple times to be able to upload everything. I didn't see a way to 'close' the FTP connection(s) and haven't run into this issue when I just click the Upload button once the locations are set/saved.

Not a huge issue now that I know what to look out for, but I was just curious about how the manual upload might differ from the auto-magic.
RobM

Posts: 4,030
Registered: 4-Aug-2006
Re: Too many connections?
Posted: 20 May 23, 10:02   in response to: K-Allen in response to: K-Allen
 
The number of upload connections is set under Preferences>Maximum simultaneous transfers. Lower that should prevent the message, but obviously slow down the uploading somewhat.

For Multimaker
Tasks are processed in a "first-come-first-served" manner, but "make" and "upload" tasks are processed simultaneously as they don't compete for the same computer resources so you may see multiple progress bars in the tasks panel. "make+upload" tasks are however scheduled sequentially so that the "make" part is executed first…
K-Allen

Posts: 47
Registered: 4-Aug-2022
Re: Too many connections?
Posted: 21 May 23, 20:24   in response to: RobM in response to: RobM
 
Ahh.. I didn't even see that in Preferences. Thx for the pointer and thx for the explanation on how the various scenarios are processed.
bholroyd

Posts: 12
Registered: 31-Mar-2013
Re: Too many connections?
Posted: 28 Dec 24, 18:51   in response to: K-Allen in response to: K-Allen
 
I, too, am being blocked by my website host for having too many connections. Their logs show I've got 213-256 connections coming from my ip address and their limit is 200 to prevent some types of DDoS attacks. I had no other apps interacting with my website other than jAlbum at the time. I've always had "max simultaneous transfers" set to 8. I was busy loading multiple albums that day, but asynchronously ie not all at the same time. Is it possible the ftp connections are not being released?
davidekholm

Posts: 3,987
Registered: 18-Oct-2002
Re: Too many connections?
Posted: 31 Dec 24, 12:23   in response to: bholroyd in response to: bholroyd
 
bholroyd wrote:
I, too, am being blocked by my website host for having too many connections. Their logs show I've got 213-256 connections coming from my ip address and their limit is 200 to prevent some types of DDoS attacks. I had no other apps interacting with my website other than jAlbum at the time. I've always had "max simultaneous transfers" set to 8. I was busy loading multiple albums that day, but asynchronously ie not all at the same time. Is it possible the ftp connections are not being released?

Anything is possible in theory, but the connections should really be released by jAlbum, and I doubt that it's a bug in the jAlbum end. If it was, we'd have numerous reports on it. Those stats seem to indicate that some other middle-man software is clinging onto the connections. Check any software firewall and update + restart your router for instance.
grockle

Posts: 9
Registered: 4-Oct-2007
Re: Too many connections?
Posted: 19 Mar 25, 15:37   in response to: davidekholm in response to: davidekholm
 
Since about 1 year now I have been having exactly the same behaviour, the ftp server has a limit of 2 apparently. Sometimes it works, sometimes not, it started with the releases last year after having worked over 10 years with multiple releases. 37.2 still has the same problem this morning.
davidekholm

Posts: 3,987
Registered: 18-Oct-2002
Re: Too many connections?
Posted: 19 Mar 25, 18:25   in response to: grockle in response to: grockle
 
If your ftp host limits to 2 connections, either switch to sftp or switch ftp provider entirely.
wziegler

Posts: 31
Registered: 26-Nov-2023
Re: Too many connections?
Posted: 20 Mar 25, 02:22   in response to: davidekholm in response to: davidekholm
 
Interesting - I did not know that several FPT connections can be utilized. Following this discussion, I saw mine max number was set to 8. My hoster allows 3 simultaneous sessions, I learned today.

I also noticed at some point a year ago that my uploads cause access to my domain being locked for 20 min or so (no error message). It always happens during update of large albums (thousands of files and >10GB) for which most files exist. Usually I believe there is a file with this information in which case the uploader comes back with the message how large the album is quickly. In some cases it seems to go through all remote folders site to check (one sees the size counter incrementing slowly) - it is then when access is blocked by the time the counter reaches about 800MByte.

I tried everything suggested here without any change (except the max connections). I routinely exchange large archives and other files without problem using WinSCP.

Since I did not know where to start to debug, I use a workaround whenever this occurred: Delete all files in that album on the server, then the upload takes a lot longer, but completes successfully.

Which file contains that information - I'd like to deliberately force this behavior and see if a lockup still occurs after setting max number to 3.
Wolfgang
grockle

Posts: 9
Registered: 4-Oct-2007
Re: Too many connections?
Posted: 20 Mar 25, 14:12   in response to: wziegler in response to: wziegler
 
I just checked mine, it was also set to 8 as default, I reset it to 2 which is the value recommended for my webserver hoster and lo and behold, the problem disappeared and not only that, but the upload in background was faster than before. No errors anymore.
Why the default in jAlbum is set to 8 I don't know? Maybe it should be set to 2?
grockle

Posts: 9
Registered: 4-Oct-2007
Re: Too many connections?
Posted: 20 Mar 25, 14:16   in response to: davidekholm in response to: davidekholm
 
I've been using the same provider for 11 years now, there are webservices, and numerous mailing systems and other services set up on it. I also want to use the jAlbum integrated FTP
make and upload in background, I don't want to go to FileZilla or another tool to upload the jAlbum photos. But now I have found that setting the jAlbum value to 2 not only removes the problem but actually speeds up the upload, I am more than happy.
wziegler

Posts: 31
Registered: 26-Nov-2023
Re: Too many connections?
Posted: 21 Mar 25, 17:44   in response to: grockle in response to: grockle
 
Setting the 'max simultaneous transfers' to 3 solved the problem I had.

After setting the maximum to 3 (max allowed by hoster), I deleted the manifest file to trigger the problem I had. On the next upload jAlbum is going through the remote site to compute the total size and decide what needs to be updated - the remote site would block access after a while during this process (for the default max setting of 8). Now the lock-up problem does not occur any more.
I did not know that multiple connections are used by jAlbum and did not know that my service provider limits the number to 3.
davidekholm

Posts: 3,987
Registered: 18-Oct-2002
Re: Too many connections?
Posted: 24 Mar 25, 09:14   in response to: wziegler in response to: wziegler
 
Thanks for getting back on this. Good that we found a workaround. It's a very limiting workaround though as the very nature of the ftp protocol impacts speed a lot for the transfer of many small files (like jAlbum albums) over few connections. jAlbum should naturally not lock up if it encounters this problem (it has code to handle such scenarios), but that said, I still think you should investigate using sftp or switching web host provider. Naturally ask them if they can increase this harsh limitation.
davidekholm

Posts: 3,987
Registered: 18-Oct-2002
Re: Too many connections?
Posted: 24 Mar 25, 09:18   in response to: davidekholm in response to: davidekholm
 
If you can get your web host to provide us a test account, we'll make further tests to see whether we can improve jAlbum's behaviour when encountering this connection limitation during album size calculation.
wziegler

Posts: 31
Registered: 26-Nov-2023
Re: Too many connections?
Posted: 25 Mar 25, 17:39   in response to: davidekholm in response to: davidekholm
 
David,
just to clarify: jAlbum does not lock up itself, I can cancel out of the blocked upload connection to my site. It appears that during the initial step of validating the files on the server, the hoster blocks access for about 30min if the max connection is set to 8. If it is set to their max of 3 it will not be blocked and yes, lots of small files will be uploaded slowly at 300kBps average.

Following your recommendation, I learned how to enable SSH and connect with SFTP (different port) and now the upload goes at full access speed! Is the max connection parameter also relevant for SFTP?

It would be nice if the hoster would provide some feed-back if this happens, but during a lock-up the site's dashboard shows: all is ok. I do not think it is something you need to handle from your side, except maybe informing users of this potential problem if they have less than 8 (default) number of connections.

As this is a hobby, I need to keep service provider cost low. Hosting here can be very expensive, my previous provider upped from $35/year to ~ $300 after they merged with someone else. I kept the collections offline for some years, but last year we got internet in our rural area and decided to have the collections online again.

Wolfgang
davidekholm

Posts: 3,987
Registered: 18-Oct-2002
Re: Too many connections?
Posted: 26 Mar 25, 09:38   in response to: wziegler in response to: wziegler
 
Yes, the # of connections applies to sftp a well, but with our hosting we're using a special scheme we call "zipstreaming" that uses only a single connection but without any of the latency due to negotiating file names between each transfer that you typically get with ordinary ftp and sftp. Normal sftp is however better than ordinary ftp in that it uses 1 single socket connection for both data and communication instead of two for ftp.
Legend
Forum admins
Helpful Answer
Correct Answer

Point your RSS reader here for a feed of the latest messages in all forums