This question is not answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 42 - Pages: 3 [ 1 2 3 | Next ] - Last Post: 16 Oct 17, 12:23 Last Post By: davidekholm Threads: [ Previous | Next ]
rothlisbergers

Posts: 13
Registered: 18-Jan-2009
Hard links for originals
Posted: 15 Sep 17, 08:27
 
  Click to reply to this thread Reply
At some point Java 7 apparently introduced the ability to create hard links via the createLink function. Is there anything that would prevent this from being used when an album includes originals, so that duplicate files aren't created on disk? Thanks.
davidekholm

Posts: 3,472
Registered: 18-Oct-2002
Re: Hard links for originals
Posted: 15 Sep 17, 13:09   in response to: rothlisbergers in response to: rothlisbergers
 
  Click to reply to this thread Reply
rothlisbergers wrote:
At some point Java 7 apparently introduced the ability to create hard links via the createLink function. Is there anything that would prevent this from being used when an album includes originals, so that duplicate files aren't created on disk? Thanks.

This looks like a great idea! I can't see the downside of supporting hard links really. It works on both Windows (NTFS), Mac and Linux, saves disk space and cuts time when making albums. I think we could implement this in future releases. Does anyone have any objections to the use of hard links instead of copies whenever possible?
JeffTucker

Posts: 8,218
Registered: 31-Jan-2006
Re: Hard links for originals
Posted: 15 Sep 17, 14:08   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
Does anyone have any objections to the use of hard links instead of copies whenever possible?

That depends. How is this different from choosing Link rather than *Copy*? What happens if the original is deleted, or moved to an external, removable drive, or to a networked drive? How do typical backup programs (e.g., robocopy, Carbonite) handle it? What happens if you try to copy the entire project to another platform?

There are reasons why we typically tell people to choose Copy.
davidekholm

Posts: 3,472
Registered: 18-Oct-2002
Re: Hard links for originals
Posted: 15 Sep 17, 14:18   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
jGromit wrote:
davidekholm wrote:
Does anyone have any objections to the use of hard links instead of copies whenever possible?

That depends. How is this different from choosing Link rather than *Copy*? What happens if the original is deleted, or moved to an external, removable drive, or to a networked drive? How do typical backup programs (e.g., robocopy, Carbonite) handle it? What happens if you try to copy the entire project to another platform?

There are reasons why we typically tell people to choose Copy.


Hard links are multiple file entries that point to the same "inode" on disk (physical storage I assume). The physical storage doesn't go away until the last reference is deleted. You may for instance create a hard link to a file, then delete the "original" file, and still refer to the same file via the hard link. Hard linked files share the same file attributes too. The only limitation to hard links seem to be that they have to reside on the same physical disk.

Links in jAlbum's world is a simulation of what's commonly called "symbolic links". They are simply entries that point to the real target file via a path. If you delete, rename or move the target file, such links become broken.

I see no reason for not creating hard links for original images in the generated album. It can reduce the album size significantly. We should not however create hard links as one adds images to an album project as editing the original will also edit the hard linked file. This may surprise users and we shouldn't cause surprises.

For more info, see https://stackoverflow.com/questions/185899/what-is-the-difference-between-a-symbolic-link-and-a-hard-link
RobM

Posts: 3,833
Registered: 4-Aug-2006
Re: Hard links for originals
Posted: 15 Sep 17, 14:25   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
Does anyone have any objections to the use of hard links instead of copies whenever possible?
Seen the link to Stackoverflow you posted, have you seen this one, https://superuser.com/questions/302051/how-do-you-create-a-working-hard-link-in-osx
That is on OS X, at least, they can get broken by opening and saving in an application. For OS X I'd much prefer the much smarter alias be supported.
JeffTucker

Posts: 8,218
Registered: 31-Jan-2006
Re: Hard links for originals
Posted: 15 Sep 17, 14:28   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
Ah, yes. I wasn't quite tuned in to the original request (still on the first cup of coffee). We're talking only about what lands in the output directory when one of the Link to originals options is chosen.

But I still see a potential for an unwelcome surprise. I add Pictures/mydog.jpg to my album, choose Link to originals, and Make Album. What's sitting in the output directory is now just a hard link to Pictures/mydog.jpg. So far, so good.

But now I tinker with Pictures/mydog.jpg, and change the cropping. That image now has a different aspect ratio than it did when I made the album. Then, without making the album again, I upload it. See where this is going?
davidekholm

Posts: 3,472
Registered: 18-Oct-2002
Re: Hard links for originals
Posted: 15 Sep 17, 14:59   in response to: RobM in response to: RobM
 
  Click to reply to this thread Reply
RobM wrote:
davidekholm wrote:
Does anyone have any objections to the use of hard links instead of copies whenever possible?
Seen the link to Stackoverflow you posted, have you seen this one, https://superuser.com/questions/302051/how-do-you-create-a-working-hard-link-in-osx
That is on OS X, at least, they can get broken by opening and saving in an application. For OS X I'd much prefer the much smarter alias be supported.

So editing the "original" sometimes have you end up with two files again. That isn't that bad, is it?
davidekholm

Posts: 3,472
Registered: 18-Oct-2002
Re: Hard links for originals
Posted: 15 Sep 17, 15:05   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
jGromit wrote:
Ah, yes. I wasn't quite tuned in to the original request (still on the first cup of coffee). We're talking only about what lands in the output directory when one of the Link to originals options is chosen.

Well, for starters, that seems safest.

But I still see a potential for an unwelcome surprise. I add Pictures/mydog.jpg to my album, choose Link to originals, and Make Album. What's sitting in the output directory is now just a hard link to Pictures/mydog.jpg. So far, so good.

But now I tinker with Pictures/mydog.jpg, and change the cropping. That image now has a different aspect ratio than it did when I made the album. Then, without making the album again, I upload it. See where this is going?


Yes, this may upload a modified image, but it depends on how the image editing software works. I tried editing the original file in PhotoShop. This converted the hard link to a separate file which contained the unedited version so no surprises there.

It effectively looks like hard links are implemented at such a low level that you can consider this as a file system level compression system :-)

(the only caveat I can think if is that all files that are hard linked share file attributes with their common target file)
JeffTucker

Posts: 8,218
Registered: 31-Jan-2006
Re: Hard links for originals
Posted: 15 Sep 17, 15:49   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
Yes, this may upload a modified image....

It's not just that the image will have been modified. It's that the HTML won't know about the change, so in most "traditional" skins (which use height and width attributes on the <img> tag), you'll get a stretched or squeezed image.

...but it depends on how the image editing software works.

Yes, tough to say without testing a bunch of them.

The potential for problems is a fairly "fringe-y" case - I suspect most users would never run into it. It wouldn't cause a problem if the user did something else to the original, like changing the gamma - it's only the "new aspect ratio" situation that's dangerous. And I suspect most users are sticking to linking to scaled images, anyway, which would be untouched by this.

It might be enough to warn users that if they link to originals and change the originals after making the album, they should do a quick Make Album before uploading. Even with a large album, it should run very quickly, since there's no new processing being done - at worst, a few new slide HTML files.

You can guess what I'm going to say next, right? I'd like this to be a settings option somewhere, rather than being hard-coded. You could bury it deep, like Tools, Preferences, Album - it doesn't need to be "front and center" in the project settings.
JeffTucker

Posts: 8,218
Registered: 31-Jan-2006
Re: Hard links for originals
Posted: 15 Sep 17, 16:03   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
BTW, if you hard-code it, anyone using FAT32 on WinXP will be out of luck. I suppose you could do hard links only if the file system supports it. Or maybe it's time to tell WinXP users that they can't go beyond jAlbum 14.something.
RobM

Posts: 3,833
Registered: 4-Aug-2006
Re: Hard links for originals
Posted: 15 Sep 17, 16:26   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
RobM wrote:
davidekholm wrote:
Does anyone have any objections to the use of hard links instead of copies whenever possible?
Seen the link to Stackoverflow you posted, have you seen this one, https://superuser.com/questions/302051/how-do-you-create-a-working-hard-link-in-osx
That is on OS X, at least, they can get broken by opening and saving in an application. For OS X I'd much prefer the much smarter alias be supported.

So editing the "original" sometimes have you end up with two files again. That isn't that bad, is it?

In the answers section the comment 'The reason for this behavior is rather straightforward, and it relates to how files are saved in most Mac OS X applications: Atomically. What happens is that a copy of the file is written to a temporary staging area, and then moved to replace the original file. *This, quite naturally, breaks hard links*'

To me, that indicates the hard link would point to a deleted file and the edited version would no longer be part of the album - so, for example a slide page would have no image. This is not much of a problem as long as the hard links are only ever used in the album directory, not the project directory.

This seems something that would benefit from an extend trial period, so users can try different file types in a project and see how things work.
JeffTucker

Posts: 8,218
Registered: 31-Jan-2006
Re: Hard links for originals
Posted: 15 Sep 17, 16:36   in response to: RobM in response to: RobM
 
  Click to reply to this thread Reply
RobM wrote:
To me, that indicates the hard link would point to a deleted file and the edited version would no longer be part of the album - so, for example a slide page would have no image. This is not much of a problem as long as the hard links are only ever used in the album directory, not the project directory.

Wouldn't that actually be the worst outcome? The finished album contains an original image (on a slide page, for example), but that image is not actually present in the output directory. There was a hard link to it, but now that's pointing to a non-existent file. Or when the original is deleted, does the hard link revert to an actual image file, with the pre-edited version of the image?
RobM

Posts: 3,833
Registered: 4-Aug-2006
Re: Hard links for originals
Posted: 15 Sep 17, 16:45   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
jGromit wrote:
Wouldn't that actually be the worst outcome? The finished album contains an original image (on a slide page, for example), but that image is not actually present in the output directory. There was a hard link to it, but now that's pointing to a non-existent file. Or when the original is deleted, does the hard link revert to an actual image file, with the pre-edited version of the image?
Yes, no, maybe - who knows?
It all depends, I think, on when a user edits the original. If they make the album and upload it, it would be fine. If they then edit the original and remake the album it again, presumably, should be fine - with a new hard link in the output (?). Only if they edit an original but don't remake the album but then upload it again, for whatever reason, would it fail.

Too many possible things to consider at this time of the day, I need coffee ;)
JeffTucker

Posts: 8,218
Registered: 31-Jan-2006
Re: Hard links for originals
Posted: 15 Sep 17, 16:57   in response to: RobM in response to: RobM
 
  Click to reply to this thread Reply
Something else to consider, and test thoroughly - what happens in a skin that has the option to offer the site visitor a zipped file of the images?

Edit: Upon reflection, this shouldn't be an issue. Any skin that's doing this is simply using the file system to fetch image files. The fact that the file system is engaging in some space-saving trickery would be invisible to whatever routine the skin is using.
RobM

Posts: 3,833
Registered: 4-Aug-2006
Re: Hard links for originals
Posted: 15 Sep 17, 17:07   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
Windows and locked files in particular and other OSS in general, what would happen should the original be open in another application? Nothing maybe, as long as you don't do a 'save it just to be safe' operation before closing the other application.
Legend
Forum admins
Helpful Answer
Correct Answer

Point your RSS reader here for a feed of the latest messages in all forums