This question is answered.


Permlink Replies: 12 - Pages: 1 - Last Post: 9 Dec 22, 17:08 Last Post By: JeffTucker
JeffTucker

Posts: 8,221
Registered: 31-Jan-2006
The "res" directory is not being cleared
Posted: 9 Dec 22, 01:25
 
  Click to reply to this thread Reply
When an album is built, the res directory should be completely emptied, and then re-created. This is no longer happening. If you switch skins, the res directory is properly cleared out. But if you're using the same skin, it isn't.

In my skins, I'm very selective about what gets written out to res. Only those files that are actually needed, given the current skin settings, land in res. The jAlbum core is now leaving all kinds of debris behind.
davidekholm

Posts: 3,480
Registered: 18-Oct-2002
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 10:33   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
We're more conservative with wiping and recreating files under "res" now as it improves album build times. What's the kind of files that are leftover? Style specific files?
JeffTucker

Posts: 8,221
Registered: 31-Jan-2006
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 10:43   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
We're more conservative with wiping and recreating files under "res" now as it improves album build times.

Well, if you don't fix this, I'll end up changing my skins to do it themselves, in finally.groovy. Delete everything in res, then write out only the files that are needed. So much for album build times.

What's the kind of files that are leftover? Style specific files?

All kinds of files. Take Neptune as an example. There's only a handful of regular res files, but then a bunch that are in the skin's resopt folder. Those are copied to the output only if they're needed by the album. For example, there are SVG files that serve as overlays on folders, videos, and weblocs. But if the user decides not to use those overlays, the SVG's don't get copied to the output. But because of this bug, once those files are used once, they stay there forever, even if the user changes the skin settings.

On top of that, there are 54 navigation SVG's that get "templated" (so they're the right color), and then written out to res. But the skin doesn't do all of them - it processes only the ones that are needed for this album. But now if he changes the skin settings, the output res stays cluttered with files that are not needed for this album.

In short, after doing a few skin settings changes, the album ends up with dozens of useless little files.

This seems like a false "economy," to me.
davidekholm

Posts: 3,480
Registered: 18-Oct-2002
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 11:10   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
I could solve this issue by letting skins flag that they want the current behavior. If not flagged, jAlbum will wipe the files like before.

This said, the DiskCache class can assist with both copying files and cleaning up no longer used files on each album build.
JeffTucker

Posts: 8,221
Registered: 31-Jan-2006
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 11:11   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
BTW, I noticed this because I'm working on a couple of beta releases, and kept finding a file in the output res directory that doesn't even exist in the skin any longer. This sent me hunting through my code to try to figure out why this now-obsolete file wasn't going away.

It never occurred to me that the core, which had been doing this properly for well over a decade, had now been instructed to do it wrong. ;)
JeffTucker

Posts: 8,221
Registered: 31-Jan-2006
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 11:22   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
I could solve this issue by letting skins flag that they want the current behavior. If not flagged, jAlbum will wipe the files like before.

Yes, please. That still means new skin releases, just to add this flag, but it's better than writing a bunch of new code just to get around this wonderful "feature."

This said, the DiskCache class can assist with both copying files and cleaning up no longer used files on each album build.

Not sure I follow you. I'm using DiskCache to take care of the oddball files that users add - WOFF files, Google fonts, music tracks, and so on. But why would I want to use it for files that the skin might need, depending upon settings? That would mean that all of these files would end up getting stored with every project. Seems like overkill.
davidekholm

Posts: 3,480
Registered: 18-Oct-2002
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 11:27   in response to: JeffTucker in response to: JeffTucker
Correct
  Click to reply to this thread Reply
DiskCache keeps track of the files you added to it last time (in a small json database file). So if you, on the next album build, feed it less files, it will delete the ones no longer added from the target folder.

Anyway, I've made a quick core update now (not bumped the version number as it's only been minutes since the last release). With this update you can set this skin property:
cleanResFolder=true
You know what it means ;-)

I'll soon release v29.2. Apart from by default containing many small bug fixes, it also allows you to right click tags/keywords in the keywords bar to rename or delete them from selected images.
JeffTucker

Posts: 8,221
Registered: 31-Jan-2006
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 11:46   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
DiskCache keeps track of the files you added to it last time (in a small json database file). So if you, on the next album build, feed it less files, it will delete the ones no longer added from the target folder.

I'll have to mull that over. It would still involve the same logic (is this resource file needed by this album?), and the files still have to be written to the output. I don't really see what this would buy me. Deleting the now unused files was taken care of very nicely by simply dumping res upon album build. Simple and effective. I'll put up with the extra one second of build time. That's just not where the time is consumed.

Anyway, I've made a quick core update now (not bumped the version number as it's only been minutes since the last release). With this update you can set this skin property:
cleanResFolder=true
You know what it means ;-)

New skin releases, coming to a theater near you. Too bad this is necessary, all to fix something that was working perfectly.

I'll soon release v29.2.

Check your email.
davidekholm

Posts: 3,480
Registered: 18-Oct-2002
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 12:04   in response to: JeffTucker in response to: JeffTucker
 
  Click to reply to this thread Reply
When building over a networked drive using Tiger, we actually shaved off a couple of seconds by avoiding clearing the res folder. It all boils down to just how many res files a skin is using I guess.

The advantages of using DiskCache? These 3 things:
  • It downloads resources from other URLs for you
  • It copies resources from a primary to a secondary folder during album build
  • It keeps a database over currently used resource files and cleans up unused resource files from the primary and secondary folders
JeffTucker

Posts: 8,221
Registered: 31-Jan-2006
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 12:41   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
When building over a networked drive using Tiger, we actually shaved off a couple of seconds by avoiding clearing the res folder. It all boils down to just how many res files a skin is using I guess.

The catch with Tiger is that it's just copying everything to the output res, whether the album needs them or not. You end up with about 50 files, clocking in at roughly 1.5MB. And the res contents really are the same after every build, no matter what skin settings are changed. Chameleon was far worse, believe it or not.

How many users are doing things over networked drives? I'll bet that 90% of users these days are building on a primary SSD, where the time probably can't even be measured accurately, it's so short. On the other hand, FTP is still a "one file at a time" protocol, so uploading a boatload of little files that you don't need is something to avoid.
RobM

Posts: 3,842
Registered: 4-Aug-2006
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 14:44   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
We're more conservative with wiping and recreating files under "res" now as it improves album build times. What's the kind of files that are leftover? Style specific files?
It was a long time ago, but I used to have do extra stuff in init because the res directory got deleted. So I can see an advantage, but sympathise with Jeff about the change in behaviour.

Being able to choose is ideal.
davidekholm

Posts: 3,480
Registered: 18-Oct-2002
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 16:45   in response to: RobM in response to: RobM
 
  Click to reply to this thread Reply
Yes, now the skin developer decides :-)
JeffTucker

Posts: 8,221
Registered: 31-Jan-2006
Re: The "res" directory is not being cleared
Posted: 9 Dec 22, 17:08   in response to: davidekholm in response to: davidekholm
 
  Click to reply to this thread Reply
davidekholm wrote:
Yes, now the skin developer decides :-)

This should be your Prime Directive, on all subjects.
Legend
Forum admins
Helpful Answer
Correct Answer

Point your RSS reader here for a feed of the latest messages in all forums