Posts:
685
Registered:
13-Apr-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 06:38
in response to: davidekholm
|
|
|
David, here we go:
1,014s ==>
AlbumBean.registerVariables: 17113 calls 7m 48,747s
AlbumObjectImpl.getMetadata: 15838 calls 6m 27,289s
AlbumObjectImpl.getXmpManager: 18489 calls 6m 23,727s
Process compiled expression: 765 calls 3m 44,56s
predir processing: 762 calls 3m 35,128s
AlbumObjectImpl.getProperties: 16362 calls 2m 12,943s
AlbumObjectProperties.load: 16100 calls 1m 30,693s
FileFilters.loadImage: 278 calls 44,964s
Executing compiled scriptlet: 100529 calls 29,299s
FileFilters.getBasicImageInfo: 15653 calls 28,091s
AlbumBean.getFolderProperties: 1419 calls 26,59s
AlbumBean.makeIndexPages: 762 calls 22,07s
HardwareSmoothScaler.scale: 264 calls 20,119s
Writing pages: 2327 calls 20,111s
Reading file attributes: 8727 calls 19,498s
AlbumBean.countTotalFiles: 1 calls 18,126s
JAlbumUtilities.countCategories: 6953 calls 11,224s
init: 1 calls 10,569s
JAlbumUtilities.getDeepCameraDates: 762 calls 10,467s
Create MediaRSS: 762 calls 7,725s
FileFilters.saveImage: 356 calls 6,56s
AlbumBean.popVars: 64413 calls 5,25s
AlbumBean.pushVars: 64414 calls 5,165s
Executing interpreted scriptlet: 105169 calls 4,054s
JSONMaker.makeDataPages: 762 calls 3,237s
FileFilters.unifyColorModel: 277 calls 2,853s
Making deep-data.json: 1 calls 1,955s
JAlbumUtilities.countWebLocationCategories: 997 calls 1,743s
Compiling scriptlets: 95769 calls 1,104s
JSONMaker.makeTree: 1 calls 0,879s
Compiling script: 4 calls 0,601s
AlbumBean.processFilters: 722 calls 0,058s
Process expression: 1 calls 0,018s
Copy res files: 1 calls 0,007s
Total: 556754 calls 36m 45,424s
AlbumObjectImpl.getMetadata: 15838 calls 6m 27,289s
Everything from the console during compilation:
https://www.markus-drueck.de/JA/2025%2001%2008%20-%20Compiling%20Project%20Norge.7z
I'll delete that file in about 11 hours.
Thanks! Cheers, Markus
Edited by: MarkusD on 8 Jan 2025, 06:52
|
|
|
Posts:
1,964
Registered:
6-Sep-2005
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 10:14
in response to: davidekholm
|
|
|
Here are the measurements on my PC (Win11, AMD Ryzen 5 5600G @3.90 GHz, 16GB)
jAlbum 36.1b10
Making "Sample Project" (All)
"Sample Project" made in 9.187s
FileFilters.saveImage: 210 calls 7.188s
FileFilters.loadImage: 57 calls 1.133s
HardwareSmoothScaler.scale: 159 calls 0.834s
AlbumBean.registerVariables: 58 calls 0.104s
FileFilters.unifyColorModel: 57 calls 0.086s
AlbumBean.makeIndexPages: 4 calls 0.063s
init: 1 calls 0.04s
Process compiled expression: 7 calls 0.035s
RecoveryTool.createLifeboat: 1 calls 0.034s
Create MediaRSS: 4 calls 0.031s
Executing compiled scriptlet: 63 calls 0.03s
AlbumObjectImpl.getProperties: 57 calls 0.03s
AlbumObjectProperties.load: 13 calls 0.028s
AlbumObjectImpl.getXmpManager: 63 calls 0.023s
AlbumObjectImpl.getMetadata: 57 calls 0.02s
Executing interpreted scriptlet: 368 calls 0.019s
FileFilters.getBasicImageInfo: 57 calls 0.017s
Writing pages: 12 calls 0.017s
Process expression: 1 calls 0.006s
predir processing: 4 calls 0.006s
JAlbumUtilities.countCategories: 8 calls 0.005s
AlbumBean.countTotalFiles: 1 calls 0.005s
AlbumBean.pushVars: 158 calls 0.005s
Reading file attributes: 55 calls 0.005s
AlbumBean.popVars: 157 calls 0.004s
Compiling scriptlets: 46 calls 0.001s
Copy res files: 1 calls 0.001s
AlbumBean.processFilters: 264 calls 0.001s
AlbumBean.getFolderProperties: 5 calls 0s
Total: 1948 calls 9.771s
jAlbum 36.1.2
Making "Sample Project" (All)
"Sample Project" made in 3.033s
0.001s ==>
FileFilters.saveImage: 210 calls 0.953s
HardwareSmoothScaler.scale: 159 calls 0.816s
FileFilters.loadImage: 57 calls 0.451s
AlbumBean.registerVariables: 58 calls 0.117s
AlbumBean.makeIndexPages: 4 calls 0.083s
init: 1 calls 0.062s
Process compiled expression: 7 calls 0.055s
Writing pages: 12 calls 0.048s
Executing compiled scriptlet: 63 calls 0.042s
Create MediaRSS: 4 calls 0.041s
RecoveryTool.createLifeboat: 1 calls 0.036s
AlbumObjectImpl.getProperties: 57 calls 0.035s
AlbumObjectProperties.load: 13 calls 0.03s
AlbumObjectImpl.getXmpManager: 63 calls 0.028s
Executing interpreted scriptlet: 368 calls 0.027s
AlbumObjectImpl.getMetadata: 57 calls 0.023s
FileFilters.getBasicImageInfo: 57 calls 0.022s
Copy res files: 1 calls 0.019s
Process expression: 1 calls 0.009s
AlbumBean.pushVars: 158 calls 0.009s
AlbumBean.countTotalFiles: 1 calls 0.008s
predir processing: 4 calls 0.008s
Reading file attributes: 55 calls 0.008s
AlbumBean.popVars: 157 calls 0.006s
JAlbumUtilities.countCategories: 8 calls 0.004s
Compiling scriptlets: 46 calls 0.001s
AlbumBean.getFolderProperties: 5 calls 0.001s
AlbumBean.processFilters: 264 calls 0.001s
Total: 1891 calls 2.943s
Both are second "Force Make" runs. It's 66% improvement.
|
|
|
Posts:
3,626
Registered:
18-Oct-2002
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 13:01
in response to: JeffTucker
|
|
|
That still leaves me head-scratching a bit about why I've never noticed this message. Could it be because I'm not using any JSON-based skins? Does generating those files account for the overhead?
Not the generation, but the gathering of the data stored within the json files takes time. I'd really need to brush up on using the profiler to get a better understanding on where exactly to optimise. EXIF/xmp reading is a good candidate for optimisation though.
|
|
|
Posts:
3,626
Registered:
18-Oct-2002
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 13:12
in response to: MarkusD
|
|
|
@Markus, I inspected the log file. You have zillions of "Camera date cache miss for ..." warnings. This indicates that jAlbum has to do far more file reading to compile the camera dates than it should have. Do you get the same warnings if you remake the album again?
jAlbum should cache the camera dates to .jalbum/albumfiles.txt. Please open a random .jalbum/albumfiles.txt and pass it to me for inspection.
What skin are you using, and are you using an ordinary hard drive or networked drive or local SSD?
|
|
|
Posts:
3,626
Registered:
18-Oct-2002
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 13:15
in response to: Laza
|
|
|
Awesome data Laza! It seems to go from 9.187s to 3.033s, doesn't it? Isn't that a 200% improvement?
|
|
|
Posts:
685
Registered:
13-Apr-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 13:30
in response to: davidekholm
|
|
|
@Markus, I inspected the log file. You have zillions of "Camera date cache miss for ..." warnings. This indicates that jAlbum has to do far more file reading to compile the camera dates than it should have. Do you get the same warnings if you remake the album again?
Unfortunately, this is not possible. I use so many and such long videos that a complete recompile would take several weeks.
jAlbum should cache the camera dates to .jalbum/albumfiles.txt. Please open a random .jalbum/albumfiles.txt and pass it to me for inspection.
Here we go:
https://www.markus-drueck.de/JA/albumfiles.txt-for-norge-2024.7z
What skin are you using,
Tiger.
and are you using an ordinary hard drive or networked drive or local SSD?
Cheers, Markus
Edited by: MarkusD on 8 Jan 2025, 13:39
|
|
|
Posts:
8,438
Registered:
31-Jan-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 13:34
in response to: davidekholm
|
|
|
Awesome data Laza! It seems to go from 9.187s to 3.033s, doesn't it? Isn't that a 200% improvement?
Back to math class! A 100% improvement would be from 9.187s to 0.000s. A time saving of more than 100% would involve some sort of quantum time travel.
(9.187 - 3.033) / 9.187 = 0.67, or a 67% saving of time.
|
|
|
Posts:
8,438
Registered:
31-Jan-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 13:43
in response to: MarkusD
|
|
|
Unfortunately, this is not possible. I use so many and such long videos that a complete recompile would take several weeks.
David: This is why we need to have Force Remake split into two options, Force Remake Images and Force Remake Videos. Anyone who wants to remake everything can just delete the local album and then Make Album.
|
|
|
Posts:
3,626
Registered:
18-Oct-2002
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 16:47
in response to: MarkusD
|
|
|
Thanks Markus. Your albumfiles.txt files are missing camera dates. I don't know why though.
You don't need to do a "Force rebuild", only another "make album" to test, and you can also test on a smaller project.
|
|
|
Posts:
3,626
Registered:
18-Oct-2002
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 16:48
in response to: JeffTucker
|
|
|
Awesome data Laza! It seems to go from 9.187s to 3.033s, doesn't it? Isn't that a 200% improvement?
Back to math class! A 100% improvement would be from 9.187s to 0.000s. A time saving of more than 100% would involve some sort of quantum time travel.
(9.187 - 3.033) / 9.187 = 0.67, or a 67% saving of time.
Can't you say that 100% faster means twice as fast? The comparison is against the current speed.
|
|
|
Posts:
8,438
Registered:
31-Jan-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 17:11
in response to: davidekholm
|
|
|
100% more apples is twice as many. 100% fewer apples is zero.
ETA: And remember, "fastness" is a bit of a misnomer. You're not trying to get more "fastness." You're trying to use less time.
|
|
|
Posts:
3,626
Registered:
18-Oct-2002
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 22:04
in response to: JeffTucker
|
|
|
100% more apples is twice as many. 100% fewer apples is zero.
I agree, and how is that different to saying 100% faster means twice as fast?
|
|
|
Posts:
685
Registered:
13-Apr-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 22:13
in response to: davidekholm
|
|
|
Thanks Markus. Your albumfiles.txt files are missing camera dates. I don't know why though.
You don't need to do a "Force rebuild", only another "make album" to test, and you can also test on a smaller project.
David, the only thing I do to "compile" an album for years is to press <F9> (which is, as far as I know, "Make album"). So, I don't know what to do to populate the missing camera dates.
Please keep in mind that I do have a lot of "pictures" which are actually only PNG files. They don't have any "camera" dates anyway.
Cheers, Markus
|
|
|
Posts:
8,438
Registered:
31-Jan-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 22:45
in response to: davidekholm
|
|
|
100% more apples is twice as many. 100% fewer apples is zero.
I agree, and how is that different to saying 100% faster means twice as fast?
"Twice as fast" means that it takes 50% less time, not 100% less time.
In some ways, the problem is a linguistic one. There is no measurable thing called a "fast." You can't have 12 fasts, or a kilo of fast, or a liter of fast. So, "100% more fast" does not compute.
Stick to phrasing like "twice as fast," or "three times as fast," and stay away from percentages, and you'll stay out of trouble. 
|
|
|
Posts:
4,008
Registered:
4-Aug-2006
|
|
|
Re: Faster image reading and writing with jpeg-turbo
Posted:
8 Jan 25, 23:42
in response to: MarkusD
|
|
|
Thanks Markus. Your albumfiles.txt files are missing camera dates. I don't know why though.
You don't need to do a "Force rebuild", only another "make album" to test, and you can also test on a smaller project.
David, the only thing I do to "compile" an album for years is to press <F9> (which is, as far as I know, "Make album"). So, I don't know what to do to populate the missing camera dates.
Please keep in mind that I do have a lot of "pictures" which are actually only PNG files. They don't have any "camera" dates anyway.
Cheers, Markus
You could use exiftool to add date information. For a single file setting the dates to 'now' you could use
exiftool pathToFile -allDates=now
exiftool can also batch process files in a directory using -P DIR switches instead of a filename.
Just a thought.
|
|
|
|
Legend
|
|
Forum admins
|
|
Helpful Answer
|
|
Correct Answer
|
|