Is it needed to upgrade all the webpages created by jAlbum when a new version of jAlbum is released to use any new features and bug fixes?
Is "Make Album" enough or is "Make All" needed?
Unless a release note specifically states it is needed, and I don't recall ever having seen one do so, a 'Make Album' is all that is needed. What may interfere with a make album though is having things like Settings/Advanced/General > Process subdirectories turned off and Process only updated subdirectories turned on.
And if an older album is working just fine, and you don't want to make any changes to it, you don't need to re-make it at all. Doing so won't accomplish anything.
Make All is usually misunderstood. It means, "reprocess all of the images and videos, even if there's no reason to." It chews up vast amounts of processing time, and since all the images are now "new," even if they haven't changed by a single pixel, they all have to be uploaded again. Usually the wrong choice.
Make changes is also misunderstood. It means, "regenerate the HTML and CSS, but don't process any images or videos, even if they need to be reprocessed." In fact, I don't think it will process any new images in the album. So, this is an option that's almost never a good idea.