If you have video support enabled in jAlbum, when you add a video file to an album project, rather than a regular JPG file, jAlbum automatically extracts a frame from the video to use as a thumbnail image for it. You can control which frame it uses by opening the image for editing (hover on the thumbnail in the main jAlbum window and choose "Edit"), and using the slider to move to a different frame
You can, however, provide your own image for jAlbum to use as the thumbnail for the video. This could be another image entirely - for example, you could use a still image of the front of your house as the thumbnail for a video taken inside the house.
Let's say your video is called mydog.mp4. Find a JPG that you want to use as the thumbnail for that video. Change the name of the JPG file (or a copy of it) to mydog.thm. Now add (or drag in) both mydog.mp4 and mydog.thm to the album project.
It is best to choose a full-sized JPG image to use as the THM file, rather than a small thumbnail. Some skins use that THM image as a placeholder in the video player, shown while the video is loading and when the video is finished. The Turtle skin might use that image as the "theme" image at the top of the album. So, a full-sized image will produce a much better display.