I have in the past gotten around this by keeping the folder icon as the thumbnail image and putting the "folder representing image" as the thumbnail background.
For more flexibility, I've taken a different approach. I trot out the regular thumbnail image, but with
position: relative;. Then I use an absolutely positioned image for the folder icon. That lets me adjust the position at will (even having it stand proud of the corner of the thumbnail), alter its size, alter its opacity, etc. Once you get it set up, tinkering with it becomes easy.
I agree that this would be a useful feature - some of the latest responsive skins, for example, have no indication of "folder-ness," and because of the way the layout is generated, adding it with HTML and CSS is not really possible.
It does, however, make it more important to have this old bug fixed: currently, if you select/de-select the video overlay,
Make Album doesn't regenerate the thumbnail images. I've sometimes had to resort to manually deleting the output thumbnails. That wasn't a big deal because users would rarely change the setting. But with a folder overlay, the problem would become a lot worse.