Windows 11/10 displays a file icon on the thumbnail ... For that, search for file explorer options (earlier called Folder Options) in the Taskbar search box and click on the individual search ...

Windows 11/10, by default, allows all the apps to pin to the Taskbar. Pinning a program to the Taskbar is easy in Windows; simply right-click on any app from the Start menu and choose Pin to Taskbar.

lets you enlarge the preview and change the folder icon style. You still need to deal with a single file preview (in Windows 7, for example, folder thumbnails can show several images ...