Windows 11/10, by default, allows all the apps to pin to the Taskbar. Pinning a program to the Taskbar is easy in Windows; simply right-click on any app from the Start menu and choose Pin to Taskbar.

lets you enlarge the preview and change the folder icon style. You still need to deal with a single file preview (in Windows 7, for example, folder thumbnails can show several images ...