What's so bad about the Sun? Two things: light and heat.
Light fades labels, which is its commonest problem. However, some minerals are themselves light sensitive, such as the ruby silvers proustite and pyrargyrite. In the presence of light, these deep red sulfosalts react with air to form silver surface layers, which in turn oxidize to black silver oxide. Proustites kept in the dark, even after many years, can be as red and lustrous as the finest cinnabars.
Some sunlight effects are due to its ultraviolet component, so UV-blocking film may be helpful if you want to show your collection in a sunny room. However, UV-blocking film can also shift an object's apparent color; and items can possibly fade in visible light. In general, to preserve collections and labels, use the least illumination necessary, and keep collection display rooms dark when they are not in use.
As anyone who has a car with a black interior knows, the Sun's warmth can be concentrated to an unpleasant level. Some minerals are especially susceptible to quick warming, including sulfur and sphalerite. (In the case of sulfur, even the warmth from holding a crystal in your hand may be enough to crack it.) Keep these out of direct sunlight.