Yes, usually, that is the case.
Evolutionarily, our bodies are not really “built” to survive and function optimally much past age 40. But since modern medicine and technology often provide us with a lifespan almost twice that, we have to deal with our various body parts wearing out!
In the human eye, the most common age-related change is a slow hardening and discoloration of the natural lens. This affects near vision by about the mid-40s requiring the use of reading glasses. The clarity and transparency of the natural lens after that slowly degrades and begins to affect distance vision as well.
The central, most sensitive part of the retina is called the macula. Its original shape and architecture also slowly fade even without developing frank pathology such as macular degeneration. There are several other aspects of aging in the human eye that can affect the quality of vision without there being any actual disease or pathology.
In general it is very common to encounter young people who can see several eye chart lines better than 20/20. But we are usually happy to get most middle-aged and older people to the 20/20 line alone.