Baby fish don't have the fat reserves that their older relatives do, and are less likely to survive the lack of food during the wintertime. Fish enter a period of dormancy in the wintertime where their metabolism slows down significantly. As food stores are less and less available, fish will utilize the fat and calories they've stored through the rest of the year until theres ready access to food once again. Baby fish haven't had the chance to build these fat stores, and cannot rely on them when other food isn't available.
Ifyou want to be sure your baby fish are protected from the cold and kept from starvation, it may be best to bring the young fish inside until temperatures warm up. Alternatively, you can use a heater to keep your pond warm, but be aware that you'll need to be providing more food for your young fish than your older fish, and any excess food left in the pond can decompose and affect water quality.