Worker bees who ingest spores before they are a week old do not digest food well, and are not capable of producing brood food. They skip the brood rearing part of their life and become foragers much sooner. Lifespan is reduced 78% according to Eric C. Mussen 2011.
Young queens that ingest spores are superseded within a month.
The effect on the colony would be loss of brood rearers and depletion of workers. This could seriously damage the colony.
A new queen being replaced means two brood less periods which may reduce the number of winter bees produced and could also damage the colony in terms of honey production and life.