Motivated by systems in which droplets grow and shrink in a turbulence-driven supersaturation field, we investigate the problem of turbulent condensation in a general manner. Using direct numerical simulations, we show that the turbulent fluctuations of the supersaturation field offer different conditions for the growth of droplets which evolve in time due to turbulent transport and mixing. Based on this, we propose a Lagrangian stochastic model for condensation and evaporation of small droplets in turbulent flows. It consists of a set of stochastic integro-differential equations for the joint evolution of the squared radius and the supersaturation along the droplet trajectories. The model has two parameters fixed by the total amount of water and the thermodynamic properties, as well as the Lagrangian integral time scale of the turbulent supersaturation. The model reproduces very well the droplet size distributions obtained from direct numerical simulations and their time evolution. A noticeable result is that, after a stage where the squared radius simply diffuses, the system converges exponentially fast to a statistical steady state independent of the initial conditions. The main mechanism involved in this convergence is a loss of memory induced by a significant number of droplets undergoing a complete evaporation before growing again. The statistical steady state is characterized by an exponential tail in the droplet mass distribution. These results reconcile those of earlier numerical studies, once these various regimes are considered.