Although 3D human pose estimation has gained impres-sive development in recent years, only a few works focus on infants, that have different bone lengths and also have limited data. Directly applying adult pose estimation mod-els typically achieves low performance in the infant domain and suffers from out-of-distribution issues. Moreover, the limitation of infant pose data collection also heavily con-strains the efficiency of learning-based models to lift 2D poses to 3D. To deal with the issues of small datasets, do-main adaptation and data augmentation are commonly used techniques. Following this paradigm, we take advantage of an optimization-based method that utilizes generative pri-ors to predict 3D infant keypoints from 2D keypoints with-out the need of large training data. We further apply a guided diffusion model to domain adapt 3D adult pose to infant pose to supplement small datasets. Besides, we also prove that our method, ZeDO-i, could attain efficient do-main adaptation, even if only a small number of data is given. Quantitatively, we claim that our model attains state-of-the-art MPJPE performance of 43.6 mm on the SyRIP dataset and 21.2 mm on the MINI-RGBD dataset.