This research project explores the significance of integrating diffusion models into natural language processing (NLP), building upon their success in computer vision. Specifically, the project addresses the open research question of incorporating diffusion methods into existing auto-aggressive models in NLP. The study also emphasizes the growing trend of large pretrained models and their computational challenges. The objective is to optimize the integration of diffusion models with pretrained models, enhancing inference speed and benefiting both academia and industry. The research focuses on text generation tasks, such as summarization and machine translation, and seeks to overcome the challenge of integrating diffusion methods into auto-aggressive sequence-to-sequence models in NLP. By leveraging pretrained models and optimizing their utilization through diffusion models, the project aims to improve efficiency without starting from scratch, addressing resource limitations, and advancing the field of NLP.