This research project investigates the integration of diffusion models into natural language processing (NLP), building on their success in computer vision. We explore incorporating diffusion methods into existing auto-regressive models and compare text generation with Large Language Models (LLMs). Our findings show that diffusion models are not superior to Transformer-based models. We assess the proficiency of LLMs in generating survey articles for NLP, focusing on 99 topics. Automated benchmarks indicate that GPT-4 outperforms GPT-3.5, PaLM2, and LLaMa2 by 2% to 20%. While GPT-created surveys are more contemporary and accessible, GPT-4 occasionally misses details or includes factual errors. We also found systematic bias in GPT-based evaluations compared to human evaluations.

• Diffusion Models and Large Language Models for Text Generation
• Large Language Models for Text Generation