Model Parameters
The numerical weights and biases that define a neural network's learned knowledge. Larger parameter counts generally indicate greater model capacity and capability.
Model parameters are the learned numerical values (weights and biases) that determine how a neural network processes input data. During training, these parameters are iteratively adjusted to minimize prediction error.
Parameter scale of notable models:
- GPT-1: 117M parameters
- GPT-3: 175B parameters
- LLaMA 3: 8B to 405B parameters
- GPT-4: Estimated 1.7T parameters (mixture of experts)
While more parameters generally mean more capability, recent research focuses on efficiency — achieving strong performance with fewer parameters through better architectures (Mixture of Experts), training techniques, and data quality. The ratio of parameters to training data is increasingly recognized as important as the raw parameter count.
Related Terms
Get smarter about AI
The sharpest AI news, curated daily. Delivered free to your inbox.