AI Terminology Glossary: 40+ Key Terms Explained

Introduction

Artificial intelligence comes with its own vocabulary. This guide covers the essential terms you need to know, organized by category. For detailed definitions of each term, visit our full AI Glossary.

Foundational Concepts

Artificial Intelligence (AI)

The broad field of creating machines capable of intelligent behavior. AI encompasses everything from simple rule-based systems to advanced deep learning.

Machine Learning (ML)

A subset of AI where systems learn from data rather than following explicit programming. ML algorithms find patterns in data and use those patterns to make predictions.

Deep Learning

A subset of machine learning using neural networks with many layers. Deep learning powers most modern AI breakthroughs in language, vision, and generation.

Neural Network

A computing system inspired by the brain, consisting of interconnected nodes (neurons) organized in layers.

Generative AI

A category of AI that creates new content — text, images, audio, video, or code — by learning patterns from existing data. This is the dominant AI paradigm powering tools like ChatGPT, Midjourney, and GitHub Copilot.

Reinforcement Learning (RL)

A learning paradigm where an agent learns by interacting with an environment and receiving rewards or penalties for its actions. RL is central to training game-playing AIs and is the foundation of RLHF.

Models and Architecture

Large Language Model (LLM)

A neural network trained on vast text data to understand and generate human language. Examples include GPT-4, Claude, and LLaMA.

Transformer

The architecture behind most modern LLMs. Transformers use self-attention to process all parts of input simultaneously, enabling much more efficient training than previous architectures.

GPT (Generative Pre-trained Transformer)

OpenAI’s family of LLMs, from GPT-1 (117M parameters) to GPT-4o (multimodal).

Mixture of Experts (MoE)

An architecture that uses multiple specialized sub-networks with a gating mechanism, activating only relevant experts per input. Used in GPT-4, Mixtral, and Gemini.

Foundation Model

A large AI model trained on broad, diverse data that can be adapted to many downstream tasks. Foundation models like GPT-4 and LLaMA are the backbone of modern generative AI.

Diffusion Models

The technology behind modern AI image generators. Diffusion models learn to create data by gradually removing noise, used by Midjourney, DALL-E 3, and Stable Diffusion.

Attention Mechanism

The core innovation behind transformers. Attention lets neural networks focus on the most relevant parts of input data, enabling models to capture long-range relationships in text.

Training Techniques

Fine-Tuning

Adapting a pre-trained model for a specific task by training it further on a smaller, specialized dataset.

RAG (Retrieval-Augmented Generation)

Combining LLMs with external knowledge retrieval to improve accuracy and provide current information.

RLHF (Reinforcement Learning from Human Feedback)

Training technique that uses human preferences to align model outputs with human values.

Prompt Engineering

The practice of designing effective inputs to get desired outputs from LLMs.

Few-Shot & Zero-Shot Learning

Performing tasks with few or no examples. Zero-shot relies purely on instructions; few-shot includes a handful of examples in the prompt to guide the model.

Chain-of-Thought (CoT)

A prompting technique where models break down complex reasoning into step-by-step chains, dramatically improving accuracy on multi-step problems.

LoRA (Low-Rank Adaptation)

A parameter-efficient fine-tuning technique that adds small, trainable matrices to existing weights, enabling fine-tuning of large models on a single GPU.

AI Safety

AI Safety

The field focused on ensuring AI systems are beneficial, aligned with human values, and do not cause unintended harm. Covers alignment, robustness, fairness, and interpretability.

AI Bias

Systematic errors or unfair outcomes in AI systems that can disadvantage certain groups. A critical concern for AI deployment in hiring, lending, criminal justice, and healthcare.

Applications & Infrastructure

AI Orchestration

The coordination of multiple AI tools, models, and agents into cohesive workflows. Frameworks like LangChain and CrewAI orchestrate complex, multi-step AI pipelines.

Key Concepts

Token

The basic unit of text processing in LLMs. One token is roughly 0.75 words in English.

Context Window

The maximum amount of text (in tokens) an LLM can process at once. Modern models range from 128K to 2M tokens.

Embeddings

Numerical vector representations that capture semantic meaning, enabling similarity search and RAG.

Hallucination

When an AI generates plausible-sounding but factually incorrect information.

Parameters

The learned weights in a neural network. More parameters generally mean greater capability, but efficiency matters too.

Explore Further

• Browse our complete AI Glossary for detailed definitions
• Read our guide on Understanding Large Language Models
• Compare ChatGPT vs Claude vs Gemini