OpenAI's GPT-4V Safety Report: Jailbreaks, DALL-E Confusion, and a Doctor in the Machine

OpenAI just dropped the system card for GPT-4 with vision (GPT-4V), and it’s a fascinating, sometimes unsettling read. This isn’t a launch announcement—the capability is already rolling out broadly. Instead, this document is the safety autopsy, detailing exactly how they tried to break the model with image inputs before letting it loose. The core tension is familiar: multimodality is widely seen as the next big thing for LLMs, but every new input channel is a fresh attack surface. The researchers built their red-teaming specifically on top of the work done for base GPT-4, diving deep into what happens when a user can upload a photo instead of just text.

The report reveals a mix of clever mitigations and stubborn, weird failure modes. One of the most striking admissions is that GPT-4V can be jailbroken by images that contain overlaid text instructions designed to override the system’s guardrails. In one example, a photo of a stop sign with a tiny, written command to ignore safety protocols successfully tricked the model. The team also had to deal with a bizarre medical issue: the model sometimes acted like an unlicensed doctor, generating differential diagnoses when shown medical images, a behavior they had to actively suppress. The “person of interest” risk was another priority—they worked to ensure you couldn’t just upload a picture of a stranger and get their identity or personal details scraped from the model’s training data.

Then there’s the DALL-E problem. The paper details extensive work to stop the vision model from confusing its own AI-generated images with reality. For instance, they didn’t want a user uploading a photorealistic DALL-E 3 image of a celebrity and asking “Who is this?” only to have the model confidently misidentify a synthetic person as a real one. The mitigations here involved fine-tuning the model to recognize its own synthetic fingerprints, a meta-awareness that feels like teaching the AI to spot deepfakes from inside the machine. They also tackled ungrounded inference, the model’s habit of making assumptions about a person’s character or socioeconomic status based purely on visual cues in a photo, like the decor of a room.

What’s clear from the system card is that safety for multimodal models is less about solving a single problem and more about a game of whack-a-mole across dozens of edge cases. The team didn’t achieve a zero-risk state; they prioritized blocking the most harmful vectors while acknowledging that some risks, like adversarial image attacks, remain an ongoing cat-and-mouse game. The document serves as a reminder that making AI see isn’t just about adding capabilities—it’s about inheriting all the bias-laden, easily-spoofed, and privacy-invading baggage that comes with vision itself.