Ask an older image model for a coffee shop sign that says OPEN and you get a beautiful shop with a sign that says OPNE, or 0PEM, or a row of shapes that only look like letters from across the street. The rest of the image can be flawless. The words are where it falls apart.
The usual explanation is that the model "cannot draw text". That is close, but it points at the wrong part of the system. The problem mostly starts before any drawing happens, in the part of the model that reads your prompt.
Why Do AI Image Models Get Text Wrong?
A text-to-image model does not read your prompt letter by letter. A text encoder turns it into tokens, and common words become a single token. The researchers behind Character-Aware Models Improve Visual Text Rendering (Liu et al., 2022) found that "87% of words in the English Top 1% bucket are single-token for T5", one of the most widely used encoders. To the model, OPEN is one symbol, not four letters.
That is why the paper calls the encoders in Stable Diffusion, DALL·E, Imagen and Parti "character-blind": they "lack character-level input features, making it much harder to predict a word's visual makeup as a series of glyphs." The image model has to learn spelling indirectly, from pictures of words it saw in training.
The paper measured how well encoders recover spelling. A large T5 managed about two thirds:
| Text encoder (frozen) | Spelling accuracy, most common 1% of words |
|---|---|
| T5-Base | 14% |
| T5-XXL | 66% (65% on the rarest half of words) |
| ByT5, which reads bytes instead of word tokens | 95% to 99% at every size |
The authors also report that robust spelling in character-blind models "is only achieved at scales over 100B parameters", far larger than the encoders image models ship with. Feeding characters in directly gave "30+ point accuracy gains over competitors on rare words."
The Encoder Matters More Than the Painter
Google's Imagen paper (Saharia et al., 2022) made the broader point first: "increasing the size of the language model in Imagen boosts both sample fidelity and image-text alignment much more than increasing the size of the image diffusion model." A better reader beats a bigger painter.
The older Stable Diffusion family relies on CLIP. The Stable Diffusion 1.5 model card uses "a fixed, pretrained text encoder (CLIP ViT-L/14)" and lists a limitation in five words: "The model cannot render legible text". CLIP also reads a short window: CLIP's text model has 77 positions, so anything past that in a long prompt is cut off.
SDXL (Podell et al., 2023) pairs two CLIP encoders, OpenCLIP ViT-bigG and CLIP ViT-L, and is still candid about the result: it "still encounters difficulties when rendering long, legible text", and generated text "may contain random characters or exhibit inconsistencies."
What Changed With SD 3.5 and FLUX.1
The newer models add a large T5 encoder alongside CLIP. The Stable Diffusion 3 paper (Esser et al., 2024) uses two CLIP models plus T5-XXL, and credits its architecture with "improving text comprehension, typography, and human preference ratings."
The same paper includes the most telling experiment on the subject. Remove T5 at inference and aesthetic quality does not change (a 50% win rate) and prompt adherence barely moves (46%), but written text drops to a 38% win rate. The authors conclude T5 is "important for complex prompts e.g. such involving a high degree of detail or longer spelled text". The encoder is the spelling engine.
FLUX.1 follows the same pattern. The Diffusers FLUX documentation lists a CLIP encoder and "T5, specifically the google/t5-v1_1-xxl variant", with a prompt length that "defaults to 512" tokens. The timestep-distilled FLUX.1 [schnell] is capped at 256.
Which Model to Pick When the Image Has to Say Something
| Model in Vision Studio | Text encoders | Expect for lettering |
|---|---|---|
| Stable Diffusion 1.5 | CLIP ViT-L/14 | No legible text; add it afterwards |
| SDXL | Two CLIP encoders | Short words sometimes; long text unreliable |
| SD 3.5 Large and Medium | Two CLIP encoders plus T5-XXL | Short headlines and signs, much more often right |
| FLUX.1 [dev] and [schnell] | CLIP plus T5-XXL | Short headlines and signs, much more often right |
The encoder rows come from the papers and docs above; the expectations are the practical reading of them, not a guarantee. Even T5-XXL spelled about a third of words wrong in the encoder test.
Prompting Habits That Help
- Put the exact words in quotation marks and keep them short. A two-word sign is far more likely to survive than a sentence.
- Say where the text goes, such as "a neon sign above the door that reads 'OPEN'", so the model has a surface to put it on.
- Avoid rare words and invented brand names when you can. Single-token common words are the easiest case.
- Fix a near miss with inpainting instead of re-rolling the whole image: mask the sign and regenerate only the lettering.
- Set body copy in a layout tool. Paragraphs, prices and legal text belong in real type, not in pixels a model guessed.
Using This in Vision Studio
All four families above are in the Vision Studio model rack: FLUX.1 [dev] and [schnell], SD 3.5 Large and Medium, SDXL and SD 1.5, each run through its Diffusers pipeline, and a model stays in memory for the session once loaded, so switching back to it does not reload it. For an image with a headline, start on SD 3.5 or FLUX.1, check the licence of the model you choose for commercial work, and keep the seed fixed while you adjust the wording.
The Short Version
A model spells about as well as its text encoder can see letters. CLIP-only models were never shown them clearly; T5-based models see a lot more. Pick the model for the words, keep the words short, and let real typography do the long copy.
Vision Studio is free, open-source (MIT), and runs locally on your GPU. The model rack, inpainting and sampling controls are documented with their source files at vision-studio-x.com/features. Download it at vision-studio-x.com/download.
-Rocky
#Typography #TextToImage #FLUX #StableDiffusion #LocalAI #OpenSource #VisionStudio