You would never download a random executable from a forum and double-click it. Plenty of people download a random checkpoint from a model hub and load it without a second thought. For a large share of model files, those are the same action.
A model file feels like data: a big blob of numbers. Whether it actually is data depends entirely on its format, and the most common legacy format is not data at all. It is a set of instructions that Python follows when it opens the file.
Why Pickle Is Code
PyTorch historically saved weights with Python's pickle module. Pickle does not just store values; it stores how to rebuild objects, and rebuilding can mean calling arbitrary functions. PyTorch's own developers put it plainly when they changed the default: the old behaviour "is inherently unsafe as the pickle module can import and execute arbitrary functions" (PyTorch dev-discuss).
Hugging Face says the same thing in its safetensors audit announcement: "PyTorch uses pickle under the hood, which is inherently unsafe", and a malicious pickle can give an attacker control of your computer without you knowing.
This Is Not Hypothetical
In February 2024, JFrog's security research team reported around 100 models on Hugging Face with real, harmful payloads, with PyTorch models the most common. One, uploaded under the account baller423, used pickle's __reduce__ method to open a reverse shell to an external IP address the moment the file was loaded with torch.load(). On Windows it launched PowerShell. On other systems it spawned a shell.
A year later, ReversingLabs found two more malicious models on the Hub using a technique they named nullifAI. The files were packed with 7z instead of the default ZIP and contained deliberately broken pickle streams. The scanner reported an error at the break, but the dangerous code before it had already run.
The lesson from the second case is the uncomfortable one: scanning helps, but a scanner is a guess about a format that was never designed to be safe.
The Safe Formats and the Safe Defaults
safetensors is the fix at the format level. It stores tensors and nothing else, so loading it cannot execute code. It was audited by Trail of Bits in an engagement commissioned by Hugging Face, EleutherAI and Stability AI, which found no critical flaw enabling arbitrary code execution.
weights_only loading is the fix at the loader level. With torch.load(..., weights_only=True), PyTorch refuses to unpickle anything except tensor payloads. PyTorch made it the default in version 2.6. Before 2.6 the default was the unsafe one, so any tool on an older PyTorch has to ask for it explicitly.
Remote code is the third trap. Some repositories ask you to run their own Python with trust_remote_code=True. The Transformers documentation tells you to "take extra precaution when loading a custom model" and to pin a specific revision so the code cannot change under you.
| Format or flag | Can run code on load? | Safe handling |
|---|---|---|
| .safetensors | No | Prefer it whenever a model offers it |
| .ckpt, .pt, .bin (pickle) | Yes | Load with weights_only, or convert to safetensors first |
| trust_remote_code | Yes, by design | Refuse unless you have read the code at a pinned revision |
How Vision Studio Enforces It
Advice in a blog post protects nobody at 2 a.m. when a download finishes. So the app enforces the policy rather than suggesting it.
- Risky formats are denied by default. Pickle-format weights and models that ask for trust_remote_code are blocked until you consent for that specific model. Nothing downloads before that.
- Remote code is never loaded. Consenting to a model does not mean running its custom Python.
- Pickle is converted, not trusted. Pickle files are converted with PyTorch's weights-only loader before use. Our engine ships PyTorch 2.5.1, which predates the 2.6 default, so the converter passes
weights_only=Trueexplicitly and writes safetensors. - Downloads are checked. Direct links are accepted only over HTTPS from CivitAI and GitHub, and verified against a SHA-256 hash.
- The exceptions are disclosed. The curated first-run model set is pre-approved, and the setup screen says so before you install it.
Why Not Simply Block Pickle?
Blocking every pickle file would be the simplest policy, and it would also lock you out of a large share of the community's work. Years of fine-tunes, LoRAs and older checkpoints exist only as .ckpt or .pt files, published before safetensors became the norm. A tool that refuses all of them pushes people toward loading those files somewhere else, with no protection at all.
So the design is consent plus conversion. You decide, per model, that you want it. The app then loads it with the restricted weights-only loader, which accepts tensors and refuses arbitrary objects, and writes the result out as safetensors, so the risky format is opened under the safest loader available. If the file tries to rebuild anything outside that loader's small allowlist of tensors and plain containers, the load fails rather than executing it. That is the behaviour you want: a loud failure instead of a quiet compromise.
Rules Worth Keeping in Any Tool
- Prefer safetensors, always. If a model offers both formats, there is no reason to take the pickle.
- Know your PyTorch version. Below 2.6, an unqualified torch.load is the unsafe default.
- Treat trust_remote_code as installing software, because that is what it does.
- Do not rely on scanners alone. nullifAI was built to slip past one.
One more habit is worth borrowing from software teams: pin what you load. The Transformers documentation recommends loading custom code from a specific commit hash so the code cannot change after you reviewed it. The same instinct applies to weights. A model repository can be updated after you vetted it, and a pinned revision is the difference between "the file I checked" and "whatever is there today".
Data Should Be Data
The model ecosystem grew up fast, and it inherited a file format from a time when everyone loading a checkpoint had trained it themselves. That time is over. Hubs now host files from strangers, at a scale nobody can review by hand. Until a file is proven to be just tensors, treat it like what it might be: a program you did not write.
Vision Studio is free, open-source (MIT), and runs locally on your GPU. Its consent gates and download checks are documented with their source files at vision-studio-x.com/features. Download it at vision-studio-x.com/download.
-Rocky
#AISecurity #SupplyChainSecurity #Safetensors #HuggingFace #PyTorch #OpenSource #VisionStudio