Watch the video, transcript and sources
Model weights are learned numbers. A model package can also bring executable instructions. Loading one malicious Hugging Face package opened a remote shell, which is an ambitious interpretation of local AI. Here's how loading becomes execution, which secrets it can reach, and what a useful boundary looks like.
Opening the parcel
Hugging Face is a platform for sharing machine learning models, datasets and applications. A repository can contain weights, configuration and Python code. Pickle stores Python objects with instructions for rebuilding them. An unsafe loader can follow those instructions and call functions while unpacking.
Downloading the bytes puts the parcel on your disk. Deserializing opens it. Somewhere, a file format has been promoted to unpaid system administrator.
JFrog reported baller423/goober2 in February 2024. Its Pickle payload opened a reverse shell, connecting outward to offer remote command access. The repository was deleted. JFrog also found clues suggesting researcher involvement, so attribution is uncertain. Credential theft from actual victims was not established.
The key is inside your permissions
Code running as your user can read whatever that process can read. A mounted cloud key or an accessible token becomes a possible target. Sending it out also needs a working network route. Numerical weights alone have no magic credential powers. Your permissions supply the interesting part of the model card.
Some models ask you to enable trust_remote_code, which authorizes Python from the repository. Safetensors files can sit beside that code. Checking the weight format covers one entrance, while the repository's code is still waiting for you to hand it a visitor badge.
Checkmarx's Model Confusion research demonstrated a missing local path resolving to a public repository with a matching name. Combine that with remote code enabled, and the wrong package can execute. The unnamed company's sample was corrected within a day. A missing folder acquired a landlord.
The error arrives late
ReversingLabs reported two nullifAI samples in February 2025. Both contained reverse shells. Their 7z packaging blocked ordinary PyTorch loading, so this path required extraction and unsafe deserialization. Broken Pickle streams also exposed a historical scanner blind spot.
In the researchers' harmless test, instructions wrote a file before the loader reached the broken part and raised an error. Imagine opening a parcel, releasing the raccoon, then noticing the shipping label is invalid. The error message arrives with excellent grammar and terrible timing.
Hugging Face removed the reported samples within 24 hours of disclosure and changed the scanner. That specific finding prompted a fix. A scanner error still deserves investigation before anyone opens an unfamiliar parcel.
Put the boundary around the loader
Prefer Safetensors and current patched loaders. PyTorch narrowed its default loader starting at version 2.6. Review custom code and pin the reviewed revision. Run unfamiliar packages in isolation without personal keys, with restricted network access. A container full of mounted credentials is a gift basket with namespaces.
Loading failure tells you where execution stopped. Earlier instructions may already have run. Keep the secrets outside its reach before opening the parcel.
I'd keep open models and put the permission boundary around the loader. The free download should come with a model, and the guest list should remain my decision.
Verdict: NEEDS REVIEW — Put the boundary around the loader
Sources
https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/
https://www.reversinglabs.com/blog/rl-identifies-malware-ml-model-hosted-on-hugging-face
https://checkmarx.com/zero-post/hugs-from-strangers-ai-model-confusion-supply-chain-attack/
https://huggingface.co/docs/hub/en/security-pickle
https://huggingface.co/docs/safetensors/en/index
https://huggingface.co/docs/transformers/en/models
https://docs.pytorch.org/docs/2.14/notes/serialization.html
https://huggingface.co/blog/jfrog
And that's the diff for today. I'm Niko from Axrisi. Merge responsibly.
YouTube · Newsletter · thedailydiff.dev · forward this to the intern who deployed on Friday.