← Back to Concept Contrast

Concept Contrast Fable

Model poisoning and data poisoning: one poisons the factory, the other adulterates the raw materials

Conclusion first:At security review meetings, two terms are often listed side by side as if they describe the same kind of attack. Model poisoning is directly tampering with the weights of an already trained model or implanting a backdoor; data poisoning is mixing malicious samples into training data so that the model learns incorrect behavior during training. The attack entry points differ, the detection methods differ, and the protection priorities differ—one guards the integrity of the model repository, the other guards the purity of the training pipeline.

At security review meetings, two terms are often listed side by side as if they describe the same kind of attack. Model poisoning is directly tampering with the weights of an already trained model or implanting a backdoor; data poisoning is mixing malicious samples into training data so that the model learns incorrect behavior during training. The attack entry points differ, the detection methods differ, and the protection priorities differ—one guards the integrity of the model repository, the other guards the purity of the training pipeline.

Fable

Two Accidents at Two Soy Sauce Factories

Two soy sauce factories in the town had accidents one after another. The first one's problem was that someone broke into the finished goods warehouse at night and injected spoiled bacterial liquid into soy sauce bottles that had already been filled and sealed. The next day at shipping, everything looked normal on the outside, but consumers suffered after opening the bottles. The factory checked surveillance and discovered it was deliberate tampering; the problem was in the warehouse security process.

The second company's problem was harder to trace. They had a batch of soy sauce that developed a strange flavor during fermentation—not sour, not spoiled, but always tasting wrong. After months of investigation, they finally found that a raw material supplier had mixed a genetically modified soybean into the beans; it looked the same, but after fermentation the amino acid ratio was completely different. The problem occurred at the raw material intake stage, but the consequences showed up in the finished product.

The first factory's loss was one-time: seal that batch of problem products, strengthen warehouse security, compensate consumers. The second factory's loss was ongoing: because it did not know how many batches of problem soybeans had been mixed in, it could only clean the entire fermentation line, replace the cultures, and rebuild the supply chain review process. And because consumers had already eaten "subtly off-tasting" soy sauce for several months, the brand's reputation dropped sharply.

The town guild wrote the two incidents into notices. The first was addressed to the finished goods warehouse keeper: seal the door at night, seal bottle mouths, and verify signatures box by box before outbound shipment, to prevent anyone from touching bottles that are already filled. The second was addressed to procurement and brewers: check the source of incoming ingredients, sample-brew, and trace abnormal flavors back to the bean field, to prevent bad things from mixing into the brewing process.

One detail in the notice made many small factories remember the difference. If the finished goods warehouse has been tampered with, clues can usually be found from seals, door locks and surveillance; if the raw materials have been adulterated, the problem spreads across hundreds of fermentation tanks and must be traced slowly through long-term flavor profiles and supply chain ledgers. One is like someone prying open the warehouse door, the other is like someone swapping your seeds.

After seeing the notice, a small factory changed its own checklist. It had originally had only one line, "prevent poisoning," which was later split into two: one line checked the keys, seals, and night patrol records of the finished goods warehouse, and the other checked the source of soybeans, sample retention, and small-batch trial brewing before going into the vat. For the first time, the workers realized that even under the same name of poisoning, the door that truly needed guarding was not in the same place.

The soy sauce factory's lesson later spread through the town as a saying: someone pours bad water into the vat you have already brewed, and you can smell it the moment you open the lid; someone mixes another kind of bean into your soybean field, and you only find out when the sauce's aroma changes. You must guard against both, but one means watching the finished-goods warehouse, the other means watching the raw-material field. Writing the two things onto the same lock-the-door checklist will only miss the real entry point.

Model poisoning

Directly tampering with the weight parameters of an already trained model or implanting hidden backdoors; the attack target is the model file itself, and it often occurs during model storage, distribution, or deployment.

Data poisoning

Injecting malicious samples into training data during the model training phase, causing the model to learn wrong behavior or produce specific biases; the attack target is the raw data in the training pipeline.

Story Mapping

  • The finished goods warehouse was injected with spoilage bacteria: model poisoning, attacking an already trained model file
  • Raw soybeans mixed with modified varieties: data poisoning, attacking the source and purity of training data
  • The first loses once, the second persists for months: model poisoning impact is concentrated and isolable, while data poisoning has high traceability costs
  • The security questionnaire split from one line into two: procurement review must distinguish two attack surfaces

Put it into an AI security solution

When filling out customer security questionnaires, separate model poisoning and data poisoning into two independent control items. The response to model poisoning is model signature verification, repository access auditing, and distribution pipeline encryption. The response to data poisoning is training data provenance, pipeline isolation, and sample-level anomaly detection. The two sets of controls require different budgets, roles, and audit evidence; combining them makes it unclear to both sides what exactly is being defended against.

京ICP备2026014272号