Image Upload Moderation Beyond Node.js: Classify NSFW and Violence with Multimodal Chat
The article, published on DEV Community, addresses image upload moderation beyond Node.js, focusing on classifying NSFW and violence using multimodal chat. The author notes there is no dedicated image moderation endpoint, so the practical design involves a policy prompt, a vision-capable chat model, schema validation, and a conservative fallback. They advise against shipping the model's prose directly into allow/block decisions, instead keeping the original decision for audits, translating it into a small internal status, and making the eval set the release gate. The model is described as one component of the policy system, not the policy system itself. Categories should come from the app's actual rules; for a general user-content product, the author starts with nudity, graphic violence, hate symbols, drugs, and minors-risk. They caution that these labels are not universal, noting that a medical forum and a marketplace need different thresholds, and a historical archive may legitimately show symbols that a profile-photo product should reject. The author's first notebook pass is deliberately boring: they assemble a small set of allowed, blocked, and ambiguous pictures, write expected category labels, and record the policy reason in the excerpt.
Developers can use multimodal chat with JSON schemas for explainable image moderation, but must treat the model as one component, not the policy itself.