AI coding tools are good at producing a plausible package name. Plausible is not the same as real, official, or safe.
That matters because adding a dependency is not just adding a line to package.json. You are choosing code that may run during installation, build steps, server startup, or a request handled by your app. When the suggestion came from an AI assistant, the package name itself needs to be verified before the rest of the review begins.
This is a narrower problem than a general pre-launch checklist. The question here is: should this AI-suggested dependency be admitted into the codebase at all?
Why an AI-suggested package deserves a hard stop
OWASP's guidance on LLM misinformation gives a concrete example: an LLM may suggest a non-existent library or package, and a developer who trusts the suggestion may integrate a malicious package instead. OWASP recommends verifying AI output, especially when it involves third-party code or libraries.
A 2025 USENIX Security study tested 576,000 code samples from 16 code-generating models. In that study's specific prompts and models, the researchers reported average hallucinated-package rates of at least 5.2% for commercial models and 21.7% for open-source models, with 205,474 unique hallucinated package names. Those figures describe that study; they are not a universal rate for every model or application.
The practical failure path is simple:
- An assistant invents or confuses a package name.
- A developer installs the first plausible match.
- The package's install or build scripts execute with developer or CI permissions.
- The app ships code nobody independently verified.
Gate 1: verify the package's identity
Start with the exact name, including its scope. Search the official registry and the project's own documentation. Compare the registry page with the official repository, organization, documentation site, and examples. A similarly named unscoped package is not an acceptable substitute for an official scoped package.
- Does the exact package exist at the expected registry URL?
- Does the official project documentation point to that exact name?
- Does the repository link on the registry lead to the expected organization?
- Is the version you are about to install actually published?
- Did the assistant quietly substitute a lookalike name or an abandoned fork?
“Slopsquatting” is a useful shorthand for attackers registering names that AI assistants may hallucinate. It is not an OWASP category, so treat it as a description of the attack pattern, not as proof that any particular package is malicious.
Gate 2: make the dependency tree reproducible
Once the identity passes, make the version decision explicit. npm's package-lock documentation describes the lockfile as the exact dependency tree generated for a project and recommends committing it so teammates, CI, and deployments can reproduce that tree.
Use a clean-install workflow such as npm ci in automation. npm documents that it uses the lockfile, removes the existing node_modules directory, and does not rewrite the package manifest or lockfile.
Reproducibility is not trust. A lockfile can faithfully reproduce a bad or unnecessary package. It prevents surprise resolution; it does not prove that the dependency should exist.
Gate 3: treat install scripts as executable code
npm's lifecycle documentation lists install-related hooks such as preinstall, install, postinstall, and prepare. These are code paths, not harmless metadata. Review them before allowing an unfamiliar dependency into a development machine or CI runner.
For a controlled first install, npm documents the ignore-scripts configuration, which prevents scripts specified in package files from running. That is a review aid, not a universal production setting: some legitimate packages require build steps, and explicitly invoked commands still run their intended command.
- Read the package's scripts and identify network access, file writes, downloads, or shell commands.
- Install unfamiliar packages in an isolated environment first.
- Decide whether every required lifecycle script is justified.
- Keep CI permissions narrow so an install hook cannot access unrelated secrets.
Gate 4: collect several trust signals
No single green check proves that a package is safe or appropriate. Use several signals, and write down what each one can actually establish.
- Known vulnerabilities:
npm auditreports known advisories in the dependency tree. A clean result does not prove that an AI-suggested package is legitimate. - Provenance: npm's provenance documentation explains how attestations can help verify where and how a package was published. Origin evidence is not a guarantee that the code is benign.
- Dependency diff: GitHub's dependency review shows added, removed, or updated dependencies and can surface vulnerability, license, age, and usage information.
- Project history: inspect the repository, release cadence, maintainers, issue activity, and build workflow. These are context, not a verdict.
- License and scope: confirm the package is allowed for the project and whether it is a direct runtime dependency, a development tool, or an unnecessary transitive addition.
Give your coding assistant a better contract
Do not ask an assistant only for “the package to use.” Ask it to return an evidence packet that a reviewer can challenge:
Keep a lightweight approval record with the package name, version, registry URL, lockfile entry, direct or transitive status, lifecycle scripts, provenance result, audit result, and reviewer decision. This turns a vague AI suggestion into a decision another person can reproduce.
Where VibeScan fits
After the dependency gate, run a first-pass check on the application and its deployed surface. VibeScan can be part of that repeatable review, but it should not be used as evidence that an individual package is trustworthy. The package decision belongs at the dependency boundary, where you can inspect identity, provenance, scripts, and diffs.
VibeScan is a first-pass security check—not a guarantee, a full software-supply-chain audit, or a replacement for a professional security assessment. Use it alongside code review, dependency controls, least-privilege CI, and a deeper assessment when the app or data warrants one.
A five-minute dependency admission sequence
- Pause when an assistant introduces a new package.
- Verify the exact registry identity and official project links.
- Review the manifest and lockfile diff before installation.
- Inspect lifecycle scripts and use an isolated, controlled install.
- Run advisory and provenance checks, record the evidence, then repeat the application check after the change.
The goal is not to reject every dependency. It is to stop “the AI suggested it” from becoming the only approval signal between a hallucinated package name and production code.
Shipping an AI-built app and want a first-pass security check?
Review the live app with VibeScan →Further reading
- OWASP Top 10 for LLM Applications — LLM09: Overreliance
- OWASP LLM03:2025 Supply Chain
- USENIX Security 2025: Package Hallucinations by Code-Generating LLMs
- npm package-lock.json documentation
- GitHub dependency review
For the broader launch sequence, read How to Secure an AI-Built Web App Before Launch. For a narrower runtime boundary, see AI-Built Webhooks: 7 Security Tests Before You Trust an Incoming Event.