Metal Mantra / AGENT INTELLIGENCEPublic evidence reportAUTOMATED SIGNAL. NOT CERTIFICATION.
Metal Mantra
Run free scan
Metal Mantra
Run free scan

Public agent signals. Clear evidence boundaries.

PUBLIC REPORT · ADDED BY METAL MANTRA / AGENT SIGNAL v0.2

agent-lightning.

Scanned repository says: Agent Lightning — agentic RL infrastructure

Metal Mantra scanned this public repository on its own initiative; the maintainer did not submit it. Maintainers: scan the repository yourself to take this report over, or contact us.

Inspect GitHub source

18.6k starsMITpushed 2d ago

0community interest

Votes show community interest, not safety. They never change this score.

AUTOMATED SIGNAL71/100BBBNot security certification
START HERE

Review findings need attention.

Remote model code trusted

Inspect the findings
WHAT THIS DOES NOT ESTABLISH

Runtime safety, real-world agent capability, fitness for your use case or independent certification.

Scan scope59 / 59Complete eligible-file selection · not the entire repository
Recorded findings10Heuristic matches requiring human review
SnapshotOct 12, 2026UTC · engine 0.4.0
DNS at scan timeUnverifiedDomain control is not security certification
01 / KNOW THE REPOSITORY

What the source
actually tells us.

Structured, cited information from the scanned archive. Repository-authored claims are labeled and never treated as verified capabilities.

INSPECTED SOURCE59 / 59Complete eligible selection
READMEReadRepository-authored information
MANIFESTS2LICENSE, pyproject.toml

Who it is for

  • Stated purpose · manifest declares · unreviewed

    Agent Lightning — agentic RL infrastructure

    pyproject.toml:4

What it can do

  • Stated capability · repository says · unreviewed

    🪶 ~3,500 lines of code: We treat simplicity as the first principle.

    README.md:22
  • Stated capability · repository says · unreviewed

    🧩 Train with real agent harnesses: Agents interact with the model through the Agent Lightning v1.0 proxy with ZERO changes, while keeping tools, context, control flow, and environments in the loop.

    README.md:23
  • Stated capability · repository says · unreviewed

    ☸️ Native Kubernetes support: Run agents directly as Kubernetes Jobs without relying on external sandbox services.

    README.md:24

What adoption takes

  • Stated deployment · repository says · unreviewed

    The following is an example installation on a CUDA 13.0 machine:

    README.md:29
  • Stated prerequisite · manifest declares · unreviewed

    Manifest declares Python >=3.12 as a requirement.

    pyproject.toml:21
  • Declared dependency · manifest declares · unreviewed

    Declares a dependency on fastapi.

    pyproject.toml:23
  • Declared dependency · manifest declares · unreviewed

    Declares a dependency on uvicorn.

    pyproject.toml:24
  • Declared dependency · manifest declares · unreviewed

    Declares a dependency on pydantic.

    pyproject.toml:25
  • Declared dependency · manifest declares · unreviewed

    Declares a dependency on httpx.

    pyproject.toml:26
  • Declared dependency · manifest declares · unreviewed

    Declares a dependency on httpx-retries.

    pyproject.toml:27
  • Declared dependency · manifest declares · unreviewed

    Declares a dependency on hydra-core.

    pyproject.toml:28

Limits and terms

  • Declared license · manifest declares · unreviewed

    A license or copying file is present; its terms have not been reviewed.

    LICENSE:1

Observed technical context

JavaScript · 3 selected filesPython · 56 selected filesAgent orchestration reference · static onlyModel API reference · static onlyTool-calling reference · static only

These are references in selected source files, not proof that a feature works at runtime.

Read from commit 69f5a40 at Oct 12, 2026, 5:33 AM UTC. Source links open that exact commit. Repository statements are unreviewed and do not affect the score.

02 / FINDINGS FIRST

What needs
a closer look.

Critical details are withheld publicly for the first 30 days after a scan. A matched pattern is a lead for investigation, not a confirmed vulnerability.

Dependencies: 35 exact runtime versions from uv.lock checked against OSV.dev on Oct 12, 2026; 4 with known advisories. Reachability is not checked.

10 findings
MAJOR

Remote model code trusted

SEC007 · Security & privacy

The code loads a model with trust_remote_code enabled, which downloads and runs Python from the model's repository. Whoever controls that repository controls code running inside the agent.

SUGGESTED ACTION

Leave trust_remote_code off, or pin the model to a reviewed revision hash.

examples/search_r1/agents/search_r1_agent.py:162
MAJOR

LLM output without runtime schema

SCH001 · Schema & tooling

Model output is used without being checked against a schema. Unexpected or malformed output can break later steps or be trusted when it should not be.

SUGGESTED ACTION

Validate model output with Zod, JSON Schema, Pydantic or provider structured output.

examples/gsm8k/gsm8k_agent.py:93
MAJOR

LLM output without runtime schema

SCH001 · Schema & tooling

Model output is used without being checked against a schema. Unexpected or malformed output can break later steps or be trusted when it should not be.

SUGGESTED ACTION

Validate model output with Zod, JSON Schema, Pydantic or provider structured output.

examples/multimodal_qa/multimodal_qa_agent.py:54
MAJOR

Model call without fallback

EFF001 · Token & cost efficiency

A model call has no visible fallback. If the provider fails or is rate limited, the whole agent fails with it.

SUGGESTED ACTION

Configure a secondary model or explicit fallback path for provider failures.

examples/gsm8k/gsm8k_agent.py:93
MAJOR

Domain ownership not verified

TRUST001 · Entity trust

Metal Mantra could not confirm that the owner controls the stated domain. The domain may belong to someone else.

SUGGESTED ACTION

Verify the domain using the Metal Mantra DNS TXT challenge.

microsoft.github.io · location unavailable
MAJOR

Dependency with a high-severity advisory

DEP001 · Security & privacy

anyio 4.13.0 (PyPI) has known advisories: GHSA-5p39-cfhj-2xmp, GHSA-82r6-8w77-94w6, PYSEC-2026-4024, PYSEC-2026-4025 (highest severity critical). AnyIO: TLSStream IDNA 2003 host name encoding enables potential TLS certificate spoofing

SUGGESTED ACTION

Upgrade to a fixed version listed in the advisory and regenerate the lockfile.

uv.lock:305
MAJOR

Dependency with a high-severity advisory

DEP001 · Security & privacy

cryptography 48.0.0 (PyPI) has known advisories: GHSA-537c-gmf6-5ccf, GHSA-g6cj-pr64-35w5, GHSA-jwv3-5hgf-82ww, GHSA-m2h6-j472-rp4c, PYSEC-2026-3552 (highest severity high). Vulnerable OpenSSL included in cryptography wheels

SUGGESTED ACTION

Upgrade to a fixed version listed in the advisory and regenerate the lockfile.

uv.lock:699
MAJOR

Dependency with a high-severity advisory

DEP001 · Security & privacy

hydra-core 1.3.2 (PyPI) has known advisories: GHSA-2cp2-2r3c-7p7r, GHSA-c3wx-c55w-pxjq, PYSEC-2026-3850 (highest severity high). Hydra: hydra.utils.instantiate with untrusted config can lead to code execution

SUGGESTED ACTION

Upgrade to a fixed version listed in the advisory and regenerate the lockfile.

uv.lock:51
MINOR

Privacy Policy link not confirmed

TRUST003 · Entity trust

No privacy policy link was found on the stated homepage. Buyers cannot see how the vendor handles their data.

SUGGESTED ACTION

Expose a clear Privacy Policy link on the official homepage.

microsoft.github.io · location unavailable
MINOR

Terms link not confirmed

TRUST004 · Entity trust

No terms link was found on the stated homepage. Buyers cannot see the conditions that apply to using the service.

SUGGESTED ACTION

Expose a clear Terms link on the official homepage.

microsoft.github.io · location unavailable
03 / EVIDENCE & LIMITS

The boundary
is part of the result.

What was scanned
59 of 59 eligible files · Complete selection.
Supported static source patterns and domain checks; no runtime execution.
What remains unknown
Unscanned files, external dependencies, production configuration, actual agent behavior and evidence beyond the configured checks.
Source location
Links open the current repository HEAD. Source may have changed since this snapshot.
Something looks wrong?
Tell us if a finding, scope or claim on this report needs a correction. Report a problem with this report
Entity trust · 15% of v0.2
Includes DNS ownership, HTTPS and policy evidence. Verification is domain control, not a security certificate.
DNS: unverifiedHTTPS: availablePrivacy: not confirmedTerms: not confirmed
04 / THE METHOD BEHIND THE NUMBER

Open the
calculation.

Each pillar starts at 100. Rule deductions reduce its score; weights determine the overall contribution.

01

Security & privacy

4 findings · 25% weight
62/10015.5 pts
02

Guardrails & loop control

0 findings · 25% weight
100/10025.0 pts
03

Schema & tooling

2 findings · 20% weight
68/10013.6 pts
04

Token & cost efficiency

1 findings · 15% weight
86/10012.9 pts
05

Entity trust

3 findings · 15% weight
25/1003.8 pts
Inspect the current standard
USER REPORTS

Did agent-lightning work for people?

No user reports yet.

Each report is one signed-in GitHub account saying what happened. We do not verify that the account ran the agent, and reports never change the automated score or grade. For a security problem, use the disclosure page instead of a public note.

Sign in with GitHub to add a report

SHARE THE EVIDENCE

A signal.
Not a seal.

Share the full report so its scope and limits travel with the score.

Metal Mantra automated signal 71/100, not certified