🤖 The Modern AI Developer · Don't Let AI Cause Trouble: Testing & Security

How to Keep API Keys Out of Prompts (and Prevent SSRF)

Credential / secret leaks and SSRF in plain words

一句话先懂 · TL;DR

Why secrets don't belong in prompts: how credentials leak through AI agents, safe ways to store keys, and what SSRF means when an agent can make requests.

A Secret Is Like Your House Key

A secret (API key, password, token) is your house key—whoever gets it can open the door, spend money, and pull data in your name. So the key should be locked in a safe, not scribbled on a sticky note and posted everywhere.

But many people take the easy route and write the secret straight into the prompt sent to the model, like "use this key=sk-xxxx to call the API." The problem: prompts get logged, may be sent to third-party model services, and may be printed verbatim in error messages. The key just fell from the "safe" onto the open street.

⚠️Whatever appears in a prompt, you can hardly guarantee it "only gets glanced at by the model and then vanishes." Writing a secret into the prompt means assuming by default that it'll be archived and spread.

The Right Way to Keep the Key

The safer approach: the secret never enters the prompt. Keep it in the program's environment variables or a dedicated secrets manager, and have your code pull it out only at the moment it actually calls the API—the whole process bypassing the model. The model only decides "whether to call this tool"; the actual key-and-door action is done by your code.

🔆It's like asking the butler to fetch a package: you don't hand over the house key—you open the door yourself, and they just carry the box. The key stays in your hands.
# 反例:密钥被写进了给模型的提示(会进日志、可能外泄)
prompt = f"请用 key={SECRET_KEY} 调用支付接口"  # 危险

# 正例:提示里不出现密钥;密钥由代码从环境变量取出,调用动作绕开模型
import os
api_key = os.environ["PAYMENT_API_KEY"]   # 钥匙留在代码侧
prompt = "请判断是否需要调用支付接口"  # 鉴权由代码侧完成,不写进 prompt
# 模型只输出“要调”,真正带着 api_key 发请求的是下面这行你的代码
# pay_client.charge(amount, api_key=api_key)

SSRF: Don't Let It Knock on the Internal Network's Door for a Bad Actor

Suppose your Agent has a tool that "visits this URL and brings back the content." A bad actor will hand it an internal address—say, an admin panel only reachable inside the company, or a cloud server's "metadata endpoint." The Agent runs on the company server, so it reaches the internal network freely, and thus fetches back internal data outsiders normally can't touch, on the attacker's behalf.

This attack of "inducing the server to access an address it shouldn't" is called SSRF (Server-Side Request Forgery). The danger: the request comes from your trusted server, and the internal firewall lets it through.

💡Defense: add an allowlist/blocklist to tools like "visit a URL"—only permit explicitly approved external domains, and forbid internal network ranges and metadata addresses. Giving the Agent an address shouldn't mean it can go visit it. (You also need to validate the final resolved IP and forbid following redirects, or the allowlist can be bypassed via DNS rebinding/redirects.)

自测 · 学完检查一下

想真正动手做题、记进度、攒连胜?到互动课里练。

Judge: putting the API key straight into the prompt sent to the model is convenient, and since the key vanishes once the task is done, it's safe.

答案:No

Prompts usually get logged, may be sent to third-party model services or printed in errors; the key doesn't "vanish when done" and is in fact prone to leaking.

Which is the safer way to keep the secrets an Agent needs?

答案:Keep the key in environment variables or a secrets manager, pulled out by code at call time, with the process bypassing the model

Keeping the key always on the code side and out of the prompt is the key to preventing credential leaks via the model path.

Judge: a bad actor induces an Agent running on the company server to access an admin panel only reachable internally and fetch its content—is this SSRF?

答案:Yes

SSRF is exactly inducing the server to request an address it shouldn't (internal panel, metadata endpoint), bypassing the firewall via a trusted server.

The English abbreviation for the attack "inducing the server to access an address it shouldn't (e.g. an internal panel)" is ____ (four letters).

答案:SSRF

SSRF = Server-Side Request Forgery.

To protect a tool that "fetches content based on a URL," which approach best counters SSRF?

答案:Only allow external domains on an allowlist; forbid internal network ranges and cloud metadata addresses

Using an allowlist to limit reachable targets and blocking internal and metadata addresses cuts off, at the source, the Agent being induced to access internal resources.

Judge: since secrets shouldn't enter the prompt, having your code pull the key from environment variables only at the moment it actually calls the API—with the whole auth process bypassing the model—is a reasonable practice.

答案:Yes

Keeping the key on the code side and having auth bypass the model is exactly the recommended way to avoid leaking the key via the prompt and model path.

想边练边学,而不只是读?

到互动课里答题、记进度、攒连胜——游客即可试学,无需注册。

进入互动课程 →

Learn something new — don't miss updates

New courses, features and learning tips. Occasional emails, unsubscribe anytime.