Every AI project that calls an external model needs credentials, and those credentials need to be stored somewhere. Getting api key management right from the start prevents the most common class of AI security incidents.
What is API key management?
API key management is the practice of controlling how credentials are stored, accessed, rotated, and monitored throughout the life of an application. For AI projects, this typically means managing keys for LLM providers like Anthropic, OpenAI, or Google, along with any supporting services your agent or application calls. Good management means each key is in exactly one place, each application has only the access it needs, and you have a clear process for what to do when a key needs to change.
Why does API key management matter for AI projects?
AI projects have a broader credential surface than traditional web applications. A single agent might call a primary LLM provider, a vector database, an embedding service, and several tool APIs. Each represents a separate key that needs to be managed. GitGuardian data shows that 28% of credential exposure happens outside source code, including in build artifacts, log files, and cloud storage. In 2025, researchers found over 24,000 exposed secrets in MCP configuration files on public GitHub. Weak api key management at any one of these points can compromise the project.
How do I store API keys securely in AI projects?
Development: use environment variables loaded from a .env file. Keep a local .env file with your development credentials and load it at startup. List .env in .gitignore before creating the file. Create a .env.example that documents which variables are needed but uses placeholder values.
Staging and production: use a secrets manager. Dedicated secrets management services keep credentials out of your codebase and deployment artifacts entirely. Common options include:
- AWS Secrets Manager or AWS Parameter Store
- HashiCorp Vault (self-hosted or HCP)
- Google Secret Manager
- Azure Key Vault
- Doppler or 1Password Secrets Automation (developer-friendly options for smaller teams)
These services inject credentials at runtime, support rotation workflows, and provide access logs so you can see which process read which secret and when.
One key per scope. Create separate keys for each project and environment. Use provider-level project scopes where available. This limits the blast radius of any single leak and makes rotation easier.
Set spending caps. Most LLM providers let you configure monthly spending limits. Set one on every key. This is not a security control, but it limits financial exposure if a key is used without your knowledge.
Scan before you push. Add a secrets scanning step to your pre-commit hooks or CI pipeline. It will catch keys that end up in unexpected places before they reach a remote repository.
What are common mistakes to avoid?
- Using a single "master" key for all environments and projects
- Storing the
.envfile in cloud storage or sharing it over chat - Skipping spending limits because the project is small or early-stage
- Not having a rotation plan before you need it
- Treating secrets management as something to add later once the project is more mature
