How to Store API Keys Securely in AI Projects

Every AI project that calls an external model needs credentials, and those credentials need to be stored somewhere. Getting api key management right from the start prevents the most common class of AI security incidents.

What is API key management?

API key management is the practice of controlling how credentials are stored, accessed, rotated, and monitored throughout the life of an application. For AI projects, this typically means managing keys for LLM providers like Anthropic, OpenAI, or Google, along with any supporting services your agent or application calls.

Good management means each key lives in exactly one place, each application has only the access it needs, and you have a clear process for what to do when a key needs to change.

Why does it matter for AI projects?

AI projects have a broader credential surface than traditional web applications. A single agent might call a primary LLM provider, a vector database, an embedding service, and several tool APIs — each a separate key to manage. GitGuardian data shows that 28% of credential exposure happens outside source code, including in build artifacts, log files, and cloud storage. In 2025, researchers found over 24,000 exposed secrets in MCP configuration files on public GitHub. Weak api key management at any one of these points can compromise the project.

What is a secrets manager?

A secrets manager is a service that stores credentials outside your codebase and delivers them to your application at runtime. Instead of writing an API key into a config file or setting it as an environment variable by hand on a server, your code asks the secrets manager for the value when it starts up. The key never sits in a file on disk.

Three things this gives you that environment variables alone do not:

No credentials in your repository or deployment artifacts. Your code references a secret name, not its value. A developer can clone the repo and read every line without seeing a single key.

Access logging. Every read is recorded. If a key is used at an unexpected time or by an unexpected process, you can see it.

Rotation without redeployment. Update the value in the secrets manager and every service that pulls it gets the new key on next startup — no code changes, no redeploy.

For local development, a .env file is a practical substitute. For anything running in a shared or production environment, a secrets manager is the correct tool.

Secrets manager options compared

The right choice depends on where your application runs, your team size, and how much operational overhead you are willing to take on.

AWS Secrets Manager

The standard choice for applications running on AWS. It integrates with IAM so EC2 instances, Lambda functions, and ECS tasks can fetch secrets without credentials of their own. The AWS identity is the authentication. Built-in rotation works natively with RDS, Redshift, and DocumentDB. Secrets can be replicated across regions for disaster recovery.

Cost: $0.40 per secret per month plus $0.05 per 10,000 API calls. For most AI projects with a handful of keys, that is under $5/month. It adds up on large deployments with many secrets.

AWS Parameter Store

The free alternative within AWS. Standard parameters (up to 10,000) cost nothing. The main trade-off: no built-in rotation. If you need to rotate a key, you build the automation yourself. Parameter Store also caps values at 8 KB versus Secrets Manager's 64 KB, and does not support cross-region replication natively.

A common pattern: use Parameter Store for configuration values and non-rotating API keys, Secrets Manager for database credentials that need scheduled rotation.

Google Secret Manager

Google Cloud Secret Manager is the GCP equivalent. Billing is per active secret version ($0.06/version/month) and per access operation ($0.03 per 10,000 calls). The free tier covers six active versions and 10,000 operations, enough for most small projects at no cost. Access control ties into Google Cloud IAM. Rotation is not automatic out of the box; it sends a Pub/Sub notification when a secret is due for rotation and your application or a Cloud Function does the actual update.

Azure Key Vault

Azure Key Vault goes beyond secrets to handle encryption keys and TLS certificates as well. The pricing model charges per operation ($0.03 per 10,000 transactions for standard secrets) with no per-secret storage fee, making it notably cheaper than AWS Secrets Manager for high-secret-count deployments. For Azure-native applications, it integrates with Managed Identity so services authenticate without separate credentials.

HashiCorp Vault

HashiCorp Vault is the most capable option and the most work to operate. Self-hosted Vault is open source and free, but you are responsible for infrastructure, high availability (minimum three nodes), upgrades, and incident response. On AWS, that infrastructure alone runs $200–$400/month before engineering time.

The feature that sets Vault apart is dynamic secrets: instead of storing a long-lived API key, Vault generates a time-limited credential on demand and revokes it automatically after use. This matters in regulated environments or complex multi-cloud setups. HCP Vault is the managed version; the self-managed operational burden disappears, but costs scale quickly (per-client fees of ~$73/month on the Essentials tier).

Worth the complexity for multi-cloud environments or teams with strict access policy requirements. Overkill for most AI application teams.

Doppler

Doppler is the easiest option to get started with. It is a SaaS platform that organizes secrets by project and environment, syncs them to any cloud provider or CI/CD tool, and provides a CLI that replaces .env files entirely. Free for up to 3 users with unlimited secrets. Team plans start at $21/user/month.

The trade-off is that it is SaaS-only (no self-hosted option) and does not offer dynamic secrets or the deep cryptographic features of Vault. For teams not yet committed to a single cloud provider, it removes friction without locking you in.

How to store API keys at each stage

Local development. Use a .env file loaded at startup via python-dotenv or equivalent. Add .env to .gitignore before creating the file, not after. Keep a .env.example in version control documenting which variables are required, with placeholder values only.

CI pipelines. Most CI platforms (GitHub Actions, GitLab CI, CircleCI) have a built-in secrets store. Set credentials there and reference them as environment variables in your pipeline config. Do not pass them as arguments or hardcode them inline.

Staging and production. Use one of the secrets managers above. Your application fetches values at startup. Credentials are not in your deployment artifacts, container images, or infrastructure config.

One key per scope. Create separate credentials for each project and environment. Use provider-level project scopes where available. This limits the blast radius of any single leak and keeps rotation scoped to one key at a time.

Set spending caps. Most LLM providers let you configure monthly spending limits. Set one on every key. It does not prevent unauthorized use, but it caps the financial damage before you catch it.

Scan before you push. Add a secrets scanning step to your pre-commit hooks or CI pipeline to catch keys that end up in unexpected places before they reach a remote repository.

Common mistakes

• One "master" key used across all environments and projects
• Storing the .env file in cloud storage or sharing it over Slack
• Skipping secrets management until "later" — later rarely comes before the incident
• No rotation plan in place before a key needs to change
• Assuming a private repository is safe (private repos get breached, and former collaborators retain history)