Managing API Keys and Secrets in Cloud Projects

A Secret Is a Lifecycle, Not a String

Most teams think about secrets exactly twice: when a service issues one, and when something breaks because it changed. Everything in between is improvisation, which is how credentials end up in a chat thread, a screenshot and three different environment files.

It helps to treat every credential as having six stages. It is issued, stored, delivered to a process, used, rotated, and eventually revoked. A gap at any stage is where the incident comes from, and most organisations have the same two gaps: no rotation and no revocation.

The goal is not perfection. It is making the common path safe enough that nobody needs a shortcut, because shortcuts are what produce a database password in a public repository.

Where a Credential Should Actually Sit

The last row is the one to aim for wherever the provider allows it. Every managed platform has a secret store: AWS offers Secrets Manager alongside encrypted Parameter Store entries, Azure has Key Vault, Google Cloud has Secret Manager, and Oracle Cloud provides Vault with its own master keys. HashiCorp Vault is the common self hosted equivalent.

What you gain over a file is version history, per identity access control, an access log and a rotation hook. The first three matter immediately; the fourth is what makes rotation something you can actually do.

Getting the Value Into a Running Process

Storage solves half the problem. The value still has to reach the application, and the delivery method has security properties of its own.

Environment variables are the default and are acceptable with caveats. They are inherited by child processes, they appear in the process environment readable through /proc by sufficiently privileged users, and they frequently end up in crash reports and debug output that dumps the environment.

A file with restrictive permissions, ideally on a memory backed filesystem, is tighter. Kubernetes mounts secrets this way by default, and the same applies for a systemd service reading a credential file owned by its own user.

Fetching at start up from the secret store is tighter still, because the value exists only in the process that needs it and rotation takes effect on the next restart without touching any configuration. The cost is a dependency during boot, so cache the value in memory and handle the store being briefly unreachable.

The Best Credential Is One That Expires

A long lived key is a permanent liability. It works from anywhere, forever, for anyone holding it, and nothing about its use looks unusual. Short lived credentials change that calculation completely, because a stolen token is worthless within the hour.

Three mechanisms cover most cases. Instance and workload identity lets code running on a cloud resource obtain temporary credentials automatically, with no stored key at all. Federation lets an external system such as a CI provider present a signed assertion and receive temporary credentials scoped to one repository. Dynamic secrets, the pattern HashiCorp Vault popularised, generate a database user on demand with a lease that expires.

The practical migration path is to inventory every long lived key, identify which of them exist only because workload identity was not configured, and remove those first. That list is usually longer than expected and shrinking it removes real risk quickly.

Where keys must remain, scope them narrowly. One key per service, per environment, with only the permissions that service uses, following the reasoning in cloud IAM fundamentals. A narrowly scoped key that leaks is an incident; a broadly scoped one is an emergency.

Catching Them Before the Commit

Source repositories are the single largest source of exposed credentials, and the reason is structural rather than careless: once a secret enters history, deleting the file changes nothing. The value stays in every clone and every fork.

Three layers help, and they stack. A pre commit hook running a scanner such as gitleaks or trufflehog blocks the obvious patterns before they reach a commit. Server side push protection on the hosting platform catches what the hook missed, including commits from machines without the hook installed. A scheduled scan of existing history finds what predates both.

Make the safe path easy at the same time. A committed .env.example listing key names with empty values, plus .env in the ignore file, means a developer never has to invent their own arrangement.

Look beyond code too. Container images carry build arguments and copied files, log output captures request headers including bearer tokens, and infrastructure state files contain generated passwords. OWASP collects the broader set of controls in its cheat sheet series, and the pipeline specific handling is discussed in CI/CD pipelines that deploy to cloud servers.

Rotation That Does Not Cause an Outage

Rotation fails when it is a single swap, because there is a moment where some consumers hold the old value and some hold the new one. Overlap removes that moment.

Issue the new credential while the old one still works. Roll consumers over to it. Confirm the old one has stopped being used, which the store's access log or the provider's last used timestamp will tell you. Only then revoke it.

Managed stores automate this. AWS Secrets Manager runs a rotation function on a schedule and can maintain two active versions during the changeover. Applications that fetch on start up pick up the new value with an ordinary restart, which is another argument for that delivery pattern.

Set a rotation interval per credential class rather than one global rule. Anything reaching production data deserves a shorter cycle than a read only token for a public API, and anything an ex employee might have seen deserves rotating the day they leave.

What to Do in the First Ten Minutes

Assume the credential is compromised the moment it is exposed, regardless of how briefly or how few people could see it. Automated scanners find public repository credentials quickly.

• Revoke or rotate first. Cleaning up history before revoking protects nothing.
• Read the audit trail for uses of that credential from unfamiliar addresses or at unusual times.
• Check for changes made with it: new identities, new keys, altered permissions, unexpected resources.
• Remove the value from history afterwards, accepting that rotation, not rewriting, was the real fix.
• Record how it escaped, and close that path so the next one does not follow it.

The second and third steps depend entirely on having logging configured in advance, which is the case made in audit logging in the cloud. Without it, you can revoke the key but you cannot answer what was done with it.

Provider documentation covers the store specific mechanics, and the Secrets Manager rotation guide is a good example of how the overlap model works in practice. If you are setting this up from scratch, the Oracle Cloud account listings describe an account where Vault and its key management are available, and the AWS versus Oracle Cloud comparison looks at how the two organise identity and key services. How the underlying encryption works is covered in encryption at rest and in transit.

The Short Version

• Prefer workload identity and federation so there is no stored key to lose.
• Keep whatever remains in a managed store with access logging, not in files that get copied.
• Deliver values as mounted files or a start up fetch rather than command line arguments.
• Scope every key to one service, one environment and the permissions it actually uses.
• Run a scanner at commit time, at push time and across existing history.
• Rotate with an overlap window and revoke only once the old value is unused.
• Revoke before cleaning up, and read the audit trail before declaring it handled.