Filter by Category
Filter by Author
Proper management of secrets, such as passwords, API keys, and certificates, is crucial for system security. In distributed environments and cloud deployments, mishandling sensitive information can lead to security breaches, data leaks, and severe consequences.
Traditionally, secrets have been stored in configuration files or code repositories, but this practice is insecure and difficult to maintain, especially when managing multiple environments and teams. The need to centralize and protect this data has driven the development of specialized secret management tools.
In this context, tools like HashiCorp Vault provide a viable solution by centralizing the lifecycle management of secrets. This enables development and operations teams to focus on innovation without compromising security.
I will explore fundamental concepts of secret management with HashiCorp Vault, its architecture, and best practices for implementing a secure and scalable system. Additionally, it will cover strategies for accessing secrets (direct queries vs. caching mechanisms) and provide practical integration examples across different programming languages.
A secret refers to any sensitive data that needs to be protected from unauthorized access. This can include, but is not limited to:
The importance of a secret lies in its ability to allow or restrict access to critical resources. Due to their sensitive nature, managing them in a centralized and secure manner is essential, ensuring that only authorized people or applications can access them.
Vault (HashiCorp Vault) has positioned itself as a robust and flexible tool for centralized secret management. It allows secure storage and policy-based access control.
It is a comprehensive secret management tool designed to provide secure, centralized storage of sensitive information and detailed control over who can access it and how. Its main features are:
Vault presents itself as an identity-based encryption and secret management system. This means that robust authentication and authorization methods strictly control secret access, guaranteeing secure, auditable and restricted access.
An additional advantage of Vault is that it is free software (initially under a BSL license and currently under an MPL – Mozilla Public License 2.0 license). The source code can be consulted in the following GitHub repository: https://github.com/hashicorp/vault/.
HashiCorp Vault is a server that stores encrypted secrets. Secrets are organized into “backends,” which can be databases, file systems, or external services. Access to secrets is controlled by “policies” that define who can read, write, or modify each secret. Additionally, Vault keeps a detailed log of all activities, making auditing and compliance easier.
In Vault, the initialization process generates what are known as the “unseal keys“, which are essential to protect the “master key” that encrypts all stored secrets. These keys are generated using Shamir’s Secret Sharing algorithm and, by default, Vault splits the master key into 5 parts (or keys) with a predefined threshold (e.g. 3 out of 5). This means that at least 3 of those 5 keys are required to reconstruct the master key and, therefore, to “unseal” (unlock) Vault and allow its normal operation. Key aspects are:
Specifically, Vault is divided into several sections:
When obtaining secrets in an application, the steps that are followed are:
The easiest way to experiment with Vault is to deploy it with Docker, for this example we are using Ubuntu 22.04 with: Docker version 26.1.3, build 26.1.3-0ubuntu1~22.04.1
To start Vault we create a file named “docker-compose.yaml” that will contain:
version: '3'
services:
vault:
container_name: vault
image: vault:1.13.3
environment:
VAULT_DEV_ROOT_TOKEN_ID: root
VAULT_ADDR: http://localhost:8200
VAULT_API_ADDR: http://0.0.0.0:8200
VAULT_ADDRESS: http://0.0.0.0:8200
cap_add:
- IPC_LOCK
ports:
- "8200:8200"
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "exit | curl --fail http://localhost:8200" ]
interval: 20s
timeout: 30s
retries: 30
command: vault server -dev -dev-listen-address=0.0.0.0:8200
To start the container we will use:
# Start the container $ docker compose up vault # At the end you will see the following: ... vault | WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory vault | and starts unsealed with a single unseal key. The root token is already vault | authenticated to the CLI, so you can immediately begin using Vault. vault | vault | You may need to set the following environment variables: vault | vault | $ export VAULT_ADDR='http://0.0.0.0:8200' vault | vault | The unseal key and root token are displayed below in case you want to vault | seal/unseal the Vault or re-authenticate. vault | vault | Unseal Key: E878WqalQNuN3Kka34wU8QF1j1Vem1njzI+OZUGGDWc= vault | Root Token: root vault | vault | Development mode should NOT be used in production installations! vault |
The system will remain in “standby“, that is, it is booted. For correct operation we will make sure that the Root Token is “root“, this will be the master password (Master Key) of Vault and will give us access to all tokens.
From a development point of view or to test Vault is the best option. In a production deployment the server will have a more complex structure (5 unseal keys), but this will not affect the developer in any way, who will continue to operate in the same way (using a token provided by the Vault administrators that will give access exclusively to the secrets protected by that token).
To verify that everything is correct, you can visit the URL: http://127.0.0.1:8200/
which will show us the Login screen in the browser:
The access token is “root” to be able to log in.
Vault comes with plenty of features:
Token versioning
Several secret engines
Groups and Leases
Several authentication methods
Multi-Factor Authenticaion (MFA)
OpenID Connect Provider
It is possible also to define your own policies:
An example written in HCL would look like:
# This block defines permissions to access all secrets
# within the path "secret/data/myapp" and its subpaths
path "secret/data/myapp/*" {
capabilities = ["read", "list"]
}
# This block allows write operations on the exact path
# "secret/data/myapp". Creating, updating, deleting, and reading the
# secret are allowed
path "secret/data/myapp" {
capabilities = ["create", "update", "delete", "read"]
}
# This block allows the authenticated user to lookup the details
# of their own token
path "auth/token/lookup-self" {
capabilities = ["read"]
}
It also provides a shell ready to use in the admin panel:
Vault also provides a REST API that can be used from any programming language:
The primary operations included in the RESTful API are:
The API uses JSON for data exchange and requires that each request be authenticated (except for those intended to obtain the initial token).
The most common actions are:
A well-documented API facilitates integration with any language that can make HTTP requests.
#
# === Python ===
#
# You need to install hvac with: pip install hvac requests
# The example assumes that the secret is stored in the KV engine version 2 at
# the path secret/data/myapp (the KV engine already exists by default in our
# container)
import hvac
# Configure the URL to Vault and the token
VAULT_ADDR = "http://localhost:8200"
VAULT_TOKEN = "root"
# Initialize the Vault client
client = hvac.Client(url=VAULT_ADDR, token=VAULT_TOKEN)
# Verify Vault status
if client.is_authenticated():
print("[✔] Sucessfully connected to Vault!")
else:
print("[❌] Authentication error!")
exit(1)
# Path to store the secrets
secret_path = "secret/data/myapp"
print("[✔] Saving a secret...")
client.secrets.kv.v2.create_or_update_secret(
path="myapp",
secret={"username": "admin", "password": "supersecret"},
)
print("[✔] Read a secret...")
read_response = client.secrets.kv.v2.read_secret_version(path="myapp")
print(f"Stored secret: {read_response['data']['data']}")
print("[✔] List secrets...")
list_response = client.secrets.kv.v2.list_secrets(path="")
print(f"Secretos disponibles: {list_response['data']['keys']}")
# print("[✔] Remove a secret...")
# client.secrets.kv.v2.delete_metadata_and_all_versions(path="myapp")
# print("[✔] Done! Secret removed sucessfully.")
<?php
/*
=== PHP Example ===
This example uses PHP's cURL function to make a GET request
You should handle the response and check for possible connection errors
*/
$token = 'root';
$vaultUrl = 'http://localhost:8200/v1/secret/data/myapp';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $vaultUrl);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("X-Vault-Token: $token"));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
if(curl_errno($ch)){
echo 'Error:' . curl_error($ch);
}
curl_close($ch);
$data = json_decode($response, true);
echo "Secret Data:\n";
print_r($data['data']['data']);
/*
The output will look like:
Secret Data:
Array
(
[username] => admin
[password] => supersecret
)
*/
/*
=== C Example ===
This program requires the libcurl library to be installed
The WriteMemoryCallback callback is responsible for storing the HTTP response in memory
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <curl/curl.h>
// Structure to store the answer
struct MemoryStruct {
char *memory;
size_t size;
};
// Callback function to write the answer into memory
static size_t WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp) {
size_t realsize = size * nmemb;
struct MemoryStruct *mem = (struct MemoryStruct *)userp;
char *ptr = realloc(mem->memory, mem->size + realsize + 1);
if(ptr == NULL) {
printf("No hay suficiente memoria (realloc retornó NULL)\n");
return 0;
}
mem->memory = ptr;
memcpy(&(mem->memory[mem->size]), contents, realsize);
mem->size += realsize;
mem->memory[mem->size] = 0;
return realsize;
}
int main(void) {
CURL *curl_handle;
CURLcode res;
struct MemoryStruct chunk;
chunk.memory = malloc(1); /* Inicializa con 1 byte */
chunk.size = 0;
curl_global_init(CURL_GLOBAL_ALL);
curl_handle = curl_easy_init();
// Prepare the URL endpoint to Vault
curl_easy_setopt(curl_handle, CURLOPT_URL, "http://localhost:8200/v1/secret/data/myapp");
// Configure the header with the token
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "X-Vault-Token: root");
curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, headers);
// Configure callback to store the answer
curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);
// Send the request
res = curl_easy_perform(curl_handle);
if(res != CURLE_OK) {
fprintf(stderr, "curl_easy_perform() falló: %s\n", curl_easy_strerror(res));
} else {
printf("%lu bytes received\n", (unsigned long)chunk.size);
printf("Answer:\n%s\n", chunk.memory);
}
// Free resources
curl_slist_free_all(headers);
curl_easy_cleanup(curl_handle);
free(chunk.memory);
curl_global_cleanup();
return 0;
}
/*
We will use the next command to compile the code:
gcc vault.c -lcurl -o vault
The output of the program will be:
344 bytes received
Answer:
{"request_id":"455b9ca7-ee7b-f69c-e8d0-51d81e90f976","lease_id":"","renewable":false,"lease_duration":0,"data":{"data":{"password":"supersecret","username":"admin"},"metadata":{"created_time":"2025-02-07T06:47:11.21923619Z","custom_metadata":null,"deletion_time":"","destroyed":false,"version":1}},"wrap_info":null,"warnings":null,"auth":null}
*/
One of the key aspects when integrating Vault into an application is deciding how and when to access the secrets.
We must consider that the use of in-memory caching avoids making calls to Vault in each request. This can improve the performance of the application, but it also introduces security risks if the cache is not managed properly. However, this system is still much more agile and secure than files with secrets.
When programming, storing tokens securely is essential to prevent unauthorized access to Vault. The best practices depend on the environment and security policies, but here are some recommended approaches:
This is the recommended approach for local development:
# Store the token in an environment variable $ export VAULT_TOKEN="token"
# Access it in your application
import os
vault_token = os.getenv("VAULT_TOKEN")
Pros: easy to configure, avoids hardcoding secrets in source code.
Cons: can be exposed in process lists (ps aux), logs, or if the system is compromised.
Vault Agent can authenticate to Vault and automatically retrieve tokens, reducing the need to manually store them. The token is stored in a response-wrapped file that applications can read.
Example configuration:
auto_auth {
method "aws" {
mount_path = "auth/aws"
config = {
role = "my-role"
}
}
}
sink "file" {
config = {
path = "/var/run/secrets/vault-token"
}
}
# The application can read the token from the filesytem
with open("/var/run/secrets/vault-token", "r") as f:
vault_token = f.read().strip()
Pros: eliminates manual token management, integrates with cloud IAM roles.
Cons: requires Vault Agent setup and proper permissions.
Store the token in a Kubernetes Secret, mount it as a file in the container and read it at runtime. Example of a Kubernetes Secret:
apiVersion: v1 kind: Secret metadata: name: vault-token type: Opaque data: token: <base64-encoded-vault-token>
Pros: integrates well with Kubernetes security practices.
Cons: kubernetes Secrets need to be encrypted and properly managed.
If running on AWS, GCP, or Azure, avoid storing tokens altogether by using IAM authentication. Example with AWS:
vault write auth/aws/login role="my-role" pkcs7=$(curl -sL \ http://169.254.169.254/latest/meta-data/iam/security-credentials/)
Pros: no need to store static tokens, authentication is dynamically managed.
Cons: requires correct IAM policies and role configurations.
If can not proceed with the previous options and don’t have any other operational choice, the applications can read from ~/.vault-token securely.
# Store the token in a restricted file echo "your-vault-token" > ~/.vault-token chmod 600 ~/.vault-token
Pros: useful for CLI tools.
Cons: less secure if file permissions are misconfigured.
🚫 Hardcoding tokens in source code
🚫 Committing tokens to version control (e.g., GitHub, GitLab)
🚫 Storing tokens in shared locations with weak access control
The integration of Hashicorp Vault into secret management provides a centralized, secure, and scalable solution, adapted to the needs of modern, distributed environments. The availability of a RESTful API facilitates communication from various languages, allowing Vault to be integrated into both legacy systems and new architectures.
The key is to find the balance between security and performance:
You can find more information at:
