Table of contents

Introduction

Secret handling is one of those tasks where you have to be very careful.

If you have been developing application there is a good chance that you came access the method of storing secrets in a .env file. Something like this

$ cat .env

USERNAME=XXXXX
PASSWORD=XXXXX
MONGO_URI=XXXX

These files are called a dotenv file and generally used while passing secrets to your application.

Why are people storing secrets in file ?

According to Twelve-Factor app which mentions that config should be should be seperated from the application. This is completely understandable. Its good when configuration is seperated from your application. This makes the application highly tunable and manageable. This was the story but later people started passing secrets along with the file and along with this “security by obsecurity” mentality took over.

Myth and reality around .env

There are several myths about .env some of which are .env is the secure way of passing secrets to your program as it is not accidentally commited to git. Since .env is not listed at part of ls command, it is secure as it is hidden.

But in reality .env is not different to any other files. Git treats all your files including .env same.

If you forget to put it as part of your .gitignore file then it will be commited and pushed to the remote repository just like any other files in your repository.

Another thing is, when values are loaded from .env into your running program which is a process, all the values are set as environment variables and can be accessed using command similar to os.Getenv("KEY") in Go or process.env.S3_BUCKET in node.

Passing secrets as environment variables has been there for a long time, but there will always be a risk of process leaking environment variables to its child process or via Remote Code Execution ( RCE ). As we all know that no application is fully secure and there is are chance someone quite competent might be able to run arbitary code via your application and pull in the environment variables.

Your application might be secure, but the library with which you spawn another subprocess or child process might be leaking the environment variables.

This sounds like a hoax but it has actually happened. Here are few reads

https://gitlab.com/gitlab-org/gitlab/-/issues/337601

There are various dangers of using these approach: https://www.trendmicro.com/en_us/research/22/h/analyzing-hidden-danger-of-environment-variables-for-keeping-secrets.html

https://towardsdatascience.com/leaking-secrets-in-web-applications-46357831b8ed

I could go on but you get the gist.

Configuration variables and secret variables are TWO DIFFERNT THINGS, Do not treat them as one

I keep telling this, configuration variable are not same as secret variables. Stop treating them as one. Due to this, people are mixing both into one either in .env file or in yaml file. This is where the confusion is. Both are different and should be managed differently. Let me tell you know alternatives way you could do it a bit better.

Alternatives

I understand, it’s easy to use .env and be done with this, but if we have any secrets in our application, then we are responsible to handle them correctly. Here are few alternative approach that could be used to best manage your secrets.

Using platform provided secret management feature

Different providers provide features for secret management i.e docker secrets from docker, k8s secrets from k8s etc. Instead of adding secrets in your repository you could create those secrets in the platform and pull in from there. At least this way, your secrets are not pushed into the repo. Security wise, this is no different than using .env. They are just base64 encoded.

Using external secret storage

The best approach is to use external secret storage or secret manager like Hhashicorp vault or AWS Secret manager. These services are designed to protect your secret. Benefit of using external secret storage it the ease of managing the secrets. You can easily rotate, update, delete secrets without breaking the application. External secrets also provide secrets encryption at rest, and in transit.

Using git committed yaml or json file to hold configuration variables and inject secrets variables to a file at runtime.

Of all the above mentioned, this is my favourite one. Why ? Let me explain

The above mentioned solution regarding external secret manager is good for corporate world, but in real life it’s hard to get hand on such services without spending some cash. Don’t remind me about the operational complexity, management, maintenance and also securing the secret manager itself.

Hence another approach to do this better ( imho ) would be to store the configuration variables in a yaml or json file with tight permission and load them into your application just like any other configuration file and injecting secret variables at runtime. This way configuration variables are committed which provides greater flexibility of manging change via git i.e version controlled and also the secrets are never stored in environment variables.

Now those variables will only be accessible only through some interface in your application and not by all the methods or functions. This minimizes the risk of your secrets being exposed via process leaking environment variables.

This way your configuration variables are

  1. Access managed

  2. Version controlled

  3. Safer updates via PRs

where as secret variables are

  1. Tightly controlled

  2. Managed

  3. Relatively secure

Demo

Here is an example how you can pass secrets as config file to your program without loading them as environment variables. I am using Go, but you can use any language

First of all define your yaml configurations

environment: "dev"

server:
  host: "localhost"
  port: 9090

So, let’s continue. Now load this file into the program natively using go or programming language of choice. Here i am usin go and viper to load the configuration variable. Here i am using golang tool called viper to do that for me.

type Configuration struct {
	Environment string

	Server struct {
		Host string
		Port int
	}

}

func LoadConfiguration(configPath, configName, ConfigType string) (*Configuration, error) {
	var config *Configuration

	viper.AddConfigPath(configPath)
	viper.SetConfigName(configName)
	viper.SetConfigType(ConfigType)

	viper.AutomaticEnv()
	viper.SetEnvKeyReplacer(strings.NewReplacer(`.`, `_`))

	err := viper.ReadInConfig()
	if err != nil {
		return nil, fmt.Errorf("could not read the config file: %v", err)
	}

	err = viper.Unmarshal(&config)
	if err != nil {
		return nil, fmt.Errorf("could not unmarshal: %v", err)
	}

	return config, nil
}

Once configuration are loaded into the program, you can use it into your code wherever as necesssary as shown beloe

	config, err := config.LoadConfiguration(configPath, configName, ConfigType)
	if err != nil {
		log.Fatalf("could not load configuration file: %v", err)
	}
	fmt.Println(config.Environment)  <==== Accessing configuration

Note: configPath, configName and configType are passed in as an constant to the application.

Here is the link to the git repository containing the working code: https://github.com/pgaijin66/go-config-yaml

Encrypting at rest

Apart from this, our work is not done here. As we discussed the secrets should be encrypted at rest, we will be also encrypting them before pushing them into the repository. Once config file is added into the repository, i like to encrypt the file using ansible vault as shown below but you can do however

ansible-vault --encrypt --vault-id app-prod@prompt config.yaml

New vault password (project):
Confirm new vault password (project):
Encryption successful

Now when we open this file we will see the file being encrypted and not in plain text like this

$ cat config.yaml

$ANSIBLE_VAULT;1.2;AES256;project
31363961366639393931396663313938333338383063623934353463323633636139366339373764
3762623663316163333163366136613336636265613534340a343266313338343864396233373033
31306239613735346565643364353866363739333431663562356464303031383136636337623063
3537303536653139320a356430306233343337343965366630386164643536343263323136356232
65636663333535326434663733346465633530343231306436663339663432363762323836633463
39336237666666623031346163613865356331346434346531303565323263376638376636356531
38353039353064396231666465326236363435383766343632666165323633643734663562623365
30333939333033646333

Now, as part of the CI/CD step, we will pass this file to the desired location and with proper file permission and control and load it into the application when its needed. Along with this, as part of the process, we will also inject secret variables into desired location with properly audited filesystem permission and acess control.

Why is it different than using .env ?

While it might look same, the key differene is to understand the approach and understand how secrets are passed to the application. Using .env to load environment variables to process.env and then used by application via process.env.PASSWORD is different to using file to load secrets to the aplication. The difference is how we are interfacing secret to the application.

Conclusion

In conclusion, whatever method you use, make sure you do not ignore the ignore files such as dockerignore or gitignore. Make sure you add these files containing secrets into those ignore files. Since .env and yaml files both are stored in plain text if one has access to the server, and are readable by anyone, i would put my best bet on files with tight filesystem permission and tight acl and inject those var into the project as a normal variables rather than environment variable.