ENV variables not loaded with config

[deepakmahakale]

I followed this but I am unable to access the environment variable with Settings

config.rb

Config.setup do |config|
  config.use_env = true
  config.env_prefix = 'SETTINGS'
  config.env_separator = '__'
  config.env_converter = :downcase
  config.env_parse_values = true
end

SETTINGS__SECTION__SERVER_SIZE=1 SETTINGS__SECTION__SERVER=google.com SETTINGS__SECTION__SSL_ENABLED=false rails c

Settings.section.server_size # => nil
Settings.section.server      # => nil
Settings.section.ssl_enabled # => nil

# They are accessible this way
ENV['SETTINGS__SECTION__SERVER_SIZE'] #=> "1"
ENV['SETTINGS__SECTION__SERVER']      #=> "google.com"
ENV['SETTINGS__SECTION__SSL_ENABLED'] #=> "false"

• ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x86_64-darwin20]
• rails (6.1.6)
• config (4.0.0)

[harrisjb]

Similar issue here:

• config 4.0.0
• ruby 2.5.8
• rails 5.1.7

Contents of config/initializers/config.rb

# set up config to look for ENV[key] instead of Settings.key
Config.setup do |config|
  config.use_env = true
end    

rails console does not load into ENV namespace:

$ rails c
Loading development environment (Rails 5.1.7)
[1] pry(main)> ENV['database_host']
=> nil
[2] pry(main)> Settings.database_host
=> "localhost"

[harrisjb]

After review I believe this is expected behavior as the gem is just trying to read from ENV namespace if things are loaded into ENV by Heroku or AWS or some other means. It doesn't intend to write environment variables into the ENV namespace itself, it's just reading from it.

[pkuczynski]

Yes @harrisjb we do not populate things back into ENV, we only read from it. So your reported issue is not valid.

[pkuczynski]

However what @deepakmahakale posted should work. @cjlarose any idea why it does not?

[cjlarose]

I agree that what @deepakmahakale posted should work. @deepakmahakale if you're able to reproduce the issue on a new rails app, we can take a deeper look into what's going on.

[jfadgen]

I am having the same issue as @deepakmahakale , in a new Rails app:

• config 4.0.0
• rails 7.0.3
• ruby 2.7.4

Config.setup do |config|
  config.use_env = true
  config.env_prefix = 'CONFIG'
  config.env_separator = '__'
  config.env_converter = :downcase
  config.env_parse_values = true
end

.env

CONFIG__COGNITO__USER_POOL_ID="test_pool_id"
CONFIG__COGNITO__CLIENT_ID="test_client_id"
CONFIG__COGNITO__CLIENT_SECRET_KEY="test_secret_key"
CONFIG__ANYTHING_TEST="anything at all"

irb(main):001:0> ENV["CONFIG__COGNITO__USER_POOL_ID"]
=> "test_pool_id"
irb(main):002:0> ENV["CONFIG__COGNITO__CLIENT_ID"]
=> "test_client_id"
irb(main):003:0> ENV["CONFIG__COGNITO__CLIENT_SECRET_KEY"]
=> "test_secret_key"
irb(main):004:0> ApplicationConfig.cognito
=> nil
irb(main):005:0> ENV["CONFIG__ANYTHING_TEST"]
=> "anything at all"
irb(main):006:0> ApplicationConfig.anything_test
=> nil

I tried downgrading to config gem 2.2, but without luck. I have another project using that version (with Rails 6.1.4.1, Ruby 2.7.4.) successfully.

Happy to try anything if you have any recommendations, thank you.

[nick-reshetnikov]

@jfadgen, @deepakmahakale, I ran into a similar issue in a recent rails project and figured out that the problem is that ENV variables haven't been loaded yet when the config initializer runs if you're using .env files.

specifically, if you're loading from a .env file, you will likely need load the .env files before the initializers run.

according to https://github.com/bkeepers/dotenv:

dotenv is initialized in your Rails app during the before_configuration callback, which is fired when the Application constant is defined in config/application.rb with class Application < Rails::Application. If you need it to be initialized sooner, you can manually call Dotenv::Railtie.load.

# config/application.rb
Bundler.require(*Rails.groups)

# Load dotenv only in development or test environment
if ['development', 'test'].include? ENV['RAILS_ENV']
  Dotenv::Railtie.load
end

note that if you're using something like heroku, where the env vars are set on the servers, they will likely be available for the initializers. so you may only need to do this for your dev and test environments.

@pkuczynski I think you guys should think about adding support for .env files - if they're detected, your gem should automatically try to load the contents of .env before config/initializers/config.rb runs

[pkuczynski]

The whole concept of this gem is to store config in the per env configuration files. Using .env and gems like dotenv is a different approach for doing the same thing. They kind of contradictory to each other. So I am not sure if we should support this. @cjlarose what do you think?

[nick-reshetnikov]

yeah, I see what you're saying, but the issue is that a lot of teams use .env files for dev/test environments, but manage environment variables directly on staging/prod servers (e.g. thru Heroku config vars). pretty much every project I've worked on has used this hybrid approach. I honestly haven't seen .env.production really used, certainly not with things like heroku.

so, you're kind of leaving the door open to a lot of confusion if you don't support env vars being preloaded thru something like dotenv, which is a very popular tool.

for me, I wanted to use this gem primarily to store certain configs in a yml file format, which allows for nesting keys and other advantages over long, messy .env files. however, I also wanted to be able to override default values in that yml file thru env vars (i.e. on heroku). this allows me to change these configs without committing new yml files. I think that's a legitimate use case.

but in any case, you guys should at least add a disclaimer about this scenario in the README, since I'm sure others have ran into this issue

[jfadgen]

Worth noting that the issue went away for me after changing my Gemfile to bundle
gem "dotenv-rails"
instead of
gem "dotenv".

[nick-reshetnikov]

@jfadgen I in fact saw this issue while using "dotenv-rails"

[nick-reshetnikov]

actually, I just discovered that you can require dotenv/rails-now (https://github.com/bkeepers/dotenv/blob/master/lib/dotenv/rails-now.rb) in your gemfile to solve the issue:

gem 'dotenv-rails', require: 'dotenv/rails-now'

this will ensure that dotenv files are loaded before initializers and solve this issue. I think modifying the gemfile is preferable to messing with application.rb in the manner I showed above.

[kmordan24]

For anyone else coming into this issue and seeing @nick-reshetnikov 's solution, you'll get a warning w/ latest version that dotenv/rails-now is deprecated and instead to use dotenv/load.

gem "dotenv-rails", require: "dotenv/load"

edit: it appears that if your gem is in development/test group, if you try to do a production build it will still try to do the require even though it's in that group, resulting in a LoadError (this might be a bundler bug with specific versions...). So you can either add to application initializer config per the comment above

Or also you can create a file in your /config/initializers that begins with letter after "d", eg "reload_config.rb" and do:

::Config.reload! if Rails.env.development? || Rails.env.test?

to force Config gem to reload and pickup the ENV values now populated from .env file after dotenv is autoloaded.