Drop multi_json support

[ekohl]

These days the built in JSON is fast enough and supporting other implementations can only complicate matters. We can also rely on JRuby to have a correct implementation since the same json gem is also built for Java and included in JRuby.

Note I made an assumption about JRuby because https://rubygems.org/gems/json also lists a Java build.

I was triggered to drop this because tests started to fail for me.

[bastelfreak]

mental note: we probably can/have to adjust puppet-runtime or openvox-server to not include multi_json in the packages

[smortex]

Haha, I just found this PR while trying to find what cause a regression when updating OpenVox on FreeBSD (I had some time today to dig in the issue). It happens that with these packages:

root@puppet:~ # pkg info | grep openv
openvox-agent8-8.24.2          Configuration management framework written in Ruby
openvox-server8-8.12.0         Puppet Server running in the JVM
openvoxdb-terminus8-8.11.0     PuppetDB storeconfigs backend terminus module
rubygem-openvoxserver-ca-3.0.0 Ruby CLI tool to interact with the OpenVox Server Certificate Authority

Applying a catalog fails with a bunch of errors:

root@puppet:~ # puppet agent -t
Info: Refreshing CA certificate
Info: CA certificate is unmodified, using existing CA certificate
Info: Using environment 'production'
Error: Server Error
Info: Loading facts
Notice: Requesting catalog from puppet:8140 (10.0.0.10)
Notice: Catalog compiled by puppet
Info: Caching catalog for puppet.lan
Info: Applying configuration version 'production-a9c260d'
Error: /Stage[main]/Choria::Config/File[/usr/local/etc/choria/machine]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/choria/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x60e6c49b>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x393e85f9>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x495711c3>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/agent]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x6d8f3a7a>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/aggregate]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x3f29730c>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/application]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x4ae27c3e>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/audit]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x40f80d73>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/connector]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x2cb1f1c6>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/data]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x2220ae79>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/discovery]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x245481a5>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/registration]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x6c560228>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/security]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x1566980d>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/util]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0xbbff64e>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/validator]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x4ca1a3c9>
Did you mean?  exec
Error: /Stage[main]/Mcollective::Plugin_dirs/File[/usr/local/share/choria/plugins/mcollective/pluginpackager]: Failed to generate additional resources using 'eval_generate': Error 500 on SERVER: Server Error: Failed to serialize Puppet::FileServing::Metadata for 'modules/mcollective/empty': Could not render_multiple to Puppet::Network::Format[json]: undefined method `except' for #<JSON::Ext::Generator::State:0x32b071d5>
Did you mean?  exec
Notice: Applied catalog in 5.88 seconds

The root caused is not directly visible (exception catched by Puppet), but digging the rabbit hole we land here in multi_json:

undefined method `except' for #<JSON::Ext::Generator::State:0x17455c06>
Did you mean?  exec
/usr/local/lib/ruby/gems/3.3/gems/multi_json-1.19.1/lib/multi_json/adapter.rb:113:in `strip_adapter_key'
/usr/local/lib/ruby/gems/3.3/gems/multi_json-1.19.1/lib/multi_json/adapter.rb:89:in `merged_dump_options'
/usr/local/lib/ruby/gems/3.3/gems/multi_json-1.19.1/lib/multi_json/adapter.rb:66:in `dump'
/usr/local/lib/ruby/gems/3.3/gems/multi_json-1.19.1/lib/multi_json.rb:195:in `dump'
/usr/local/lib/ruby/site_ruby/3.3/puppet/util/json.rb:84:in `dump'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/format_support.rb:107:in `to_json'
json/ext/GeneratorState.java:226:in `generate'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/common.rb:305:in `generate'
uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/json/common.rb:626:in `dump'
/usr/local/lib/ruby/gems/3.3/gems/multi_json-1.19.1/lib/multi_json/adapters/json_gem.rb:48:in `dump'
/usr/local/lib/ruby/gems/3.3/gems/multi_json-1.19.1/lib/multi_json/adapter.rb:66:in `dump'
/usr/local/lib/ruby/gems/3.3/gems/multi_json-1.19.1/lib/multi_json.rb:195:in `dump'
/usr/local/lib/ruby/site_ruby/3.3/puppet/util/json.rb:84:in `dump'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/formats.rb:134:in `render_multiple'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/format_support.rb:28:in `render_multiple'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/api/indirected_routes.rb:157:in `block in do_search'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/api/indirected_routes.rb:187:in `block in first_response_formatter_for'
org/jruby/RubyEnumerable.java:664:in `find'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/api/indirected_routes.rb:186:in `first_response_formatter_for'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/api/indirected_routes.rb:156:in `do_search'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/api/indirected_routes.rb:54:in `block in call'
/usr/local/lib/ruby/site_ruby/3.3/puppet/context.rb:64:in `override'
/usr/local/lib/ruby/site_ruby/3.3/puppet.rb:292:in `override'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/api/indirected_routes.rb:53:in `call'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/api/server/v3.rb:18:in `block in wrap'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/route.rb:85:in `block in process'
org/jruby/RubyArray.java:2009:in `each'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/route.rb:84:in `process'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/route.rb:91:in `process'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/route.rb:91:in `process'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/handler.rb:88:in `block in process'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/handler.rb:71:in `block in with_request_profiling'
/usr/local/lib/ruby/site_ruby/3.3/puppet/util/profiler/around_profiler.rb:59:in `profile'
/usr/local/lib/ruby/site_ruby/3.3/puppet/util/profiler.rb:53:in `profile'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/handler.rb:67:in `with_request_profiling'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/handler.rb:87:in `block in process'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/handler.rb:94:in `respond_to_errors'
/usr/local/lib/ruby/site_ruby/3.3/puppet/network/http/handler.rb:86:in `process'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:69:in `block in handleRequest'
/usr/local/lib/ruby/site_ruby/3.3/puppet/context.rb:64:in `override'
/usr/local/lib/ruby/site_ruby/3.3/puppet.rb:292:in `override'
uri:classloader:/puppetserver-lib/puppet/server/master.rb:68:in `handleRequest'

After applying the patch, the issue vanished 😁

root@puppet:~ # puppet agent -t
Info: Refreshing CA certificate
Info: CA certificate is unmodified, using existing CA certificate
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Notice: Requesting catalog from puppet:8140 (10.0.0.10)
Notice: Catalog compiled by puppet
Info: Caching catalog for puppet.lan
Info: Applying configuration version 'production-a9c260d'
Notice: Applied catalog in 5.04 seconds

Thank you for fixing issues before I find them 😍

[ekohl]

I saw those exact same failures in CI, which prompted me to drop it. Glad it helps you too