diff --git a/doc/admin-guide/files/index.en.rst b/doc/admin-guide/files/index.en.rst index 38b1db9b41a..f703b972f79 100644 --- a/doc/admin-guide/files/index.en.rst +++ b/doc/admin-guide/files/index.en.rst @@ -39,6 +39,8 @@ Configuration Files ssl_multicert.yaml.en sni.yaml.en storage.yaml.en + storage.config.en + volume.config.en strategies.yaml.en jsonrpc.yaml.en @@ -90,6 +92,14 @@ Configuration Files :doc:`storage.yaml.en` Configures all storage devices and paths to be used for the |TS| cache and defines cache space usage by individual protocols. +:doc:`storage.config.en` + Legacy line-based storage configuration. Loaded when :file:`storage.yaml` + is absent. Use ``traffic_ctl config convert storage`` to migrate. + +:doc:`volume.config.en` + Legacy line-based volume configuration. Loaded together with + :file:`storage.config` when :file:`storage.yaml` is absent. + :doc:`strategies.yaml.en` Configures NextHop strategies used with `remap.config` and replaces parent.config. diff --git a/doc/admin-guide/files/storage.config.en.rst b/doc/admin-guide/files/storage.config.en.rst new file mode 100644 index 00000000000..38f2b803fe3 --- /dev/null +++ b/doc/admin-guide/files/storage.config.en.rst @@ -0,0 +1,194 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +============== +storage.config +============== + +.. configfile:: storage.config + +.. deprecated:: 11.0 + + Use :file:`storage.yaml` instead. :file:`storage.config` and + :file:`volume.config` are still loaded if :file:`storage.yaml` is absent, + for backward compatibility. Use ``traffic_ctl config convert storage`` to + migrate. If both :file:`storage.yaml` and :file:`storage.config` exist, + :file:`storage.yaml` takes precedence and a warning is logged. + +The :file:`storage.config` file (by default, located in +``/usr/local/etc/trafficserver/``) lists all the files, directories, and/or +hard disk partitions that make up the Traffic Server cache. After you +modify the :file:`storage.config` file the new settings will not be effective until Traffic Server is restarted. + +Format +====== + +The format of the :file:`storage.config` file is a series of lines of the form + + *pathname* *size* [ ``volume=``\ *number* ] [ ``id=``\ *string* ] + +where :arg:`pathname` is the name of a partition, directory or file, :arg:`size` is the size of the +named partition, directory or file (in bytes), and :arg:`volume` is the volume number used in the +files :file:`volume.config` and :file:`hosting.config`. :arg:`id` is used for seeding the +:ref:`assignment-table`. You must specify a size for directories; size is optional for files and raw +partitions. :arg:`volume` and :arg:`id` are optional. + +.. note:: + + The :arg:`volume` option is independent of the :arg:`id` option and either can be used with or without the other, + and their ordering on the line is irrelevant. + +.. note:: + + If the :arg:`id` option is used every use must have a unique value for :arg:`string`. + +.. note:: + + Any change to this files can (and almost always will) invalidate the existing cache in its entirety. + +You can use any partition of any size. For best performance: + +- Use raw disk partitions. +- For each disk, make all partitions the same size. +- For each node, use the same number of partitions on all disks. +- Group similar kinds of storage into different volumes. For example + split out SSD's or RAM drives into their own volume. + +Specify pathnames according to your operating system requirements. See +the following examples. In the :file:`storage.config` file, a formatted or +raw disk must be at least 128 MB. + +When using raw disk or partitions, you should make sure the :ts:cv:`Traffic +Server user ` used by the Traffic Server process +has read and write privileges on the raw disk device or partition. One good +practice is to make sure the device file is set with 'g+rw' and the Traffic +Server user is in the group which owns the device file. However, some +operating systems have stronger requirements - see the following examples for +more information. + +As with standard ``records.yaml`` integers, human readable prefixes are also +supported. They include + + - ``K`` Kilobytes (1024 bytes) + - ``M`` Megabytes (1024^2 or 1,048,576 bytes) + - ``G`` Gigabytes (1024^3 or 1,073,741,824 bytes) + - ``T`` Terabytes (1024^4 or 1,099,511,627,776 bytes) + +.. _assignment-table: + +Assignment Table +---------------- + +Each storage element defined in :file:`storage.config` is divided in to :term:`stripes `. The +assignment table maps from an object URL to a specific stripe. The table is initialized based on a +pseudo-random process which is seeded by hashing a string for each stripe. This string is composed +of a base string, an offset (the start of the stripe on the storage element), and the length of the +stripe. By default the path for the storage is used as the base string. This ensures that each +stripe has a unique string for the assignment hash. This does make the assignment table very +sensitive to the path for the storage elements and changing even one can have a cascading effect +which will effectively clear most of the cache. This can be problem when drives fail and a system +reboot causes the path names to change. + +The :arg:`id` option can be used to create a fixed string that an administrator can use to keep the +assignment table consistent by maintaining the mapping from physical device to base string even in the presence of hardware changes and failures. + +Examples +======== + +The following basic example shows 128 MB of cache storage in the +``/big_dir`` directory:: + + /big_dir 134217728 + +You can use the ``.`` symbol for the current directory. Here is an +example for 64 MB of cache storage in the current directory:: + + . 134217728 + +As an alternative, using the human readable prefixes, you can express a 64GB +cache file with:: + + /really_big_dir 64G + + +.. note:: + When using on-filesystem cache disk storage, you can only have one such + directory specified. This will be addressed in a future version. + +Linux Example +------------- +.. note:: + Rather than refer to disk devices like ``/dev/sda``, ``/dev/sdb``, etc., + modern Linux supports `alternative symlinked names for disk devices + `_ in the ``/dev/disk`` + directory structure. As noted for the :ref:`assignment-table` the path used for the disk can effect + the cache if it changes. This can be ameliorated in some cases by using one of the alternate paths + in via ``/dev/disk``. Note that if the ``by-id`` or ``by-path`` style is used, replacing a failed drive will cause + that path to change because the new drive will have a different physical ID or path. The original hash string can + be kept by adding :arg:`id` or :arg:`path` with the original path to the storage line. + + If this is not sufficient then the :arg:`id` or :arg:`path` argument should be used to create a more permanent + assignment table. An example would be:: + + /dev/sde id=cache.disk.0 + /dev/sdg id=cache.disk.1 + +The following example will use an entire raw disk in the Linux operating +system:: + + /dev/disk/by-id/[DiskA_ID] volume=1 + /dev/disk/by-path/[DiskB_Path] volume=2 + +In order to make sure :program:`traffic_server` will have access to this disk +you can use :manpage:`udev(7)` to persistently set the right permissions. The +following rules are targeted for an Ubuntu system, and stored in +``/etc/udev/rules.d/51-cache-disk.rules``:: + + # Assign DiskA and DiskB to the tserver group + # make the assignment final, no later changes allowed to the group! + SUBSYSTEM=="block", KERNEL=="sd[ef]", GROUP:="tserver" + +In order to apply these settings, trigger a reload with :manpage:`udevadm(8)`::: + + udevadm trigger --subsystem-match=block + + +FreeBSD Example +--------------- + +Starting with 5.1 FreeBSD dropped support for explicit raw devices. All +devices on FreeBSD can be accessed raw now. + +The following example will use an entire raw disk in the FreeBSD +operating system:: + + /dev/ada1 + /dev/ada2 + +In order to make sure :program:`traffic_server` will have access to this disk +you can use :manpage:`devfs(8)` to persistently set the right permissions. The +following rules are stored in :manpage:`devfs.conf(5)`:: + + # Assign /dev/ada1 and /dev/ada2 to the tserver user + own ada[12] tserver:tserver + +Advanced +-------- + +Because relative paths in :file:`storage.config` are relative to the base prefix, when using customized runroot +it may be necessary to adjust such paths in :file:`storage.config` or adjust ``runroot.yaml`` itself. +Despite the name, the cachedir value is not used for this file. diff --git a/doc/admin-guide/files/storage.yaml.en.rst b/doc/admin-guide/files/storage.yaml.en.rst index cbbbde4a705..99492980290 100644 --- a/doc/admin-guide/files/storage.yaml.en.rst +++ b/doc/admin-guide/files/storage.yaml.en.rst @@ -559,3 +559,27 @@ shares the remaining 1.5GB (4GB - 2GB - 512MB) and caches objects up to 1MB. - id: 3 size: 40% ram_cache_cutoff: 1M + + +Backward compatibility +====================== + +For backward compatibility |TS| also accepts the legacy +:file:`storage.config` and :file:`volume.config` files. The selection rule at +startup is: + +* If :file:`storage.yaml` exists in the configuration directory, it is loaded + and any :file:`storage.config` / :file:`volume.config` present alongside it + is ignored. A warning is logged in this case so that operators notice the + legacy file is being skipped. +* If :file:`storage.yaml` does not exist, |TS| falls back to + :file:`storage.config` (required) and :file:`volume.config` (optional). + A missing :file:`volume.config` is treated as "no volumes configured". + +Use the ``traffic_ctl config convert storage`` command to convert a legacy +configuration to :file:`storage.yaml`:: + + traffic_ctl config convert storage storage.config volume.config storage.yaml + +The legacy formats are described in :doc:`storage.config.en` and +:doc:`volume.config.en`. diff --git a/doc/admin-guide/files/volume.config.en.rst b/doc/admin-guide/files/volume.config.en.rst new file mode 100644 index 00000000000..2fb15a2ea78 --- /dev/null +++ b/doc/admin-guide/files/volume.config.en.rst @@ -0,0 +1,203 @@ +.. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +============= +volume.config +============= + +.. configfile:: volume.config + +.. deprecated:: 11.0 + + Use :file:`storage.yaml` instead. :file:`storage.config` and + :file:`volume.config` are still loaded if :file:`storage.yaml` is absent, + for backward compatibility. Use ``traffic_ctl config convert storage`` to + migrate. + +The :file:`volume.config` file enables you to manage your cache space more +efficiently and restrict disk usage by creating cache volumes of +different sizes. By distributing the cache across multiple volumes, +you can help decrease single-lock pressure when there are not many hard drives +present. You can further configure these volumes to store data from certain +origin servers and/or domains in the :file:`hosting.config` file. + +Format +====== + +For each volume you want to create, enter a line with the following +format: :: + + volume=volume_number scheme=protocol_type size=volume_size + +where ``volume_number`` is a number between 1 and 255 (the maximum +number of volumes is 255) and ``protocol_type`` is ``http``. Traffic +Server supports ``http`` for HTTP volume types; ``volume_size`` is the +amount of cache space allocated to the volume. This value can be either +a percentage of the total cache space or an absolute value. The absolute +value must be a multiple of 128 MB, where 128 MB is the smallest value. +If you specify a percentage, then the size is rounded down to the +closest multiple of 128 MB. + +Each volume is striped across several disks to achieve parallel I/O. For +example: if there are four disks, then a 1-GB volume will have 256 MB on +each disk (assuming each disk has enough free space available). If you +do not allocate all the disk space in the cache, then the extra disk +space is not used. You can use the extra space later to create new +volumes without deleting and clearing the existing volumes. + +.. important:: + + Changing this file to add, remove or modify volumes effectively invalidates + the cache. + + +Optional ramcache setting +------------------------- + +You can also add an option ``ramcache=true/false`` to the volume configuration +line. True is the default setting and so not needed unless you want to explicitly +set it. Setting ``ramcache=false`` will disable the ramcache that normally +sits in front of a volume. This may be desirable if you are using something like +ramdisks, to avoid wasting RAM and cpu time on double caching objects. + + +Optional directory entry sizing +------------------------------- + +You can also add an option ``avg_obj_size=`` to the volume configuration +line. This overrides the global :ts:cv:`proxy.config.cache.min_average_object_size` +configuration for this volume. The size supports multipliers (K, M, G, T) for +convenience (e.g., ``avg_obj_size=64K`` or ``avg_obj_size=1M``). This is useful +if you have a volume that is dedicated for say very small objects, and you need +a lot of directory entries to store them. + +Optional fragment size setting +------------------------------ + +You can also add an option ``fragment_size=`` to the volume configuration +line. This overrides the global :ts:cv:`proxy.config.cache.target_fragment_size` +configuration for this volume. The size supports multipliers (K, M, G, T) for +convenience (e.g., ``fragment_size=512K`` or ``fragment_size=2M``). This allows +for a smaller, or larger, fragment size for a particular volume. This may be +useful together with ``avg_obj_size`` as well, since a larger fragment size could +reduce the number of directory entries needed for a large object. + +Note that this setting has a maximum value of 4MB. + +Optional RAM cache size allocation +----------------------------------- + +You can add an option ``ram_cache_size=`` to the volume configuration line +to allocate a dedicated RAM cache pool for this volume. The size supports +multipliers (K, M, G, T) for convenience (e.g., ``ram_cache_size=512M`` or +``ram_cache_size=2G``). Setting ``ram_cache_size=0`` disables the RAM cache +for this volume, which is equivalent to ``ramcache=false``. + +When ``ram_cache_size`` is specified for a volume, that amount is **automatically +subtracted** from the global :ts:cv:`proxy.config.cache.ram_cache.size` setting, +and the remainder is shared among volumes without private allocations. This ensures +total RAM cache usage never exceeds the configured global limit. + +For example, if the global RAM cache size is 4GB and you allocate 1GB to volume 1 +and 512MB to volume 2, the remaining 2.5GB will be distributed among other volumes +using the normal proportional allocation based on disk space. + +**Important notes:** + +* If the sum of all ``ram_cache_size`` allocations exceeds the global RAM cache size, + Traffic Server will fail to start with a fatal error. Increase + :ts:cv:`proxy.config.cache.ram_cache.size` or reduce the per-volume allocations. +* If ``ramcache=false`` is set alongside ``ram_cache_size``, the ``ram_cache_size`` + is ignored (with a warning) since the RAM cache is disabled for that volume. +* This setting only takes effect when :ts:cv:`proxy.config.cache.ram_cache.size` + is set to a positive value (not ``-1`` for automatic sizing). + +Optional RAM cache cutoff override +----------------------------------- + +You can add an option ``ram_cache_cutoff=`` to the volume configuration line +to override the global :ts:cv:`proxy.config.cache.ram_cache_cutoff` setting for +this specific volume. The size supports multipliers (K, M, G, T) for convenience +(e.g., ``ram_cache_cutoff=64K`` or ``ram_cache_cutoff=1M``). + +This cutoff determines the maximum object size that will be stored in the RAM cache. +Objects larger than this size will only be stored on disk. Setting different cutoffs +per volume allows you to: + +* Use larger cutoffs for volumes serving frequently accessed large objects +* Use smaller cutoffs for volumes with many small objects to maximize RAM cache hits +* Disable RAM caching entirely for certain objects by setting a very low cutoff + +Exclusive spans and volume sizes +================================ + +In the following sample configuration 2 spans `/dev/disk1` and `/dev/disk2` are defined +in :file:`storage.config`, where span `/dev/disk2` is assigned to `volume 3` exclusively +(`volume 3` is forced to an "exclusive" span `/dev/disk2`). +In :file:`volume.config` there are 3 volumes defined, where `volume 1` and `volume 2` +occupy span `/dev/disk1` taking each 50% of its space and `volume 3` takes 100% of span +`/dev/disk2` exclusively. + +storage.config:: + + /dev/disk1 + /dev/disk2 volume=3 # <- exclusive span + +volume.config:: + + volume=1 scheme=http size=50% + volume=2 scheme=http size=50% + volume=3 scheme=http size=512 # <- volume forced to a specific exclusive span + +It is important to note that when percentages are used to specify volume sizes +and "exclusive" spans are assigned (forced) to a particular volume (in this case `volume 3`), +the "exclusive" spans (in this case `/dev/disk2`) are excluded from the total cache +space when the "non-forced" volumes sizes are calculated (in this case `volume 1` and `volume 2`). + + +Examples +======== + +The following example partitions the cache across 5 volumes to decreasing +single-lock pressure for a machine with few drives. The last volume being +an example of one that might be composed of purely ramdisks so that the +ramcache has been disabled.:: + + volume=1 scheme=http size=20% + volume=2 scheme=http size=20% + volume=3 scheme=http size=20% + volume=4 scheme=http size=20% avg_obj_size=4K + volume=5 scheme=http size=20% ramcache=false fragment_size=512K + +The following example shows advanced RAM cache configuration with dedicated +allocations and custom cutoffs:: + + # Volume 1: General content with 2GB dedicated RAM cache + volume=1 scheme=http size=40% ram_cache_size=2G + + # Volume 2: Small API responses with custom cutoff and 512MB RAM cache + volume=2 scheme=http size=20% ram_cache_size=512M ram_cache_cutoff=64K + + # Volume 3: Large media with higher cutoff for thumbnails + volume=3 scheme=http size=40% ram_cache_cutoff=1M + +In this example, assuming a global ``proxy.config.cache.ram_cache.size`` of 4GB: + +* Volume 1 gets a dedicated 2GB RAM cache allocation +* Volume 2 gets a dedicated 512MB RAM cache allocation and only caches objects up to 64KB +* Volume 3 shares from the remaining 1.5GB pool (4GB - 2GB - 512MB) and caches objects up to 1MB +* The automatic subtraction ensures total RAM usage stays within the 4GB limit diff --git a/include/iocore/cache/Store.h b/include/iocore/cache/Store.h index d6c7b0f23b9..b742264bb3a 100644 --- a/include/iocore/cache/Store.h +++ b/include/iocore/cache/Store.h @@ -158,7 +158,7 @@ struct Store { Store(){}; ~Store(); - unsigned n_spans_in_config = 0; ///< The number of disks/paths defined in storage.yaml + unsigned n_spans_in_config = 0; ///< The number of disks/paths defined in storage.yaml (or legacy storage.config) unsigned n_spans = 0; ///< The number of disks/paths we could actually read and parse Span **spans = nullptr; diff --git a/include/tscore/Filenames.h b/include/tscore/Filenames.h index b36e282aebe..26a464d8376 100644 --- a/include/tscore/Filenames.h +++ b/include/tscore/Filenames.h @@ -25,7 +25,10 @@ namespace ts { namespace filename { - constexpr const char *STORAGE = "storage.yaml"; + constexpr const char *STORAGE_YAML = "storage.yaml"; + constexpr const char *STORAGE = "storage.config"; //< deprecated by STORAGE_YAML + constexpr const char *VOLUME = "volume.config"; //< deprecated by STORAGE_YAML + constexpr const char *RECORDS = "records.yaml"; constexpr const char *PLUGIN = "plugin.config"; constexpr const char *PLUGIN_YAML = "plugin.yaml"; diff --git a/src/iocore/cache/Cache.cc b/src/iocore/cache/Cache.cc index 69692d890d9..c36118a30f5 100644 --- a/src/iocore/cache/Cache.cc +++ b/src/iocore/cache/Cache.cc @@ -971,6 +971,6 @@ ink_cache_init(ts::ModuleVersion v) Result result = theCacheStore.read_config(); if (result.failed()) { - Fatal("Failed to read cache configuration %s: %s", ts::filename::STORAGE, result.message()); + Fatal("Failed to read cache configuration: %s", result.message()); } } diff --git a/src/iocore/cache/Store.cc b/src/iocore/cache/Store.cc index c0896dea3b8..2b3cf170932 100644 --- a/src/iocore/cache/Store.cc +++ b/src/iocore/cache/Store.cc @@ -34,6 +34,8 @@ #include "tscore/ink_file.h" #include "tsutil/DbgCtl.h" +#include "swoc/swoc_file.h" + #if defined(__linux__) #include #endif @@ -218,25 +220,96 @@ Span::~Span() Result Store::read_config() { - ats_scoped_str storage_path(RecConfigReadConfigPath(nullptr, ts::filename::STORAGE)); + config::StorageConfig parsed; + + // Check storage.yaml, storage.config and volume.config + ats_scoped_str storage_yaml_path(RecConfigReadConfigPath(nullptr, ts::filename::STORAGE_YAML)); + const bool yaml_exists = storage_yaml_path && swoc::file::exists(swoc::file::path(storage_yaml_path.get())); + + ats_scoped_str storage_config_path(RecConfigReadConfigPath(nullptr, ts::filename::STORAGE)); + const bool storage_config_exists = storage_config_path && swoc::file::exists(swoc::file::path(storage_config_path.get())); - Note("%s loading ...", ts::filename::STORAGE); + ats_scoped_str volume_config_path(RecConfigReadConfigPath("proxy.config.cache.volume_filename", ts::filename::VOLUME)); + const bool volume_config_exists = volume_config_path && swoc::file::exists(swoc::file::path(volume_config_path.get())); config::StorageParser parser; - auto parse_result = parser.parse(storage_path.get()); - if (!parse_result.ok()) { - std::string msg; - for (auto const &annotation : parse_result.errata) { - if (!msg.empty()) { - msg += "; "; + + if (yaml_exists) { + Note("%s loading ...", ts::filename::STORAGE_YAML); + + auto parse_result = parser.parse(storage_yaml_path.get()); + if (!parse_result.ok()) { + std::string msg; + for (auto const &annotation : parse_result.errata) { + if (!msg.empty()) { + msg += "; "; + } + msg += std::string(annotation.text()); } - msg += std::string(annotation.text()); + return Result::failure("failed to load %s: %s", ts::filename::STORAGE_YAML, msg.c_str()); } - return Result::failure("failed to load %s: %s", ts::filename::STORAGE, msg.c_str()); + parsed = std::move(parse_result.value); + + // If a legacy storage.config or volume.config also exists, warn that it is being ignored. + if (storage_config_exists) { + Note("%s exists alongside %s; the legacy file is ignored. " + "To resolve, either: (a) migrate %s to %s (e.g. 'traffic_ctl config convert storage') and remove %s, " + "or (b) remove %s to fall back to %s.", + ts::filename::STORAGE, ts::filename::STORAGE_YAML, ts::filename::STORAGE, ts::filename::STORAGE_YAML, + ts::filename::STORAGE, ts::filename::STORAGE_YAML, ts::filename::STORAGE); + } + + if (volume_config_exists) { + Note("%s exists alongside %s; the legacy file is ignored. " + "To resolve, either: (a) migrate %s to %s (e.g. 'traffic_ctl config convert storage') and remove %s, " + "or (b) remove %s to fall back to %s.", + ts::filename::VOLUME, ts::filename::STORAGE_YAML, ts::filename::VOLUME, ts::filename::STORAGE_YAML, ts::filename::VOLUME, + ts::filename::STORAGE_YAML, ts::filename::VOLUME); + } + } else { + // Fall back to legacy storage.config + volume.config. + Note("%s not found, falling back to %s + %s", ts::filename::STORAGE_YAML, ts::filename::STORAGE, ts::filename::VOLUME); + + if (!storage_config_exists) { + return Result::failure("neither %s nor %s found", ts::filename::STORAGE_YAML, ts::filename::STORAGE); + } + + auto storage_result = parser.parse(storage_config_path.get()); + if (!storage_result.ok()) { + std::string msg; + for (auto const &annotation : storage_result.errata) { + if (!msg.empty()) { + msg += "; "; + } + msg += std::string(annotation.text()); + } + return Result::failure("failed to load %s: %s", ts::filename::STORAGE, msg.c_str()); + } + + config::StorageConfig volume_config; + if (volume_config_exists) { + config::VolumeParser volume_parser; + config::ConfigResult volume_result = volume_parser.parse(volume_config_path.get()); + if (!volume_result.ok()) { + std::string msg; + for (auto const &annotation : volume_result.errata) { + if (!msg.empty()) { + msg += "; "; + } + msg += std::string(annotation.text()); + } + return Result::failure("failed to load %s: %s", ts::filename::VOLUME, msg.c_str()); + } + volume_config = std::move(volume_result.value); + } else { + Warning("%s not found", ts::filename::VOLUME); + } + + parsed = config::merge_legacy_storage_configs(storage_result.value, volume_config); } // Convert StorageVolumeEntry -> ConfigVol and populate config_volumes. - for (auto const &vv : parse_result.value.volumes) { + for (auto const &vv : parsed.volumes) { auto *vol = new ConfigVol(); vol->number = vv.id; vol->scheme = (vv.scheme == "http") ? CacheType::HTTP : CacheType::NONE; @@ -261,7 +334,7 @@ Store::read_config() } config_volumes.complement(); - auto const &storage_spans = parse_result.value.spans; + auto const &storage_spans = parsed.spans; this->n_spans_in_config = storage_spans.size(); int n_dsstore = 0; @@ -331,7 +404,11 @@ Store::read_config() } } - Note("%s finished loading", ts::filename::STORAGE); + if (yaml_exists) { + Note("%s finished loading", ts::filename::STORAGE_YAML); + } else { + Note("%s & %s finished loading", ts::filename::STORAGE, ts::filename::VOLUME); + } return Result::ok(); } diff --git a/src/records/RecordsConfig.cc b/src/records/RecordsConfig.cc index 4586f039a2d..37b12a0d216 100644 --- a/src/records/RecordsConfig.cc +++ b/src/records/RecordsConfig.cc @@ -852,6 +852,8 @@ static constexpr RecordElement RecordsConfig[] = , {RECT_CONFIG, "proxy.config.cache.hosting_filename", RECD_STRING, ts::filename::HOSTING, RECU_DYNAMIC, RR_NULL, RECC_NULL, nullptr, RECA_NULL} , + {RECT_CONFIG, "proxy.config.cache.volume_filename", RECD_STRING, ts::filename::VOLUME, RECU_RESTART_TS, RR_NULL, RECC_NULL, nullptr, RECA_NULL} + , {RECT_CONFIG, "proxy.config.cache.permit.pinning", RECD_INT, "0", RECU_DYNAMIC, RR_NULL, RECC_INT, "[0-1]", RECA_NULL} , // # default the ram cache size to AUTO_SIZE (-1) diff --git a/src/traffic_layout/info.cc b/src/traffic_layout/info.cc index dfdbc6915a0..b04e61b64ab 100644 --- a/src/traffic_layout/info.cc +++ b/src/traffic_layout/info.cc @@ -176,7 +176,7 @@ produce_layout(bool json) print_var(ts::filename::REMAP, RecConfigReadConfigPath("proxy.config.url_remap.filename"), json); print_var(ts::filename::PLUGIN, RecConfigReadConfigPath(nullptr, ts::filename::PLUGIN), json); print_var(ts::filename::SSL_MULTICERT, RecConfigReadConfigPath("proxy.config.ssl.server.multicert.filename"), json); - print_var(ts::filename::STORAGE, RecConfigReadConfigPath(nullptr, ts::filename::STORAGE), json); + print_var(ts::filename::STORAGE_YAML, RecConfigReadConfigPath(nullptr, ts::filename::STORAGE_YAML), json); print_var(ts::filename::HOSTING, RecConfigReadConfigPath("proxy.config.cache.hosting_filename"), json); print_var(ts::filename::IP_ALLOW, RecConfigReadConfigPath("proxy.config.cache.ip_allow.filename"), json, true); if (json) { diff --git a/src/traffic_server/traffic_server.cc b/src/traffic_server/traffic_server.cc index a9ea60fcab1..db209d32b84 100644 --- a/src/traffic_server/traffic_server.cc +++ b/src/traffic_server/traffic_server.cc @@ -784,7 +784,7 @@ register_config_files() ConfigSource::FileOnly); // Static (non-reloadable) files only. - reg.register_static_file("storage", ts::filename::STORAGE, {}, true); + reg.register_static_file("storage", ts::filename::STORAGE_YAML); reg.register_static_file("socks", ts::filename::SOCKS, "proxy.config.socks.socks_config_file"); reg.register_static_file("plugin", ts::filename::PLUGIN); reg.register_static_file("jsonrpc", ts::filename::JSONRPC, "proxy.config.jsonrpc.filename"); diff --git a/tests/gold_tests/autest-site/trafficserver.test.ext b/tests/gold_tests/autest-site/trafficserver.test.ext index 4f38cf8dd69..8d909395381 100755 --- a/tests/gold_tests/autest-site/trafficserver.test.ext +++ b/tests/gold_tests/autest-site/trafficserver.test.ext @@ -60,7 +60,8 @@ def MakeATSProcess( dump_runroot=True, enable_proxy_protocol=False, enable_proxy_protocol_cp_src=False, - disable_log_checks=False): + disable_log_checks=False, + use_legacy_storage=False): """Create a traffic server process. :param block_for_debug: if True, causes traffic_server to run with the @@ -68,6 +69,9 @@ def MakeATSProcess( triggered by running traffic_server under a debugger. In the debugger, `set cmd_block = 0`, set any desired break points, then `c` to continue to let the test proceed. + :param use_legacy_storage: if True, skip the default storage.yaml setup + so the test can stage legacy storage.config / volume.config files and + exercise the runtime fallback path. """ ##################################### # common locations @@ -148,6 +152,8 @@ def MakeATSProcess( p.chownForATSProcess(config_dir) for f in os.listdir(cfg_dir): + if use_legacy_storage and f == 'storage.yaml': + continue p.Setup.CopyAs(os.path.join(cfg_dir, f), config_dir) ######################################################### @@ -317,7 +323,8 @@ def MakeATSProcess( fname = "storage.yaml" tmpname = os.path.join(config_dir, fname) - p.Disk.File(tmpname, id=make_id(fname), typename="ats:config") + if not use_legacy_storage: + p.Disk.File(tmpname, id=make_id(fname), typename="ats:config") # The big motivation in exposing this file is that we need to tell the traffic_ctl # where to find the socket to interact with the TS. traffic_ctl cannot rely only diff --git a/tests/gold_tests/config/legacy_storage_config/storage.config b/tests/gold_tests/config/legacy_storage_config/storage.config new file mode 100644 index 00000000000..6b323919a73 --- /dev/null +++ b/tests/gold_tests/config/legacy_storage_config/storage.config @@ -0,0 +1,4 @@ +# Legacy storage.config for the legacy fallback autest. +# The 'storage' path is relative to the ATS run directory; MakeATSProcess +# pre-creates it under //storage. +storage 256M diff --git a/tests/gold_tests/config/legacy_storage_config/volume.config b/tests/gold_tests/config/legacy_storage_config/volume.config new file mode 100644 index 00000000000..e4331333a6d --- /dev/null +++ b/tests/gold_tests/config/legacy_storage_config/volume.config @@ -0,0 +1,2 @@ +# Legacy volume.config for the legacy fallback autest. +volume=1 scheme=http size=100% diff --git a/tests/gold_tests/config/legacy_storage_config_fallback.test.py b/tests/gold_tests/config/legacy_storage_config_fallback.test.py new file mode 100644 index 00000000000..cec3a939adb --- /dev/null +++ b/tests/gold_tests/config/legacy_storage_config_fallback.test.py @@ -0,0 +1,94 @@ +''' +Verify the legacy storage.config + volume.config fallback path. +''' +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +from enum import Enum + +Test.Summary = ''' +Verify Apache Traffic Server falls back to legacy storage.config and +volume.config when storage.yaml is absent. +''' + + +class LegacyStorageConfigFallbackTest: + """ + When storage.yaml is missing, ATS should load the legacy + storage.config + volume.config files instead. + """ + + class State(Enum): + """ + State of process + """ + INIT = 0 + RUNNING = 1 + + def __init__(self): + self.state = self.State.INIT + self.__setupTS() + + def __setupTS(self): + self.ts = Test.MakeATSProcess("ts", use_legacy_storage=True) + + self.ts.Setup.CopyAs( + os.path.join(Test.TestDirectory, 'legacy_storage_config', 'storage.config'), self.ts.Variables.CONFIGDIR) + self.ts.Setup.CopyAs( + os.path.join(Test.TestDirectory, 'legacy_storage_config', 'volume.config'), self.ts.Variables.CONFIGDIR) + + self.ts.Disk.records_config.update({ + 'proxy.config.diags.debug.enabled': 1, + 'proxy.config.diags.debug.tags': 'cache_init', + }) + + self.ts.Disk.diags_log.Content += Testers.ContainsExpression( + r'storage\.yaml not found, falling back to storage\.config \+ volume\.config', + 'storage.yaml fallback Note should be logged when only legacy files are present') + + self.ts.Disk.diags_log.Content += Testers.ContainsExpression( + r'storage\.config & volume\.config finished loading', + 'Legacy storage.config + volume.config should be reported as finished loading') + + def __checkProcessBefore(self, tr): + if self.state == self.State.RUNNING: + tr.StillRunningBefore = self.ts + else: + tr.Processes.Default.StartBefore(self.ts) + self.state = self.State.RUNNING + + def __checkProcessAfter(self, tr): + assert (self.state == self.State.RUNNING) + tr.StillRunningAfter = self.ts + + def __testFallback(self): + """ + ATS starts cleanly using legacy storage.config + volume.config + when storage.yaml is absent. + """ + tr = Test.AddTestRun("Verify ATS started cleanly with legacy storage.config + volume.config") + self.__checkProcessBefore(tr) + tr.Processes.Default.Env = self.ts.Env + tr.Processes.Default.Command = 'traffic_ctl --debug metric get proxy.process.cache.bytes_total' + tr.Processes.Default.ReturnCode = 0 + self.__checkProcessAfter(tr) + + def run(self): + self.__testFallback() + + +LegacyStorageConfigFallbackTest().run()