fix(manifest): preserve remote link scope

[xylophonez]

Fix manifest child link resolution so manifest paths can fall back to remote stores when the manifest is cached locally but the child data item is not.

Problem

Manifest path resolution can fail even when both of these are true:

• the manifest data item is available and readable
• the linked child data item is available through the configured remote gateway store

The failure occurs when resolving a manifest path such as:

/<manifest-id>/assets/ArticleBlock-Dtwjc54T.js

The manifest itself is fetched, parsed, and linkified, but the generated child link is resolved as local-only. If the child data item is not already in the local cache, resolution fails instead of falling back to the remote gateway store.

Root Cause

dev_manifest:linkify/2 was constructing link options with atom keys:

maps:with([store], Opts)#{ scope => [local, remote]}

The surrounding HyperBEAM option maps use binary keys such as <<"store">> and <<"scope">>.

That meant:

• maps:with([store], Opts) dropped the configured <<"store">> value
• scope => [local, remote] was not read by callers expecting canonical/binary option keys
• hb_cache fell back to local-only scope for the linked child

The fix preserves the existing store and sets the scope with the same binary-key convention used elsewhere:

maps:with([<<"store">>], Opts)#{ <<"scope">> => [local, remote]}

URL Proof

curl -sS -L -m 30 -o /tmp/arweave-proof.out \
  -w 'status=%{http_code} content_type=%{content_type} bytes=%{size_download} final=%{url_effective}\n' \
  'https://arweave.net/42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA/assets/ArticleBlock-Dtwjc54T.js'

Observed:

status=200 content_type=application/javascript bytes=65805 final=https://4nuojs5tw6xtfjbq47dqk6ak7n6tqyr3uxgemkq5z5vmunhxphya.arweave.net/42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA/assets/ArticleBlock-Dtwjc54T.js

The linked child item itself is also available on Arweave:

curl -sS -L -m 30 -o /tmp/arweave-child-proof.out \
  -w 'status=%{http_code} content_type=%{content_type} bytes=%{size_download} final=%{url_effective}\n' \
  'https://arweave.net/oLnQY-EgiYRg9XyO7yZ_mC0Ehy7TFR3UiDhFvxcohC4'

Observed:

status=200 content_type=application/javascript bytes=65805 final=https://uc45ay7beceyiyhvpsho6jt7tawqjbzo2mkr3veihbc36fziqqxa.arweave.net/oLnQY-EgiYRg9XyO7yZ_mC0Ehy7TFR3UiDhFvxcohC4

Clean upstream edge fails for that same manifest child path:

curl -sS -L -m 45 -o /tmp/edge-proof.out \
  -w 'status=%{http_code} content_type=%{content_type} bytes=%{size_download} final=%{url_effective}\n' \
  'http://localhost:28736/42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA/assets/ArticleBlock-Dtwjc54T.js'

Observed:

status=500 content_type=text/html bytes=2693 final=http://localhost:28736/42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA/assets/ArticleBlock-Dtwjc54T.js

The clean-edge node log shows the manifest was fetched from Arweave, then child link resolution failed:

received, status: 200, method: GET, peer: https://arweave.net:443, path: /raw/42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA, body_size: 5868

{necessary_message_not_found,<<>>,
 <<"Link (to link): oLnQY-EgiYRg9XyO7yZ_mC0Ehy7TFR3UiDhFvxcohC4">>}

The fixed branch succeeds under the same store shape:

curl -sS -L -m 45 -o /tmp/fixed-proof.out \
  -w 'status=%{http_code} content_type=%{content_type} bytes=%{size_download} final=%{url_effective}\n' \
  'http://localhost:28735/42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA/assets/ArticleBlock-Dtwjc54T.js'

Observed:

status=200 content_type=application/javascript bytes=65805 final=http://localhost:28735/42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA/assets/ArticleBlock-Dtwjc54T.js

The fixed node log shows it fetching the linked child item from Arweave and returning it:

received, status: 200, method: GET, peer: https://arweave.net:443, path: /raw/oLnQY-EgiYRg9XyO7yZ_mC0Ehy7TFR3UiDhFvxcohC4, body_size: 65805
sent, status: 200, body_size: 65805, method: GET, path: /42jky7O3rzKkMOfHBXgK-304YjulzEYqHc9qyjT3efA/assets/ArticleBlock-Dtwjc54T.js

[samcamwilliams]

Isn't this the opposite of what we want? Implies a deeper problem: We end up with both the atom version and the binary version, possibly with differing values 🤦‍♂️

[samcamwilliams]

No need for the specific manifest test if we have the pinpoint test above, I think.

[xylophonez]

Got it, removed in latest commit