Skip to content

Add author/dc:creator support to fulltext feed output using feedsmith#964

Draft
Copilot wants to merge 11 commits intomasterfrom
copilot/add-author-element-to-fulltext-feed
Draft

Add author/dc:creator support to fulltext feed output using feedsmith#964
Copilot wants to merge 11 commits intomasterfrom
copilot/add-author-element-to-fulltext-feed

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 20, 2026

  • Replace feed library with feedsmith for native dc:creator support
  • Map item.creator from rss-parser → dc: { creator } for RSS and authors for JSON Feed
  • Fix build: upgrade TypeScript from ^4.9.5 to ^5.0.0
  • Remove skipLibCheck: true — upgraded @types/node to ^22 instead
  • Delete src/author.test.ts (tested feedsmith directly, not our code)
  • Use ...item spread in getFullTextFeed item mapping
  • Use ...data spread in feedToRss function
  • Handle Atom author and id via rss-parser's generic type (no customFields needed — already present at runtime)
  • Clean JSON Feed output — explicit property mapping to prevent rss-parser property leakage
  • Add agents.md documenting success criteria and test feeds
  • Build passes, all tests pass
Original prompt

This section details on the original issue you should resolve

<issue_title>[Feature Request] Add the author element to the fulltext feed (when specified on the original feed)</issue_title>
<issue_description>Hello,

Using this script, I noticed the author of an article is not set on the fulltext feed, even if this information is on the original feed.
I would greatly appreciate to see this added, when it exists on the original feed, on a future release.

Here is the missing tag on an RSS feed :
<author>...</author>

Sometimes replaced with : <dc:creator>...</dc:creator>
The creator element is from the Dublin Core Metadata Initiative.
Here's the schema: https://dublincore.org/schemas/xmls/qdc/2008/02/11/dc.xsd
This element is seen all the time on feeds associated with WordPress blogs. It has become popular and generally preferred over the author element because the latter requires publishing an email address. The format for the creator element content is less restrictive and usually is just a name or username/handle.</issue_description>

Comments on the Issue (you are @copilot in this section)

@whtsky I'm not an RSS export and to be honest have no context on this field or the expected usage. would appreciate an PR for this @whtsky could you provide an test link for this? i could let copilot try working on this

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
fulltextrssplz Ready Ready Preview, Comment Mar 25, 2026 10:51am

Map item.creator from rss-parser (which captures both <author> and
<dc:creator> elements) to the feed library's author format.

For RSS2 output, post-process the XML to inject <dc:creator> elements
since the feed library only outputs <author> when both email and name
are present. The dc:creator element is widely used (especially by
WordPress) and preferred over author as it doesn't require an email.

For JSON Feed output, the author name is automatically included by
the feed library.

Also adds comprehensive unit tests for the new functionality.

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
…ction

- Move parseCreator and addDcCreators to src/author.ts for clean imports
- Import from author.ts in both app.ts and author.test.ts
- Sanitize ]]> in author names to prevent CDATA injection
- Add test for CDATA injection protection

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
Copilot AI changed the title [WIP] Add author element to the fulltext feed if specified Add author/dc:creator support to fulltext feed output Mar 20, 2026
Copilot AI requested a review from whtsky March 20, 2026 14:45
src/author.ts Outdated
const newlinePos = result.lastIndexOf('\n', closeIdx)
const itemIndent = result.substring(newlinePos + 1, closeIdx)
const childIndent = itemIndent + ' '
const dcLine = '\n' + childIndent + `<dc:creator><![CDATA[${name}]]></dc:creator>`
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this implementation feels too hack. if our current library don't support this, could you explore and find a library with good supporting on author & creator fields>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced the feed library with feedsmith which has native support for Dublin Core namespace elements including dc:creator. The hacky XML post-processing in src/author.ts is now completely removed — feedsmith handles dc:creator natively via dc: { creator: '...' } on items, and automatically manages namespace declarations. See commit 4ca6317.

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
Switch from the `feed` library (which required hacky XML post-processing
for dc:creator) to `feedsmith` which natively supports Dublin Core
namespace elements including dc:creator.

- Replace feed generation with feedsmith's generateRssFeed/generateJsonFeed
- Map item.creator from rss-parser directly to dc.creator in feedsmith
- Remove src/author.ts (no longer needed - no XML string manipulation)
- Rewrite tests to verify feedsmith's native dc:creator output
- Remove `feed` dependency, add `feedsmith` dependency

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 20, 2026

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • download.cypress.io
    • Triggering command: /opt/hostedtoolcache/node/24.14.0/x64/bin/node /opt/hostedtoolcache/node/24.14.0/x64/bin/node index.js --exec install (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title Add author/dc:creator support to fulltext feed output Add author/dc:creator support to fulltext feed output using feedsmith Mar 20, 2026
Copilot AI requested a review from whtsky March 20, 2026 15:09
@whtsky
Copy link
Copy Markdown
Owner

whtsky commented Mar 20, 2026

@copilot the project can't even build

feedsmith uses const type parameters (TS 5.0+) in its .d.cts files.
Upgrade TypeScript from ^4.9.5 to ^5.0.0 and add skipLibCheck for
@types/node compatibility.

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 20, 2026

@copilot the project can't even build

Fixed in 5aef183. The build was failing because feedsmith uses TypeScript 5.0+ const type parameters in its .d.cts declaration files, but the project was on TypeScript 4.9.5. Upgraded TypeScript to ^5.0.0 and added skipLibCheck: true for @types/node compatibility. Build and all 9 tests pass now.

@@ -0,0 +1,132 @@
// @ts-nocheck
import { generateRssFeed, generateJsonFeed } from 'feedsmith'
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot don't think this file should still exist

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted in 0d10fee. The file was testing feedsmith's API directly rather than our code.

tsconfig.json Outdated
"esModuleInterop": true,
"allowSyntheticDefaultImports": true
"allowSyntheticDefaultImports": true,
"skipLibCheck": true
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is unacceptable. but you're welcome to upgrade the node version to 24 if @types/node is blocking in your way @copilot

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed skipLibCheck: true and upgraded @types/node to ^22 instead, which resolves the TS 5 compatibility issue. See 0d10fee.

newItem.content = content
return newItem
return {
title: item.title!,
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use ...item here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 0d10fee — now uses ...item spread with overrides for transformed fields (date, content, description).

src/app.ts Outdated

function feedToRss(data: FeedData): string {
return generateRssFeed({
title: data.title,
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same question, ...data?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 0d10fee — both feedToRss and feedToJson now use ...data and ...item spreads with overrides for fields that need transformation.

…d operators

- Remove skipLibCheck: true from tsconfig.json, upgrade @types/node to ^22
- Delete src/author.test.ts (tested feedsmith directly, not our code)
- Use ...item spread in getFullTextFeed item mapping
- Use ...data and ...item spreads in feedToRss/feedToJson functions

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
@whtsky
Copy link
Copy Markdown
Owner

whtsky commented Mar 25, 2026

@copilot let's be clear about the success criteria: we should be able to process:

without any issue, with full content retrieved, and essential fields kept. You can add an agnets.md to document this, and fix this PR

…n JSON output

- Add fetchUrl() with gzip/deflate/brotli decompression (fixes UN News feed)
- Use parser.parseString() instead of parseURL() for decompressed content
- Map Atom author (item.author) and id (item.id) via rss-parser custom fields
- Fix feedToJson to explicitly list properties (feedsmith leaks unknown props in JSON)
- Remove accidentally committed test-feeds.ts
- Add agents.md documenting success criteria and test feeds

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/whtsky/fulltextrssplz/sessions/f5c83600-9b21-4df2-85dd-93c1abadd1d7
Gzip decompression is a transport-level concern, not application logic.
Reverted to using rss-parser's built-in parseURL() instead of custom fetchUrl().

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/whtsky/fulltextrssplz/sessions/f5c83600-9b21-4df2-85dd-93c1abadd1d7
rss-parser already exposes `author` and `id` at runtime for Atom feeds.
No customFields config needed — just the TypeScript generic type parameter
to declare the properties that are already present.

Co-authored-by: whtsky <876694+whtsky@users.noreply.github.com>
Agent-Logs-Url: https://github.com/whtsky/fulltextrssplz/sessions/f5c83600-9b21-4df2-85dd-93c1abadd1d7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Add the author element to the fulltext feed (when specified on the original feed)

2 participants