Experimental markdown output #1303

jrturton · 2025-09-26T13:35:50Z

Summary

This PR addresses the changes proposed in #1301. It involves a new translator / semantic visitor / markup walker to create the new outputs from DocumentationNode. This is done for each node, after the render JSON is generated.

Dependencies

None

Testing

There are extensive unit tests in MarkdownOutputTests.swift. Testers can also run the convert command against any catalog with the --enable-experimental-markdown-output and --enable-experimental-markdown-output-manifest flags set.

Checklist

Make sure you check off the following items. If they cannot be completed, provide a reason.

Added tests
Ran the ./bin/test script and it succeeded
Updated documentation if necessary

…ert feature flags

…wn output

d-ronnqvist

There's a lot to unpack here and I've only had the time to skim thought the code but these are a few things that stand out to me:

If this is intended as experimental feature, then it can't add actual public API because consumers of SwiftDocC would have no way of knowing what API is stable and what is experimental and could still have breaking changes or be removed.

If a goal of this API is that clients can use it while it's still experimental then it needs to be marked as SPI (using @_spi(SomeName) so that those clients make a deliberate decision to use it and acknowledge that it may have breaking changes.
The added tests follow a pattern not seen elsewhere in the repo. I left some more specific comments about that.

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputManifest.swift

Sources/SwiftDocC/Model/MarkdownOutput/Translation/MarkdownOutputSemanticVisitor.swift

Tests/SwiftDocCTests/Rendering/Markdown/MarkdownOutputTests.swift

Tests/SwiftDocCTests/Test Bundles/MarkdownOutput.docc/MarkdownOutput.symbols.json

…nt unneccessary diffs on re-run

…list rendering

Sources/SwiftDocC/Model/MarkdownOutput/Translation/MarkdownOutputSemanticVisitor.swift

d-ronnqvist · 2025-12-09T16:41:09Z

Sources/SwiftDocC/Model/MarkdownOutput/Translation/MarkdownOutputSemanticVisitor.swift

+    var manifestRelationship: MarkdownOutputManifest.RelationshipSubType? {
+        // Structured like this to cause a compiler error if a new case is added
+        switch self {
+        case .conformingTypes: .conformingTypes
+        case .conformsTo: .conformsTo
+        case .inheritsFrom: .inheritsFrom
+        case .inheritedBy: .inheritedBy
+        }
+    }


Question: what's the goal of this code? Should this enumeration be defined so that new callers need to handle unknown cases?

d-ronnqvist · 2025-12-09T16:43:14Z

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputManifest.swift

+        /// The URI of the document
+        public let uri: String


Hm.. maybe a better question is "what is a caller meant to do with this value"?

d-ronnqvist · 2025-12-09T16:51:21Z

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputManifest.swift

+        }
+    }
+
+    public func children(of parent: Document) -> Set<Document> {


Please note that the added comment doesn't fully answer my question.

What does it mean for a "document" to be the "parent" of another document?

Is this based on the developer defined documentation hierarchy or is it based on the base symbol hierarchy?

What is a caller meant to use this function for?

If the caller is meant to process the document's hierarchically, would it make more sense to have the data also have a hierarchy?

d-ronnqvist · 2025-12-09T16:53:12Z

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputNode.swift

            public let modules: [String]

+            public enum CodingKeys: String, CodingKey {
+                case kindDisplayName = "kind"


Question: why not also use the more descriptive name in the file?

d-ronnqvist · 2025-12-09T17:01:17Z

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputNode.swift

+            public let introduced: String?
+            public let deprecated: String?


I should clarify; I wasn't asking about how this is represented on the page or in the file. I was asking about the type of this property. That's why I asked what a caller is meant to do with the version information and if the caller is meant to inspect or process the information somehow.

d-ronnqvist · 2025-12-09T17:02:02Z

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputNode.swift

+        public var role: String?
+        public let uri: String
+        public var title: String
+        public let framework: String


What do you mean by "location" and what do you mean by "document"? Is a "document" the same thing as a "documentation page"?

Assuming that a symbol is a kind of "document";

if "SomeModule" extends a type in "ExtendedModule" to add a doSomething() function. What's would be the "location" for that document?

if a cross import overlay between "ModuleA" and "ModuleB" defines a doSomething() function. What's would be the "location" for that document?

Alternatively, if that distinction isn't relevant for this property, what is a caller meant to do with this information?

d-ronnqvist · 2025-12-09T17:07:54Z

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputManifest.swift

+        /// One or more protocols to which a type conforms.
+        case conformsTo
+        /// One or more types that conform to a protocol.
+        case conformingTypes
+        /// One or more types that are parents of the symbol.
+        case inheritsFrom
+        /// One or more types that are children of the symbol.
+        case inheritedBy


This appears to be a subset of the cases in Relationship.

Is there a reason for not including the other cases and is there a reason for not using the already existing type?

d-ronnqvist · 2025-12-09T17:21:05Z

Sources/SwiftDocC/SwiftDocCMarkdownOutput/MarkdownOutputNode.swift

-            return data
-        }
+    /// Data for this node to be rendered to disk as a markdown file. This method renders the metadata as a JSON header wrapped in an HTML comment block, then includes the document content.
+    public func generateDataRepresentation() throws -> Data {


Question: is the client meant to do anything else with this raw data or is it only "be rendered to disk as a markdown file"?

Alternatively, is the client meant to write these files at all? I was sort of under the impression that it was DocC that created these files and the the client would only read them. If the client doesn't create and write its own files, does this need to be SPI or could it be package or internal access?

Sources/SwiftDocCUtilities/Action/Actions/Convert/JSONEncodingRenderNodeWriter.swift

Sources/SwiftDocC/Model/Rendering/RenderNode/RenderMetadata.swift

Sources/SwiftDocC/Model/MarkdownOutput/Translation/MarkdownOutputMarkdownWalker.swift

d-ronnqvist · 2025-12-15T17:29:43Z

Sources/SwiftDocC/Model/MarkdownOutput/Translation/MarkdownOutputMarkdownWalker.swift

+import Markdown
+
+/// Performs any markup processing necessary to build the final output markdown
+internal struct MarkdownOutputMarkupWalker: MarkupWalker {


FYI (not blocking): it's redundant to specify internal here because it's the default access level.

d-ronnqvist · 2025-12-15T18:32:39Z

Sources/SwiftDocCUtilities/Action/Actions/Convert/ConvertFileWritingConsumer.swift

+    }
+
+    func consume(markdownManifest: MarkdownOutputManifest) throws {
+        let url = targetFolder.appendingPathComponent("\(markdownManifest.title)-markdown-manifest.json", isDirectory: false)


Question: what's the reason for including the title in the manifest file?

Have I misunderstood that archives would only have a single manifest file?

One archive will only have a single manifest file, but if you are processing the output of multiple archives this allows you to distinguish them

Wouldn't the caller already know what archive the manifest belongs to based on what archive the manifest file is inside?

Sources/SwiftDocCUtilities/Action/Actions/Convert/JSONEncodingRenderNodeWriter.swift

Tests/SwiftDocCTests/Rendering/Markdown/MarkdownOutputTests.swift

d-ronnqvist · 2025-12-15T18:38:45Z

Tests/SwiftDocCTests/Rendering/Markdown/MarkdownOutputTests.swift

+                
+                ## Overview
+                
+                Nothing to see here


nit: This is just noise (and just slightly malformed because the rendered page would have two "Overview" sections)

d-ronnqvist

There continues to be lot of API with a higher access level than what's needed and a handful of unused API.

Richard Turton added 23 commits September 2, 2025 16:25

Add experimental markdown output flag and pass it through to the conv…

cf7ebc6

…ert feature flags

Initial export of Markdown from Article

4b1b94a

Initial processing of a type-level symbol

55e7836

Adds symbol declarations and article reference links

53a6196

Output tutorials to markdown

3513a73

Be smarter about removing indentation from within block directives

81d2d5a

Baseline for adding new tests for markdown output

f3fa5ab

Basic test infrastructure for markdown output

5013a09

Adds symbol link tests to markdown output

6798162

Tutoorial code rendering markdown tests

132cd6c

Adding metadata to markdown output

1753cca

Include package source for markdown output test catalog

301d7da

Output metadata updates

9607e3b

Adds default availability for modules to markdown export

5244f0f

Move availability out of symbol and in to general metadata for markdo…

a6a740e

…wn output

Refactor markdown output so the final node type is standalone

d0dbf44

Add generated markdown flag to render node metadata

595831a

Only include unavailable in markdown header if it is true

104f9eb

Initial setup of manifest output, no references

a2aa8f1

Output of manifest / relationships

b5ed559

Manifest output format updates

74b6023

Remove member symbol relationship

198d8e8

More compact availability, deal with metadata availability for symbols

53ba222

jrturton requested review from d-ronnqvist, franklinsch and heckj September 26, 2025 13:35

Merge branch 'main' into 159600318/experimental-markdown-ouput

af890be

jrturton linked an issue Sep 26, 2025 that may be closed by this pull request

LLM-accessible output #1301

Open

d-ronnqvist requested changes Sep 26, 2025

View reviewed changes

Update tests so all inputs are locally defined

4784380

Richard Turton added 9 commits December 3, 2025 14:57

Deal with internal links with anchor tags

4a8b196

Deal with unresolved symbol links

fc6cf5c

Updates to improve public API / clarity of MarkdownOutputNode

778d33a

Improve public API of manifest, sort contents before writing to preve…

e23115a

…nt unneccessary diffs on re-run

Don't use unstructured Data for WritableMarkdownOutputNode

3659560

Reduce public SPI surface around markdown writing

ff960cd

Make relationship subtype a specific type rather than a string

53ae33b

Make it clear that path component is a fallback title, harden linked …

c0d356c

…list rendering

Add todos and missing tests

a85e59f

jrturton requested a review from d-ronnqvist December 9, 2025 10:16

Merge branch 'main' into 159600318/experimental-markdown-ouput

21c1165

d-ronnqvist reviewed Dec 9, 2025

View reviewed changes

Richard Turton added 4 commits December 12, 2025 12:12

Move markdown output types from public SPI to package

293246a

Merge branch 'main' into 159600318/experimental-markdown-ouput

737be98

don't include scheme or host in documentation links

33fa249

Reformat multi-condition if statements

9481ccf

d-ronnqvist reviewed Dec 15, 2025

View reviewed changes

Richard Turton and others added 10 commits December 16, 2025 10:08

remove whitespace-only change

0df68dd

Remove unused declaration

172674c

Remove redundant return types

8a91913

Remove MarkdownOutputNodeTranslator

f9331f5

Add TODO for alternate declarations

c9f9131

Add TODO for structural review

5a4cedf

Add stacks todo, set writable type back to package

ae69449

uri -> identifier

0a4c251

Use RelationshipsGroup.Kind instead of creating a mirror type

86e40fa

Merge branch 'main' into 159600318/experimental-markdown-ouput

fcb3734

jrturton requested a review from d-ronnqvist December 17, 2025 12:11

d-ronnqvist reviewed Dec 17, 2025

View reviewed changes

Lower access level of manifest, remove children method

78361eb

		public let introduced: String?
		public let deprecated: String?

Experimental markdown output #1303

Are you sure you want to change the base?

Experimental markdown output #1303

Uh oh!

Conversation

jrturton commented Sep 26, 2025

Summary

Dependencies

Testing

Checklist

Uh oh!

d-ronnqvist left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

d-ronnqvist left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

d-ronnqvist left a comment •

edited

Loading