diff --git a/.gitignore b/.gitignore index 8b0291459..963710b67 100644 --- a/.gitignore +++ b/.gitignore @@ -7,5 +7,6 @@ resources # Local Netlify folder .netlify # Generated content -/src/pages/awesome-swarm.mdx +/src/pages/awesome-swarm.mdx +/static/openapi.yaml test \ No newline at end of file diff --git a/README.md b/README.md index 484a6f956..98d82f7a5 100644 --- a/README.md +++ b/README.md @@ -61,6 +61,29 @@ npm run swizzle docusaurus-lunr-search SearchBar -- --eject --danger See the documentation for the above command and the plugin at its github repo [here](https://github.com/praveenn77/docusaurus-lunr-search). +## LLM-Friendly Documentation (`llms.txt`) + +The site serves two files for AI agents at the root: + +- **`/llms.txt`** — Hand-crafted index file (`static/llms.txt`). A curated, categorised list of every documentation page with one-line descriptions. This is the entry point AI agents use to find relevant pages. +- **`/llms-full.txt`** — Auto-generated by `docusaurus-plugin-llms` at build time. Contains the full text of every page concatenated into a single file. + +### Keeping `llms.txt` up to date + +The validation script `scripts/validate-llms-txt.mjs` runs automatically during `npm run build` (via the `prebuild` hook). It cross-checks `static/llms.txt` against the actual doc files and prints warnings for: + +- **Stale links** — a URL in `llms.txt` points to a doc page that no longer exists (renamed/deleted). +- **Missing coverage** — a doc file exists that isn't listed in `llms.txt` (new page added without updating the index). + +The script is **informational only** (exit 0) — it won't block the build. + +### What to do when warnings appear + +1. **Stale link**: Open `static/llms.txt`, find the flagged URL, and either update the path to match the renamed page or remove the entry if the page was deleted. +2. **Missing coverage**: Decide which section the new page belongs in and add a line in `static/llms.txt` following the existing format: `- [Title](https://docs.ethswarm.org/docs/path): One-line description`. If the page is a landing/index page with no unique content, it's fine to leave it out — the warning is expected. + +A few pages are intentionally excluded (intro/landing pages that only contain navigation cards). Their warnings are expected and can be ignored. + ## Bumping Version Don't forget to find and replace the version number for the whole of the docs folder. diff --git a/docs/bee/working-with-bee/bee-api.md b/docs/bee/working-with-bee/bee-api.md index bd1d9de47..93365d65e 100644 --- a/docs/bee/working-with-bee/bee-api.md +++ b/docs/bee/working-with-bee/bee-api.md @@ -315,7 +315,7 @@ From the results we can see that our node's neighborhood size and batch commitme * `"lastWonRound"` - The last round number in which your node won the redistribution game. * `"lastPlayedRound"` - The last round number in which your node participating in the redistribution game. If this number matches the number of the current round shown in `"round"`, then your node is participating in the current round. * `"lastFrozenRound"` - The last round in which your node was frozen. - * `"lastSelectedRound"` - The last round in which your node's neighborhood was selected. Note that it is possible for your node's neighborhood to be selected without your node playing in the redistribution game. This may potentially indicate your node's hardware is not sufficient to calculate the commitment hash fast enough. See [section on the `/rchash` endpoint](#) for more information. + * `"lastSelectedRound"` - The last round in which your node's neighborhood was selected. Note that it is possible for your node's neighborhood to be selected without your node playing in the redistribution game. This may potentially indicate your node's hardware is not sufficient to calculate the commitment hash fast enough. See [section on the `/rchash` endpoint](#rchash) for more information. * `"lastSampleDuration"` - The time it took for your node to calculate the sample commitment hash in nanoseconds. * `"block"` - current Gnosis block number * `"reward"` - The total all-time reward in PLUR earned by your node. @@ -472,163 +472,68 @@ From the results we can see that our node's neighborhood size and batch commitme ### _/rchash_ -Calling the `/rchash` endpoint triggers the generation of a reserve commitment hash, which is used in the [redistribution game](/docs/concepts/incentives/redistribution-game), and will report the amount of time it took to generate the hash. This is useful for getting a performance benchmark to ensure that your node's hardware is sufficient. +Calling the `/rchash` endpoint triggers the generation of a reserve commitment hash, +which is used in the [redistribution game](/docs/concepts/incentives/redistribution-game), +and will report the amount of time it took to generate the hash. This is useful for +getting a performance benchmark to ensure that your node's hardware is sufficient. -The `/rchash` endpoint has 3 parameters: `depth` and `anchor_01` and `anchor_02`. For both of the anchor parameters, you should use the first 4 digits from your node's overlay address (which you can find from the `/addresses` endpoint). For depth, you should use the current storage depth of your node which you can find using the `/status` endpoint (storage depth is equivalent to the `storageRadius` value returned from `/status`): +The `/rchash` endpoint has 3 parameters: `depth`, `anchor1`, and `anchor2`. +For both anchor parameters, use the first 4 hex digits from your node's overlay +address (which you can find from the `/addresses` endpoint). For depth, use the +current storage depth of your node from the `/status` endpoint (`storageRadius` value): -``` -/rchash/{depth}/{anchor_01}/{anchor_02} +```text +/rchash/{depth}/{anchor1}/{anchor2} ``` :::info anchor parameter details -- The anchor parameters must match the prefix bits of the node's overlay address up to at least the current storage depth (with each hex digit equal to 4 bits). +- The anchor parameters must match the prefix bits of the node's overlay address + up to at least the current storage depth (with each hex digit equal to 4 bits). - The anchor parameters also must have an even number of digits. -Therefore you can use the first four digits of your node's overlay address since it will work for depths up to depth 16, which will not be approached unless the depth increases up to depth 17, which is not likely to happen in the near future. If it does increase to depth 17, then the first 6 overlay digits should be used. +Therefore you can use the first four digits of your node's overlay address since +it will work for depths up to depth 16, which will not be approached unless the +depth increases up to depth 17, which is not likely to happen in the near future. +If it does increase to depth 17, then the first 6 overlay digits should be used. ::: ```bash -sudo curl -sX GET http://localhost:1633/rchash/10/1e20/1e20 | jq +curl -sX GET http://localhost:1633/rchash/10/1e20/1e20 | jq ``` -It should not take much longer than 6 minutes at most for results to be returned: +The response includes a `hash`, `proofs` (used by the redistribution smart contract), +and `durationSeconds` which is the benchmark metric. Here is an example of a +successful result: -```bash +```json { "hash": "a1d6e1700dff0c5259029c8a58904251855911eb298b45fab4b0c26d4de0fa5f", - "proofs": { - "proof1": { - "proofSegments": [ - "00001099542c8958b2a3a0f7803ee0c07a34de91dd34d4567bf16b367686a387", - "75b51d98c9346a9cb2120bd3a10c0febb43a1d900ff1e4247a29e46974cd4e02", - "d3e289937aa2e2a49e27c9b525580002cbcab4e4d4a972a97ef8e272e0bafebb", - "2215b68c492114e3b4f78080f924bf9cdb3b7da9d5f4e811631f50d2ac45af14", - "c115c46c632fcefea88def9a241fb37acaf222bbbd74a4435d3cde48b6210436", - "0eb01ebfc9ed27500cd4dfc979272d1f0913cc9f66540d7e8005811109e1cf2d", - "887c22bd8750d34016ac3c66b5ff102dacdd73f6b014e710b51e8022af9a1968" - ], - "proveSegment": "1e09adea90a0935fdcd03b5b8193b95767f2f40308d4ea29323845de1270cdd7", - "proofSegments2": [ - "74656e656666656b74000c4469676974616c20636f707900052e63616c6c000d", - "b5d06699c843bc0cc39abd993553865a01c1366174e9e87539ac491dce0a3b77", - "1513dd5a4a3633ecf370ef6f27af99b0f23d73d4773e504314bbf2c690887999", - "a16bdb02905d76eb17965eca8a32f303f0a6129a125c30767a097f80a9e30ded", - "ec01379e1a2ae10fd4e0ee4e4fc4b0d3f6f9ac55038ced386fc7f4294d4bb103", - "456189653d7fdcd3f1ef6b27d00cb8700cd3abb59a1d60b08b61af1d77c74a13", - "4f5384191e60b983ee26b2d42bdcce3fba7041f4cde80b78d808a893d2dc8e79" - ], - "proveSegment2": "436f6465706167652039353000054f72626f7400064d50432d4843000c536569", - "chunkSpan": 4096, - "proofSegments3": [ - "74656e656666656b74000c4469676974616c20636f707900052e63616c6c000d", - "d99a7bf301af141ce9e1acb909c631975c2f7bc60984bbfb9d4097415d2ea723", - "f77855426ebacca7833d31b559f8ef0f5909849e8604856ef7c2344e536252bd", - "273b259e9fb51852d8bbfe409c648605f60fcdaec7fff4ec20120d6f5898f412", - "7d0e379980bf75f76e9a128eb05fdfb8d9db7da40060eff06629ee48465b3a70", - "1be5155660d125ee125cceaf3ea81313321b9ac36aefb6b7b3b35c4a4c0fd5b2", - "8b1a6990327bfb0738025c57b6f06d7255e49b89a2d5f4bed1aed202816843ae" - ], - "postageProof": { - "signature": "d13f85223e1fe47f3689e31471f322d61b10b25231db5319819c8f68548fdc907acf17a0341ba81114d1667693b1e6f35f37c50b4365474baa03f8b233566fa51b", - "postageId": "9af1b37f38d4af75dbee9ba713b95bcc6d03e1c759ac774ec8bd4864e68d2c03", - "index": "1e0900000e13", - "timeStamp": "17ca57481e839b79" - }, - "socProof": null - }, - "proof2": { - "proofSegments": [ - "0000336f98de7901000b056246b5b105611a56a9e45c452fe65288751f90de2e", - "55f65e720d9d53fc7f6d73c87e9a7fd1ad73343b96bd4fc679fdb0d188fe872c", - "7a753e4aa045cc1ce5834a669755bd43f6344ade1890f0080e5c50c23489f098", - "4081e7d623e62c35a17a1167f7f295a26cbf7705a443ed06dfb8fcc416dfc2e4", - "306564763d673ca314aaa072d79ac4b888b69aad80da3ca387aafd2752afce8e", - "0eb01ebfc9ed27500cd4dfc979272d1f0913cc9f66540d7e8005811109e1cf2d", - "887c22bd8750d34016ac3c66b5ff102dacdd73f6b014e710b51e8022af9a1968" - ], - "proveSegment": "1e1b2adb768e9d05021c514914292acc56a8273a4f2ea196f50add7c3e5bff45", - "proofSegments2": [ - "cf19af651fdc9ae2aab3c240860b11156691c6561e0291633cc00aea719c3db5", - "91afb2735a5f002134c31fbf4f1b0c2b99b47ea1d84da75a3d8da597e1d27034", - "9b9e75f236b13ae250a9cf3ae59655054583166c6f6379871c5f2c9b1fdcdb1d", - "4cad5229b21c2f1ca0e5dcf34e61253709dcef1d11e9346c9daf6d6a7a7d6ea2", - "ca49b26daaaffb849045d55f25f3a05c942f8b5844c71a8fcc2172c1ba5e2f86", - "3bbf39b295aa2a01a8b6de929d85cb3fc375bddeaf70ac0ad9ff17f64bca3892", - "3a8343e42db633b929473ccbd90e36320cf0bb6a19474c8a3a01dbbc9ebc99ad" - ], - "proveSegment2": "4cb106406558d74c99ed246e22f73f55abd96f633c1e00025511b98930faafb4", - "chunkSpan": 4096, - "proofSegments3": [ - "cf19af651fdc9ae2aab3c240860b11156691c6561e0291633cc00aea719c3db5", - "8d4665ad0c7ba6a24fe285613057e6289c3b341887fedba9a4190d236f62f31e", - "5fdf57b41fb119b29275edcb55a00b7a0ea5119e1ad78dc97ea615236f9112cb", - "32bc2d3b400b6837117f7720af512913d85388cafe365e06f96172531a59622f", - "d393a441a2dcabfe6ae7a5da201badcd1da877f33092e1f823b816fa826879ac", - "557d9f4ca3655f6bc06f4d13b87da1ddc5c2f7b6eb60b043a7a5a989bb4b2b5f", - "f81cf8329547445e4427e91f8682ef05e482c7b8219adc34066bd9942aec6d05" - ], - "postageProof": { - "signature": "efae95247cae78db875194de1851a6866456f9cb881079676289bd3b72f27af53d783441bc1992f5439e1815179afb05b1a12c7a86b36b41fd411fa29529f1981c", - "postageId": "922b7387276adeea51df5aeb80f597a62c7c236b4387f7f06ced2883762f8fef", - "index": "1e1b0000006a", - "timeStamp": "17ad450de2b34084" - }, - "socProof": null - }, - "proofLast": { - "proofSegments": [ - "00003dbfed587dc866be862e65ebf4da79b54397c6dee4cefa2754e8e0ad2f37", - "0b6fa766b60f03b9efec530413243318f277adccdd974d316cf7b52b5f072681", - "1629dad7244d3efcfb5bddcc794cbd42eb5d6e81ecd47711a7ef08ea9c80c405", - "4081e7d623e62c35a17a1167f7f295a26cbf7705a443ed06dfb8fcc416dfc2e4", - "306564763d673ca314aaa072d79ac4b888b69aad80da3ca387aafd2752afce8e", - "0eb01ebfc9ed27500cd4dfc979272d1f0913cc9f66540d7e8005811109e1cf2d", - "887c22bd8750d34016ac3c66b5ff102dacdd73f6b014e710b51e8022af9a1968" - ], - "proveSegment": "1e2fbcf6d9a79786ce93970536eb4f18ec148cefd2f5e7e15a203f99ad2b1d0a", - "proofSegments2": [ - "1a00fb8c44cf0f65d12d2aed8fcde9517684dd7afc6e3a488cc5a882a92e8cbf", - "5936e75eea7818850651a3e50fab30826766f9777c9099b045c1e6454355111c", - "0ed1fbde9703139dcf9aa8c008c64ac8a85fea8fcfb237986c4652844b49c318", - "f1904237d00d889455695fe711b3f4c7a6e0c40b5e85ae66eefcbf34631b9802", - "19a4dcef98735175b96c675fb1ffd5c59a2fa1a3515724d3d3c2849fa2b75672", - "1449ca5c3b4a701dde9a4dd38f8db6216a5fe8c72e8701dd20263622b3375c77", - "1160eaff3f42ad8c90f8d95fdee20daa5f1fece84947e265878be9db543c4651" - ], - "proveSegment2": "57b3f11f2fe2ba52d0897d936901db164d04fa5a9684009a17a39aa718b2b1c8", - "chunkSpan": 4096, - "proofSegments3": [ - "1a00fb8c44cf0f65d12d2aed8fcde9517684dd7afc6e3a488cc5a882a92e8cbf", - "5f99524dc31e7bf8ca75011c74f739b30bcb855c3ad7e46701d30dc6901cb563", - "0fc1e41481afc47ae70056c780149a50ab9c48630eb44607b9100dfda0ba9dfe", - "e4d7e158b9a446e8f2bad51acf26fc1874013dc23352471ba945fbec0a97d910", - "dddc32e1f2ac86aa4bb711bb6b812afa5d6ca9405d04816e2dcfefd303eb8294", - "e5384c29f838e85b71bc2eb5cb14d59201e8e1997a9779d193a6a9b327637693", - "6f08720c155d4a53fd94ed403fb4476a4c7e6058bf021df23efdfdca1746027d" - ], - "postageProof": { - "signature": "8039da66b4b6e45d5559d0d060f968ccfa11b052234b5645d90d396a0069b6ff40aecb62e9c45d13a68dda0b1fd68dee1de575e28f72c3766d64e824e7c9f9211b", - "postageId": "9af1b37f38d4af75dbee9ba713b95bcc6d03e1c759ac774ec8bd4864e68d2c03", - "index": "1e2f000038a3", - "timeStamp": "17dd336464f4f0bc" - }, - "socProof": null - } - }, - "durationSeconds": 1191.652640831 + "proofs": { "proof1": { ... }, "proof2": { ... }, "proofLast": { ... } }, + "durationSeconds": 287.52 } ``` -If the `Time` value is much longer than 6 minutes then it likely means that the node's hardware performance is not sufficient. In the example above, it took almost 20 minutes to complete, indicating that the hardware is not sufficient. In such cases, consider upgrading to use faster memory or processor. +The `durationSeconds` value should not exceed roughly 6 minutes (360 seconds). + +:::warning Slow results +If `durationSeconds` is much longer than 360 seconds (for example, 1191 seconds / +~20 minutes), the node will likely fail to submit proofs in time during the +redistribution game, resulting in missed rewards or freezing. SSD speed and RAM +are typically the main bottlenecks, since the sampler does heavy random I/O +across the reserve. Upgrade to a faster SSD first, then consider more RAM or a +faster processor. +::: -If while running the `/rchash` command there is an evictions related error such as the one below, try running the call to the `/rchash` endpoint again. +If while running the `/rchash` command there is an evictions related error such +as the one below, try running the call to the `/rchash` endpoint again. -``` +```text error: "level"="error" "logger"="node/storageincentives" "msg"="make sample" "error"="sampler: failed creating sample: sampler stopped due to ongoing evictions" ``` -While evictions are a normal part of Bee's standard operation, the event of an eviction will interrupt the sampler process. +While evictions are a normal part of Bee's standard operation, the event of an +eviction will interrupt the sampler process. ### _/health_ diff --git a/docusaurus.config.mjs b/docusaurus.config.mjs index 52a8c9576..33e823af7 100644 --- a/docusaurus.config.mjs +++ b/docusaurus.config.mjs @@ -18,6 +18,17 @@ export default { scripts: [{ src: "/matomo.js", async: true }], + headTags: [ + { + tagName: 'link', + attributes: { + rel: 'api-description', + href: '/openapi.yaml', + type: 'application/openapi+yaml', + }, + }, + ], + stylesheets: [ { href: 'https://cdn.jsdelivr.net/npm/katex@0.13.24/dist/katex.min.css', @@ -35,23 +46,21 @@ export default { [ 'docusaurus-plugin-llms', { - // Include core developer documentation - include: [ + generateLLMsTxt: false, + generateLLMsFullTxt: true, + title: 'Ethereum Swarm Documentation', + description: 'Swarm is a decentralised storage and communication system for a sovereign digital society.', + excludeImports: true, + removeDuplicateHeadings: true, + includeOrder: [ + 'docs/concepts/**', + 'docs/bee/installation/**', + 'docs/bee/working-with-bee/**', + 'docs/bee/faq.*', 'docs/develop/**', - 'docs/api/**', - 'docs/learn/technology/**' - ], - // Exclude non-essential content - exclude: [ - 'docs/learn/ecosystem/**', - 'docs/desktop/**' + 'docs/desktop/**', + 'docs/references/**', ], - // Prioritize essential developer content - priority: { - 'docs/develop/getting-started': 'high', - 'docs/api/': 'high', - 'docs/learn/technology/**': 'medium' - } } ], [ diff --git a/package.json b/package.json index 3cbff25de..a7b53fe4b 100644 --- a/package.json +++ b/package.json @@ -5,7 +5,7 @@ "scripts": { "docusaurus": "docusaurus", "start": "docusaurus start", - "prebuild": "node scripts/fetch-awesome-swarm.mjs", + "prebuild": "cp openapi/Swarm.yaml static/openapi.yaml && node scripts/fetch-awesome-swarm.mjs && node scripts/validate-llms-txt.mjs", "build": "docusaurus build", "swizzle": "docusaurus swizzle", "deploy": "docusaurus deploy", diff --git a/scripts/validate-llms-txt.mjs b/scripts/validate-llms-txt.mjs new file mode 100644 index 000000000..87e76b56e --- /dev/null +++ b/scripts/validate-llms-txt.mjs @@ -0,0 +1,105 @@ +// Validate static/llms.txt against the actual docs directory. +// Reports stale links (URLs pointing to non-existent pages) and +// missing coverage (doc pages not listed in llms.txt). +// Exits 0 — warnings only, does not block the build. + +import { readFileSync, readdirSync } from 'node:fs'; +import { join, dirname, basename, extname, relative } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const ROOT = join(__dirname, '..'); +const DOCS_DIR = join(ROOT, 'docs'); +const LLMS_TXT = join(ROOT, 'static', 'llms.txt'); +const BASE_URL = 'https://docs.ethswarm.org/docs/'; + +/** Recursively find all .md/.mdx files under `dir`. */ +function findDocFiles(dir) { + const results = []; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = join(dir, entry.name); + if (entry.isDirectory()) { + results.push(...findDocFiles(full)); + } else if (/\.(md|mdx)$/.test(entry.name)) { + results.push(full); + } + } + return results; +} + +/** Read the frontmatter `id` from a doc file (returns null if absent). */ +function getFrontmatterId(filePath) { + const content = readFileSync(filePath, 'utf-8'); + const fm = content.match(/^---\n([\s\S]*?)\n---/); + if (!fm) return null; + const id = fm[1].match(/^id:\s*(.+)$/m); + return id ? id[1].trim() : null; +} + +/** + * Compute the URL path segment for a doc file. + * e.g. docs/bee/installation/shell-script.md with id "shell-script-install" + * → "bee/installation/shell-script-install" + * + * When a file's basename matches its parent directory (case-insensitive), + * Docusaurus treats it as a category index and serves it at the directory + * URL (with trailing slash). e.g. DISC/DISC.mdx → "concepts/DISC/" + */ +function getUrlPath(filePath) { + const relDir = relative(DOCS_DIR, dirname(filePath)); + const id = getFrontmatterId(filePath) || basename(filePath, extname(filePath)); + const dirName = basename(dirname(filePath)); + const fileName = basename(filePath, extname(filePath)); + + // Category index: file name matches parent directory (case-insensitive) + if (dirName.toLowerCase() === fileName.toLowerCase()) { + return `${relDir}/`; + } + + return relDir ? `${relDir}/${id}` : id; +} + +/** Extract all docs.ethswarm.org/docs/... paths from llms.txt. */ +function extractDocPaths(file) { + const content = readFileSync(file, 'utf-8'); + const escaped = BASE_URL.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); + const re = new RegExp(`${escaped}([^)\\s]+)`, 'g'); + const paths = []; + let m; + while ((m = re.exec(content)) !== null) { + paths.push(m[1]); + } + return paths; +} + +// ── Main ──────────────────────────────────────────────────────────── + +const docFiles = findDocFiles(DOCS_DIR); +const docPathSet = new Set(docFiles.map(getUrlPath)); + +const llmsPaths = extractDocPaths(LLMS_TXT); +const llmsPathSet = new Set(llmsPaths); + +let warnings = 0; + +// 1. Stale links — URL in llms.txt but no matching doc file +for (const p of llmsPaths) { + if (!docPathSet.has(p)) { + console.warn(` ⚠ Stale link in llms.txt: ${BASE_URL}${p}`); + warnings++; + } +} + +// 2. Missing coverage — doc file exists but not in llms.txt +for (const p of docPathSet) { + if (!llmsPathSet.has(p)) { + console.warn(` ⚠ Missing from llms.txt: ${BASE_URL}${p}`); + warnings++; + } +} + +if (warnings > 0) { + console.log(`llms.txt validation: ${warnings} warning(s)`); +} else { + console.log('llms.txt validation: all links verified, full coverage ✓'); +} diff --git a/static/llms.txt b/static/llms.txt new file mode 100644 index 000000000..5f2b397ba --- /dev/null +++ b/static/llms.txt @@ -0,0 +1,109 @@ +# Ethereum Swarm Documentation + +> Swarm is a decentralised storage and communication system built on the Ethereum blockchain, providing censorship-resistant infrastructure for storing data, hosting websites, and building applications. + +This is the documentation for [Swarm](https://www.ethswarm.org/) and its reference node implementation, [Bee](https://github.com/ethersphere/bee). Swarm splits data into 4 KB chunks, distributes them across a Kademlia-based network (DISC), and uses postage stamps (paid in xBZZ on Gnosis Chain) for storage incentives. Bee nodes expose an HTTP API on port 1633 (gateway) and 1635 (debug/admin). For the full unabridged documentation, see [llms-full.txt](https://docs.ethswarm.org/llms-full.txt). + +- [API Reference](https://docs.ethswarm.org/api/): Interactive OpenAPI documentation for all Bee endpoints +- [OpenAPI spec (Swarm.yaml)](https://docs.ethswarm.org/openapi.yaml): Machine-readable API definition +- [bee-js documentation](https://bee-js.ethswarm.org/): JavaScript/TypeScript SDK for Swarm +- [The Book of Swarm](https://www.ethswarm.org/The-Book-of-Swarm.pdf): Comprehensive technical reference +- [Swarm Whitepaper](https://www.ethswarm.org/swarm-whitepaper.pdf): Original design paper + +## Install and run a Bee node + +- [Getting Started](https://docs.ethswarm.org/docs/bee/installation/getting-started): Overview of node requirements and installation options +- [Quickstart](https://docs.ethswarm.org/docs/bee/installation/quick-start): Fastest path to a running Bee node +- [Shell Script Install](https://docs.ethswarm.org/docs/bee/installation/shell-script-install): Automated install script for Linux +- [Docker Install](https://docs.ethswarm.org/docs/bee/installation/docker): Run Bee in a Docker container +- [Package Manager Install](https://docs.ethswarm.org/docs/bee/installation/package-manager-install): Install via apt, brew, or rpm +- [Build from Source](https://docs.ethswarm.org/docs/bee/installation/build-from-source): Compile Bee from the Go source code +- [Set Target Neighborhood](https://docs.ethswarm.org/docs/bee/installation/set-target-neighborhood): Choose a network neighborhood for your node +- [Connectivity](https://docs.ethswarm.org/docs/bee/installation/connectivity): NAT traversal, ports, and peer connectivity +- [Fund Your Node](https://docs.ethswarm.org/docs/bee/installation/fund-your-node): Add xBZZ and xDAI to operate your node + +## Configure and operate a Bee node + +- [Configuration](https://docs.ethswarm.org/docs/bee/working-with-bee/configuration): All Bee config options (YAML, env vars, CLI flags) +- [Node Types](https://docs.ethswarm.org/docs/bee/working-with-bee/node-types): Full, light, and ultra-light node modes +- [Bee API](https://docs.ethswarm.org/docs/bee/working-with-bee/bee-api): Overview of the HTTP API endpoints +- [Logging](https://docs.ethswarm.org/docs/bee/working-with-bee/logs-and-files): Log levels, file locations, and data directories +- [Swarm CLI](https://docs.ethswarm.org/docs/bee/working-with-bee/swarm-cli): Command-line tool for interacting with Bee +- [Staking](https://docs.ethswarm.org/docs/bee/working-with-bee/staking): Stake xBZZ to participate in storage incentives +- [Cashing Out](https://docs.ethswarm.org/docs/bee/working-with-bee/cashing-out): Withdraw earnings from SWAP cheques +- [Monitoring](https://docs.ethswarm.org/docs/bee/working-with-bee/monitoring): Prometheus metrics and Grafana dashboards +- [Backups](https://docs.ethswarm.org/docs/bee/working-with-bee/backups): Back up keys, statestore, and stamps +- [Upgrading Bee](https://docs.ethswarm.org/docs/bee/working-with-bee/upgrading-bee): Upgrade process and migration notes + +## Upload, download, and manage data + +- [Upload & Download](https://docs.ethswarm.org/docs/develop/upload-and-download): Store and retrieve files and directories via the Bee API +- [Manage Files](https://docs.ethswarm.org/docs/develop/files): File operations, collections, and tags +- [Postage Stamp Batches](https://docs.ethswarm.org/docs/develop/tools-and-features/buy-a-stamp-batch): Buy and manage stamps for uploads +- [Store with Encryption](https://docs.ethswarm.org/docs/develop/tools-and-features/store-with-encryption): Client-side encryption for uploaded data +- [Pinning](https://docs.ethswarm.org/docs/develop/tools-and-features/pinning): Pin content locally to prevent garbage collection +- [Erasure Coding (usage)](https://docs.ethswarm.org/docs/develop/tools-and-features/erasure-coding): Add redundancy to uploads for reliability + +## Host websites on Swarm + +- [Host a Webpage](https://docs.ethswarm.org/docs/develop/host-your-website): Upload and serve static websites on Swarm +- [Website Routing](https://docs.ethswarm.org/docs/develop/routing): Custom domains, ENS resolution, and routing configuration +- [Run a Gateway](https://docs.ethswarm.org/docs/develop/gateway-proxy): Serve Swarm content over HTTP to regular browsers + +## Build applications on Swarm + +- [Bee JS](https://docs.ethswarm.org/docs/develop/tools-and-features/bee-js): JavaScript/TypeScript SDK for Swarm +- [Chunk Types](https://docs.ethswarm.org/docs/develop/tools-and-features/chunk-types): Content-addressed chunks (CAC) and single-owner chunks (SOC) +- [Feeds](https://docs.ethswarm.org/docs/develop/tools-and-features/feeds): Mutable data references using single-owner chunks +- [Manifests](https://docs.ethswarm.org/docs/develop/tools-and-features/manifests): Trie-based path mapping for file collections +- [PSS Messaging (usage)](https://docs.ethswarm.org/docs/develop/tools-and-features/pss): Send messages through the Swarm network +- [GSOC](https://docs.ethswarm.org/docs/develop/tools-and-features/gsoc): Graffiti SOC for pub/sub messaging patterns +- [Add Access Control (usage)](https://docs.ethswarm.org/docs/develop/act): Encrypt content for specific recipients using ACT +- [Gateway Proxy](https://docs.ethswarm.org/docs/develop/tools-and-features/gateway-proxy): Proxy for browser-based Swarm access +- [Developer Mode](https://docs.ethswarm.org/docs/develop/tools-and-features/bee-dev-mode): Run Bee without blockchain for local testing +- [Starting a Private Network](https://docs.ethswarm.org/docs/develop/tools-and-features/starting-a-test-network): Multi-node test clusters +- [Dynamic Content](https://docs.ethswarm.org/docs/develop/dynamic-content): Updatable content using feeds +- [Ultra Light Nodes](https://docs.ethswarm.org/docs/develop/ultra-light-nodes): Minimal nodes for lightweight client use cases + +## Understand Swarm concepts + +- [What is Swarm?](https://docs.ethswarm.org/docs/concepts/what-is-swarm): Architecture overview and design goals +- [DISC](https://docs.ethswarm.org/docs/concepts/DISC/): Distributed Immutable Store for Chunks +- [Kademlia](https://docs.ethswarm.org/docs/concepts/DISC/kademlia): Distributed hash table and routing +- [Neighborhoods](https://docs.ethswarm.org/docs/concepts/DISC/neighborhoods): Address space, proximity, and storage responsibility +- [Erasure Coding (theory)](https://docs.ethswarm.org/docs/concepts/DISC/erasure-coding): Reed-Solomon redundancy for chunk reliability +- [Incentives Overview](https://docs.ethswarm.org/docs/concepts/incentives/overview): Economic model for storage and bandwidth +- [Redistribution Game](https://docs.ethswarm.org/docs/concepts/incentives/redistribution-game): Staking and rewards for storage providers +- [Postage Stamps (theory)](https://docs.ethswarm.org/docs/concepts/incentives/postage-stamps): Prepaid storage with batch depth and amount +- [PSS (theory)](https://docs.ethswarm.org/docs/concepts/pss): Postal Service over Swarm messaging layer +- [Access Control (theory)](https://docs.ethswarm.org/docs/concepts/access-control): ACT-based encryption and grantee management + +## Reference + +- [Smart Contracts](https://docs.ethswarm.org/docs/references/smart-contracts): Deployed contract addresses and ABIs +- [Tokens](https://docs.ethswarm.org/docs/references/tokens): BZZ, xBZZ, and bridging between chains +- [Glossary](https://docs.ethswarm.org/docs/references/glossary): Key terms and definitions + +## Optional + +- [Desktop: Introduction](https://docs.ethswarm.org/docs/desktop/introduction): Swarm Desktop GUI app overview +- [Desktop: Install](https://docs.ethswarm.org/docs/desktop/install): Install Swarm Desktop +- [Desktop: Configuration](https://docs.ethswarm.org/docs/desktop/configuration): Configure the Desktop app +- [Desktop: Access Content](https://docs.ethswarm.org/docs/desktop/access-content): Browse Swarm content via Desktop +- [Desktop: Postage Stamps](https://docs.ethswarm.org/docs/desktop/postage-stamps): Manage stamps in the Desktop app +- [Desktop: Upload Content](https://docs.ethswarm.org/docs/desktop/upload-content): Upload files through the Desktop app +- [Desktop: Backup and Restore](https://docs.ethswarm.org/docs/desktop/backup-restore): Back up and restore Desktop data +- [Desktop: Publish a Website](https://docs.ethswarm.org/docs/desktop/publish-a-website): Host a website using Swarm Desktop +- [Desktop: Start a Blog](https://docs.ethswarm.org/docs/desktop/start-a-blog): Create a blog on Swarm +- [Hive](https://docs.ethswarm.org/docs/bee/installation/hive): Run multiple Bee nodes together +- [Uninstalling Bee](https://docs.ethswarm.org/docs/bee/working-with-bee/uninstalling-bee): Remove Bee and clean up its data +- [Bcrypt Utility](https://docs.ethswarm.org/docs/bee/working-with-bee/bcrypt): Password hashing for API authentication +- [Bee FAQ](https://docs.ethswarm.org/docs/bee/bee-faq): Common node operator questions +- [Bandwidth Incentives](https://docs.ethswarm.org/docs/concepts/incentives/bandwidth-incentives): SWAP protocol for bandwidth accounting +- [Price Oracle](https://docs.ethswarm.org/docs/concepts/incentives/price-oracle): On-chain price feed for storage costs +- [Contributing](https://docs.ethswarm.org/docs/develop/contribute/introduction): How to contribute to Swarm +- [Protocols](https://docs.ethswarm.org/docs/develop/contribute/protocols): Swarm protocol specifications +- [Community](https://docs.ethswarm.org/docs/references/community): Official channels and resources +- [Fair Data Society](https://docs.ethswarm.org/docs/references/fair-data-society): Fair data principles and ecosystem +- [General FAQ](https://docs.ethswarm.org/docs/references/faq): Frequently asked questions about Swarm +- [Awesome Swarm](https://docs.ethswarm.org/docs/references/awesome-list): Curated list of Swarm projects and tools