Skip to content

fix: handle escaped pipes (\|) in table cells#12

Open
maou-shonen wants to merge 1 commit intofranlol:mainfrom
maou-shonen:fix/escaped-pipe-handling
Open

fix: handle escaped pipes (\|) in table cells#12
maou-shonen wants to merge 1 commit intofranlol:mainfrom
maou-shonen:fix/escaped-pipe-handling

Conversation

@maou-shonen
Copy link

@maou-shonen maou-shonen commented Mar 2, 2026

Problem

Tables with escaped pipes (\|) in cell content don't get formatted. The formatter treats \| as a column separator, which inflates the column count for that row. Since the separator row (| --- | --- |) has fewer columns, isValidTable() sees a mismatch and rejects the whole table.

For example, this 3-column table:

| Type | Value       | Desc |
| ---- | ----------- | ---- |
| enum | A \| B \| C | ...  |

The data row gets parsed as 5 columns (splitting on every | including the escaped ones), but the header and separator have 3 — so the formatter skips it and appends <!-- table not formatted: invalid structure -->.

Cause

All 4 call sites use .split("|") which doesn't distinguish between real delimiters and escaped pipes:

  • isTableRow() — column count check
  • isSeparatorRow() — cell extraction
  • isValidTable() — row parsing
  • formatTable() — row parsing

Fix

Added a splitByUnescapedPipe() helper that uses a negative lookbehind regex to split on | only when it's not preceded by \. Replaced all .split("|") calls with it.

Test script

You can verify with the test below — run it against the repo before and after the fix:

// bug-test.ts — run with: bun run bug-test.ts
import { FormatTables } from "./index.ts"

const hooks = await FormatTables()
const hook = hooks["experimental.text.complete"] as (
  input: { sessionID: string; messageID: string; partID: string },
  output: { text: string },
) => Promise<void>

const dummyInput = { sessionID: "test", messageID: "test", partID: "test" }

async function format(markdown: string): Promise<string> {
  const output = { text: markdown }
  await hook(dummyInput, output)
  return output.text
}

let passed = 0
let failed = 0

function assert(name: string, actual: string, notContains: string, mustContain?: string) {
  const hasBad = actual.includes(notContains)
  const hasGood = mustContain ? actual.includes(mustContain) : true
  if (!hasBad && hasGood) {
    console.log(`  ✓ ${name}`)
    passed++
  } else {
    console.log(`  ✗ ${name}`)
    if (hasBad) console.log(`    unexpected: found "${notContains}" in output`)
    if (mustContain && !hasGood) console.log(`    missing: expected "${mustContain}" in output`)
    console.log(`    output:\n${actual.split("\n").map(l => "      " + l).join("\n")}`)
    failed++
  }
}

console.log("Escaped pipe tests (against actual index.ts)\n")

const out1 = await format(
  `| Type | A \\| B \\| C | Desc |\n| --- | --- | --- |\n| foo | bar | baz |`,
)
assert("escaped pipes in header cell", out1, "not formatted")

const out2 = await format(
  `| Type | Value | Desc |\n| --- | --- | --- |\n| A \\| B | bar | baz |\n| foo | X \\| Y | qux |`,
)
assert("escaped pipes in data rows", out2, "not formatted")

const out3 = await format(
  `| Col1 | Col2 |\n| --- | --- |\n| A \\| B | normal |`,
)
assert("single escaped pipe in one cell", out3, "not formatted")

const out4 = await format(
  `| Col1 | Col2 |\n| --- | --- |\n| A \\| B | normal |`,
)
assert("escaped pipe preserved in output", out4, "not formatted", "A \\| B")

const out5 = await format(
  `| Type | Value | Desc |\n| --- | --- | --- |\n| foo | bar | baz |`,
)
assert("normal table without escaped pipes", out5, "not formatted")

const out6 = await format(
  `| Left | Center | Right |\n| :--- | :---: | ---: |\n| A \\| B | C \\| D | E \\| F |`,
)
assert("alignment works with escaped pipes", out6, "not formatted")

console.log(`\nResult: ${passed} passed, ${failed} failed`)
process.exit(failed > 0 ? 1 : 0)

Before fix: 1 passed, 5 failed. After fix: 6 passed, 0 failed.

@maou-shonen maou-shonen force-pushed the fix/escaped-pipe-handling branch from 0e2c970 to a91bca8 Compare March 2, 2026 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant