Capture Mongo view definitions in repository

[tim-rohrer]

Problem

The API depends on 10 MongoDB views currently in Atlas. None of these views have their aggregation pipelines defined in the repository.

Current State Analysis

Views that exist in Atlas (10):

1. combined-meetings
2. scheduled-meetings
3. unscheduled-meetings
4. unique-languages-view
5. unique-languages-scheduled
6. unique-languages-unscheduled
7. unique-types-view
8. unique-types-scheduled
9. unique-types-unscheduled
10. group-view — Used by the /api/v1/meetings/:slug/related-group-info endpoint

These views exist only in MongoDB Atlas, defined via Compass on a local machine. There is no documented path to recreating them from source control.

Current Impact

• Single point of failure: View definitions live only in Atlas and on one local machine
• No disaster recovery path: If Atlas project is deleted or Compass definitions are lost, views cannot be recreated
• No onboarding path: New contributors cannot spin up a working dev environment without manual intervention
• No change tracking: Relationship between source schema and view contract must be rediscovered by reading application code
• README contains corrupted pipeline: The one pipeline example in README is for a name the code doesn't use and contains corrupted JSON (Slack URL embedded mid-pipeline)

At Risk

The API does not work without these views. If the view definitions are lost, the system must be reverse-engineered from application code to restore functionality.

Note: The events resource and events-view are excluded from this issue as that code will be removed in a future version.

Proposed Solution

Implement the script approach: create a migrate/views/ directory with executable migration scripts that recreate all views from scratch.

Implementation Steps

1. Export view definitions from MongoDB

Export all 10 views from Atlas

For each view, export the aggregation pipeline:

// In Compass or mongo shell - get full view definition including viewOn
db.getCollectionInfos({ name: "scheduled-meetings" })

// This returns an object like:
// {
//   name: "scheduled-meetings",
//   type: "view",
//   options: {
//     viewOn: "meeting",  // ← This is the source collection
//     pipeline: [ ... ]   // ← This is what you need
//   }
// }

For each of the 10 views, save both the viewOn value and the pipeline array. You'll need both for the migration script.

Save each pipeline as a JSON file in migrate/views/:

• scheduled-meetings.json
• unscheduled-meetings.json
• combined-meetings.json
• unique-languages-view.json
• unique-languages-scheduled.json
• unique-languages-unscheduled.json
• unique-types-view.json
• unique-types-scheduled.json
• unique-types-unscheduled.json
• group-view.json

2. Create migration script

Create migrate/views/create-views.ts (or .js) that:

• Connects to MongoDB using the same connection logic as the app
• Reads each JSON file from migrate/views/
• Calls db.createCollection(name, { viewOn: "source-collection", pipeline: [...] }) for each view
• Handles idempotency (drops existing view before creating, or checks if it exists)
• Provides clear success/failure feedback

Important: The viewOn parameter in the script must match the actual source collection/view for each:

• Base meeting views (scheduled-meetings, unscheduled-meetings, combined-meetings) → viewOn: 'meeting'
• Language views (unique-languages-*) → verify what these aggregate from (likely the meeting views)
• Type views (unique-types-*) → verify what these aggregate from (likely the meeting views)
• Group view (group-view) → viewOn: 'group'

You may need to inspect the exported pipelines to determine the correct viewOn values.

Example structure:

import { MongoClient } from 'mongodb'
import { readFileSync } from 'fs'
import { join } from 'path'

const VIEWS = [
  { name: 'scheduled-meetings', viewOn: 'meetings', file: 'scheduled-meetings.json' },
  { name: 'unscheduled-meetings', viewOn: 'meetings', file: 'unscheduled-meetings.json' },
  { name: 'combined-meetings', viewOn: 'meetings', file: 'combined-meetings.json' },
  { name: 'unique-languages-view', viewOn: 'meetings', file: 'unique-languages-view.json' },
  { name: 'unique-languages-scheduled', viewOn: 'scheduled-meetings', file: 'unique-languages-scheduled.json' },
  { name: 'unique-languages-unscheduled', viewOn: 'unscheduled-meetings', file: 'unique-languages-unscheduled.json' },
  { name: 'unique-types-view', viewOn: 'meetings', file: 'unique-types-view.json' },
  { name: 'unique-types-scheduled', viewOn: 'scheduled-meetings', file: 'unique-types-scheduled.json' },
  { name: 'unique-types-unscheduled', viewOn: 'unscheduled-meetings', file: 'unique-types-unscheduled.json' },
  { name: 'group-view', viewOn: 'group', file: 'group-view.json' },
]

async function createViews() {
  const client = new MongoClient(process.env.MONGO_URI!)
  await client.connect()
  const db = client.db(process.env.MONGO_DB_NAME!)

  for (const view of VIEWS) {
    try {
      // Drop existing view if it exists
      await db.collection(view.name).drop().catch(() => {})
      
      // Read pipeline from file
      const pipeline = JSON.parse(
        readFileSync(join(__dirname, view.file), 'utf-8')
      )
      
      // Create view
      await db.createCollection(view.name, {
        viewOn: view.viewOn,
        pipeline
      })
      
      console.log(`✓ Created view: ${view.name}`)
    } catch (err) {
      console.error(`✗ Failed to create view ${view.name}:`, err)
      throw err
    }
  }

  await client.close()
}

createViews()
  .then(() => console.log('\nAll views created successfully'))
  .catch(err => {
    console.error('\nView creation failed:', err)
    process.exit(1)
  })

3. Add npm script

In package.json:

{
  "scripts": {
    "migrate:views": "tsx migrate/views/create-views.ts"
  }
}

4. Update documentation

In README.md:

• Add section "Database Setup" explaining how to run npm run migrate:views
• Document required environment variables (MONGO_URI, MONGO_DB_NAME)
• Note that the existing pipeline block is for reference only; migrate/views/ is source of truth
• Add instructions for exporting views after making changes in Compass

Create migrate/views/README.md:

• Document what each view does and its source (viewOn) collection/view
• List all 10 views with their purposes:
  • scheduled-meetings, unscheduled-meetings, combined-meetings - filtered views of meetings
  • unique-languages-*, unique-types-* - faceted aggregations for filters
  • group-view - group information lookup
• Explain the relationship between views and source collections
• Provide instructions for modifying and re-exporting views
• Document the migration script usage

5. Test the script

1. Create a fresh Atlas database (or local MongoDB instance)
2. Load source data (meetings, groups, events collections)
3. Run npm run migrate:views
4. Verify all 12 views are created successfully
5. Verify API works against the newly created views
6. Test idempotency: run script again, should succeed without errors

Acceptance Criteria

• All 10 view definitions exported as JSON files in migrate/views/
• Migration script successfully creates all views from JSON definitions
• Script is idempotent (can run multiple times without errors)
• Script provides clear success/failure feedback for each view
• npm run migrate:views command documented and working
• README updated with database setup instructions
• migrate/views/README.md created with view documentation
• Tested against fresh database: can recreate entire view layer from scratch
• Each view's purpose and source collection documented
• Instructions for modifying and re-exporting views included

Success Indicators

A new contributor with Atlas access can:

1. Clone the repository
2. Run npm run migrate:views
3. Have a working set of views against a fresh database
4. Start the API successfully without manual view configuration

Tim's laptop is no longer the only place the view definitions live.

Notes

This is the single-keystone risk in the system. The API depends entirely on these views, and they exist nowhere in source control. If the Atlas project is deleted, if Compass definitions are lost, or if you step away from the project, there is no recovery path.

This is also the easiest risk to fix.

What This Does NOT Include

• Migrating to a different data store (the views work and are the right tool)
• Rewriting views as application code (views are appropriate for this use case)
• Versioning views as proper migrations (worth considering later, not now)
• Automated testing of view output (separate concern, can be added later)

Estimated Effort

4-5 hours including:

• Exporting all 10 view definitions
• Creating the migration script
• Testing against a fresh database
• Writing documentation