diff --git a/doc/user/content/concepts/sources.md b/doc/user/content/concepts/sources.md index db8344f707c90..36281d1d13b39 100644 --- a/doc/user/content/concepts/sources.md +++ b/doc/user/content/concepts/sources.md @@ -12,32 +12,7 @@ aliases: ## Overview -Sources describe external systems you want Materialize to read data from, and -provide details about how to decode and interpret that data. A simplistic way to -think of this is that sources represent streams and their schemas; this isn't -entirely accurate, but provides an illustrative mental model. - -In terms of SQL, sources are similar to a combination of tables and -clients. - -- Like tables, sources are structured components that users can read from. -- Like clients, sources are responsible for reading data. External - sources provide all of the underlying data to process. - -By looking at what comprises a source, we can develop a sense for how this -combination works. - -[//]: # "TODO(morsapaes) Add details about source persistence." - -## Source components - -Sources consist of the following components: - -Component | Use | Example ----------------|---------------------------------------------------------------------------------------------------|--------- -**Connector** | Provides actual bytes of data to Materialize | Kafka -**Format** | Structures of the external source's bytes, i.e. its schema | Avro -**Envelope** | Expresses how Materialize should handle the incoming data + any additional formatting information | Upsert +{{% include-headless "/headless/source-definition" %}} ### Connectors @@ -45,9 +20,6 @@ Materialize bundles native connectors for the following external systems: {{< include-md file="shared-content/multilink-box-native-connectors.md" >}} -For details on the syntax, supported formats and features of each connector, check out the dedicated `CREATE SOURCE` documentation pages. - - ## Sources and clusters Sources require compute resources in Materialize, and so need to be associated diff --git a/doc/user/content/headless/source-definition.md b/doc/user/content/headless/source-definition.md new file mode 100644 index 0000000000000..da35815925085 --- /dev/null +++ b/doc/user/content/headless/source-definition.md @@ -0,0 +1,14 @@ +--- +headless: true +--- +A source in Materialize represents an external data source. More concretely, it +specifies the connection and the ingestion configuration to use for a particular +external data source (e.g., PostgreSQL, Kafka). For those familiar with +PostgreSQL's foreign servers and foreign tables, a source is like a foreign +server, and the tables (or subsources) created from the source are like foreign +tables. + +Before creating a source in Materialize, you must ensure that the external data +source is properly configured and accessible so that Materialize can establish a +connection and ingest its data. The exact configuration depends on the type of +data source. diff --git a/doc/user/content/sql/create-source/_index.md b/doc/user/content/sql/create-source/_index.md index ba3a03d7d28a4..bacac36c9f074 100644 --- a/doc/user/content/sql/create-source/_index.md +++ b/doc/user/content/sql/create-source/_index.md @@ -8,7 +8,7 @@ menu: identifier: 'create-source' --- -A [source](/concepts/sources/) describes an external system you want Materialize to read data from, and provides details about how to decode and interpret that data. +{{% include-headless "/headless/source-definition" %}} ## Syntax summary