Skip to content

Commit b050485

Browse files
committed
Add blobs fix links
1 parent d6a2586 commit b050485

File tree

9 files changed

+369
-231
lines changed

9 files changed

+369
-231
lines changed

concepts/discovery.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,13 @@ title: "Discovery"
33
---
44

55

6-
Discovery is the glue that connects an [Endpoint](/docs/concepts/endpoint#endpoint-identifiers) to something we can dial. Discovery services resolve EndpointIds to either a home Relay URL or direct-dialing information.
6+
Discovery is the glue that connects an [Endpoint](/concepts/endpoints#endpoint-identifiers) to something we can dial. Discovery services resolve EndpointIds to either a home Relay URL or direct-dialing information.
77

88
<Note>
99
More details on discovery in the discovery module [documentation](https://docs.rs/iroh/latest/iroh/discovery/index.html)
1010
</Note>
1111

12-
Endpoint discovery is an automated system for an [Endpoint](/docs/concepts/endpoint) to retrieve addressing information. Each iroh endpoint will automatically publish their own addressing information with configured discovery services. Usually this means publishing which [Home Relay](/docs/concepts/relay#home-relay) an endpoint is findable at, but they could also publish their direct addresses.
12+
Endpoint discovery is an automated system for an [Endpoint](/concepts/endpoints) to retrieve addressing information. Each iroh endpoint will automatically publish their own addressing information with configured discovery services. Usually this means publishing which [Home Relay](/concepts/relay#home-relay) an endpoint is findable at, but they could also publish their direct addresses.
1313

1414
## Discovery Services
1515

concepts/endpoints.mdx

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,11 @@
22
title: "Endpoints"
33
---
44

5-
An _endpoint_ is the main API interface to create connections to, and accept connections from other iroh endpoints. The connections are peer-to-peer and encrypted, a [Relay](/docs/concepts/relay) server is used to make the connections reliable. Endpoints have a `EndpointID` (the public half of an Ed25519 keypair) and the private key used to sign and decrypt messages.
5+
An _endpoint_ is the main API interface to create connections to, and accept
6+
connections from other iroh endpoints. The connections are peer-to-peer and
7+
encrypted, a [Relay](/concepts/relays) server is used to make the
8+
connections reliable. Endpoints have a `EndpointID` (the public half of an
9+
Ed25519 keypair) and the private key used to sign and decrypt messages.
610

711
Generally, an application will have a single endpoint instance. This ensures all the connections made share the same peer-to-peer connections to other iroh endpoints, while still remaining independent connections. This will result in more optimal network behaviour.
812

@@ -25,4 +29,9 @@ Due to the light-weight properties of QUIC streams a stream can only be accepted
2529

2630
## Building on Endpoints
2731

28-
Endpoints are a low-level primitive that iroh exposes on purpose. For some projects like [dumbpipe](https://dumbpipe.dev), endpoints are 95% of what you need to connect any two devices on the planet. For others, like [iroh-blobs](/proto/iroh-blobs), endpoints are the foundation that higher-level protocols are built on. Most projects will not work with endpoints beyond constructing one and feeding it to one or more [protocols](/docs/concepts/protocol).
32+
Endpoints are a low-level primitive that iroh exposes on purpose. For some
33+
projects like [dumbpipe](https://dumbpipe.dev), endpoints are 95% of what you
34+
need to connect any two devices on the planet. For others, like
35+
[iroh-blobs](/proto/iroh-blobs), endpoints are the foundation that higher-level
36+
protocols are built on. Most projects will not work with endpoints beyond
37+
constructing one and feeding it to one or more [protocols](/concepts/protocols).

deployment/wasm-browser-support.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Add `iroh` to your browser project's dependencies and keep building it using [wa
1111

1212
For end-to-end examples of integrating iroh into a browser page, see these examples on our [iroh-examples repository](https://github.com/n0-computer/iroh-examples):
1313
- An [iroh echo server in the browser](https://github.com/n0-computer/iroh-examples/tree/main/browser-echo). Check it out [in action](https://n0-computer.github.io/iroh-examples/main/browser-echo/index.html)
14-
- An [browser chat room UI](https://github.com/n0-computer/iroh-examples/tree/main/browser-chat) based on the [iroh gossip chat example](/docs/examples/gossip-chat). Check it out [in action](https://n0-computer.github.io/iroh-examples/main/browser-chat/index.html)
14+
- An [browser chat room UI](https://github.com/n0-computer/iroh-examples/tree/main/browser-chat) based on the [iroh gossip chat example](/examples). Check it out [in action](https://n0-computer.github.io/iroh-examples/main/browser-chat/index.html)
1515

1616

1717
## Limitations

docs.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
"pages": [
1919
"what-is-iroh",
2020
"quickstart",
21+
"examples",
2122
{
2223
"group": "Concepts",
2324
"expanded": true,

protocols/blobs.mdx

Lines changed: 257 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -22,51 +22,278 @@ cargo add iroh-blobs
2222
- [Primary docs and API reference](https://docs.rs/iroh-blobs/latest/iroh_blobs/)
2323
- [Code samples on GitHub](https://github.com/n0-computer/iroh-blobs/tree/main/examples)
2424

25-
## Examples
25+
## Tutorial
2626

27-
The crate is typically used by creating a store, a `BlobsProtocol` handler,
28-
and registering it with an `iroh::Router`. The code below is an illustrative
29-
outline — see the upstream examples for full runnable code.
27+
Let's dive into iroh by building a simple peer-to-peer file transfer tool in rust!
28+
29+
At the end we should be able to transfer a file from one device by running this:
30+
31+
```sh
32+
$ cargo run -- send ./file.txt
33+
Indexing file.
34+
File analyzed. Fetch this file by running:
35+
cargo run -- receive blobabvojvy[...] file.txt
36+
```
37+
38+
And then fetch it on any other device like so:
39+
```sh
40+
$ cargo run -- receive blobabvojvy[...] file.txt
41+
Starting download.
42+
Finished download.
43+
Copying to destination.
44+
Finished copying.
45+
Shutting down.
46+
```
47+
48+
In this guide we'll be omitting the import statements required to get this working.
49+
If you're ever confused about what to import, take a look at the imports in the [complete example](https://github.com/n0-computer/iroh-blobs/blob/main/examples/transfer.rs).
50+
51+
52+
### Get set up
53+
54+
We'll assume you've set up [rust](https://www.rust-lang.org/) and [cargo](https://doc.rust-lang.org/cargo/) on your machine.
55+
56+
Initialize a new project by running:
57+
58+
```bash
59+
cargo init file-transfer
60+
cd file-transfer
61+
```
62+
63+
Now, add the dependencies we'll need for this tutorial:
64+
65+
```bash
66+
cargo add iroh iroh-blobs tokio anyhow
67+
```
68+
69+
From here on we'll be working inside the `src/main.rs` file.
70+
71+
72+
### Create an `iroh::Endpoint`
73+
74+
To start interacting with other iroh endpoints, we need to build an `iroh::Endpoint`.
75+
This is what manages the possibly changing network underneath, maintains a connection to the closest relay, and finds ways to address devices by `EndpointId`.
3076

3177
```rust
32-
use iroh::{protocol::Router, Endpoint};
33-
use iroh_blobs::{store::mem::MemStore, BlobsProtocol, ticket::BlobTicket};
78+
#[tokio::main]
79+
async fn main() -> anyhow::Result<()> {
80+
// Create an endpoint, it allows creating and accepting
81+
// connections in the iroh p2p world
82+
let endpoint = Endpoint::bind().await?;
83+
84+
// ...
3485

86+
Ok(())
87+
}
88+
```
89+
90+
There we go, this is all we need to [open connections](https://docs.rs/iroh/latest/iroh/endpoint/struct.Endpoint.html#method.connect) or [accept them](https://docs.rs/iroh/latest/iroh/endpoint/struct.Endpoint.html#method.accept).
91+
92+
### Using an existing protocol: iroh-blobs
93+
94+
Instead of writing our own protocol from scratch, let's use [iroh-blobs](/proto/iroh-blobs), which already does what we want:
95+
It loads files from your file system and provides a protocol for seekable, resumable downloads of these files.
96+
97+
```rust
3598
#[tokio::main]
3699
async fn main() -> anyhow::Result<()> {
37-
// create an iroh endpoint that includes the standard discovery mechanisms
38-
// we've built at number0
100+
// Create an endpoint, it allows creating and accepting
101+
// connections in the iroh p2p world
39102
let endpoint = Endpoint::bind().await?;
40103

41-
// create a protocol handler using an in-memory blob store.
104+
// We initialize an in-memory backing store for iroh-blobs
42105
let store = MemStore::new();
43-
let tag = store.add_slice(b"Hello world").await?;
44-
45-
let _ = endpoint.online().await;
46-
let addr = endpoint.addr();
47-
let ticket = BlobTicket::new(addr, tag.hash, tag.format);
48-
49-
// build the router
106+
// Then we initialize a struct that can accept blobs requests over iroh connections
50107
let blobs = BlobsProtocol::new(&store, None);
51-
let router = Router::builder(endpoint)
52-
.accept(iroh_blobs::ALPN, blobs)
53-
.spawn();
54-
55-
println!("We are now serving {}", ticket);
56108

57-
// wait for control-c
58-
tokio::signal::ctrl_c().await;
109+
// ...
59110

60-
// clean shutdown of router and store
61-
router.shutdown().await?;
62111
Ok(())
63112
}
64113
```
65114

66-
## Notes & best practices
115+
<Note>
116+
Learn more about what we mean by "protocol" on the [protocol documentation page](/concepts/protocols).
117+
</Note>
118+
119+
With these two lines, we've initialized iroh-blobs and gave it access to our `Endpoint`.
120+
121+
At this point what we want to do depends on whether we want to accept incoming iroh connections from the network or create outbound iroh connections to other endpoints.
122+
Which one we want to do depends on if the executable was called with `send` as an argument or `receive`, so let's parse these two options out from the CLI arguments and match on them:
123+
124+
```rust
125+
// Grab all passed in arguments, the first one is the binary itself, so we skip it.
126+
let args: Vec<String> = std::env::args().skip(1).collect();
127+
// Convert to &str, so we can pattern-match easily:
128+
let arg_refs: Vec<&str> = args.iter().map(String::as_str).collect();
129+
130+
match arg_refs.as_slice() {
131+
["send", filename] => {
132+
todo!();
133+
}
134+
["receive", ticket, filename] => {
135+
todo!();
136+
}
137+
_ => {
138+
println!("Couldn't parse command line arguments: {args:?}");
139+
println!("Usage:");
140+
println!(" # to send:");
141+
println!(" cargo run -- send [FILE]");
142+
println!(" # this will print a ticket.");
143+
println!();
144+
println!(" # to receive:");
145+
println!(" cargo run -- receive [TICKET] [FILE]");
146+
}
147+
}
148+
```
149+
150+
We're also going to print some simple help text when there's no arguments or we can't parse them.
151+
152+
What's left to do now is fill in the two `todo!()`s!
153+
154+
155+
### Getting ready to send
156+
157+
If we want to make a file available over the network with iroh-blobs, we first need to hash this file.
158+
159+
<Note>
160+
What does this step do?
161+
162+
It hashes the file using [BLAKE3](https://en.wikipedia.org/wiki/BLAKE_(hash_function)) and remembers a so-called ["outboard"](https://github.com/oconnor663/bao?tab=readme-ov-file#outboard-mode) for that file.
163+
This outboard contains information about hashes of parts of this file.
164+
All of this enables some extra features with iroh-blobs like automatically verifying the integrity of the file *while it's streaming*, verified range downloads and download resumption.
165+
</Note>
166+
167+
```rust
168+
let filename: PathBuf = filename.parse()?;
169+
let abs_path = std::path::absolute(&filename)?;
170+
171+
println!("Hashing file.");
172+
173+
// When we import a blob, we get back a "tag" that refers to said blob in the store
174+
// and allows us to control when/if it gets garbage-collected
175+
let tag = store.blobs().add_path(abs_path).await?;
176+
```
177+
178+
<Note>
179+
For other use cases, there are other ways of importing blobs into iroh-blobs, you're not restricted to pulling them from the file system!
180+
You can see other options available, such as [`add_slice`](https://docs.rs/iroh-blobs/latest/iroh_blobs/api/blobs/struct.Blobs.html#method.add_slice).
181+
Make sure to also check out the options you can pass and their documentation for some interesting tidbits on performance.
182+
</Note>
183+
184+
The return value `tag` contains the final piece of information such that another endpoint can fetch a blob from us.
185+
186+
We'll use a `BlobTicket` to put the file's BLAKE3 hash and our endpoint's `EndpointId` into a single copy-able string:
187+
188+
```rust
189+
let endpoint_id = endpoint.id();
190+
let ticket = BlobTicket::new(endpoint_id.into(), tag.hash, tag.format);
191+
192+
println!("File hashed. Fetch this file by running:");
193+
println!(
194+
"cargo run -- receive {ticket} {}",
195+
filename.display()
196+
);
197+
```
198+
199+
Now we've imported the file and produced instructions for how to fetch it, but we're actually not yet actively listening for incoming connections yet! (iroh-blobs won't do so unless you specifically tell it to do that.)
200+
201+
For that we'll use iroh's `Router`.
202+
Similar to routers in webserver libraries, it runs a loop accepting incoming connections and routes them to the specific handler.
203+
However, instead of handlers being organized by HTTP paths, it routes based on "ALPNs".
204+
Read more about ALPNs and the router on the [protocol](/concepts/protocols) page.
205+
206+
In our case, we only need a single protocol, but constructing a router also takes care of running the accept loop, so that makes our life easier:
207+
208+
```rs
209+
// For sending files we build a router that accepts blobs connections & routes them
210+
// to the blobs protocol.
211+
let router = Router::builder(endpoint)
212+
.accept(iroh_blobs::ALPN, blobs)
213+
.spawn();
214+
215+
tokio::signal::ctrl_c().await?;
216+
217+
// Gracefully shut down the endpoint
218+
println!("Shutting down.");
219+
router.shutdown().await?;
220+
```
221+
222+
And as you can see, as a final step we wait for the user to stop the file providing side by hitting `Ctrl+C` in the console and once they do so, we shut down the router gracefully.
223+
224+
225+
### Connecting to the other side to receive
226+
227+
On the connection side, we got the `ticket` and the `path` from the CLI arguments and we can parse them into their `struct` versions.
228+
229+
With them parsed
230+
- we first construct a `Downloader` (that can help us coordinate multiple downloads from multiple peers if we'd want to)
231+
- and then call `.download` with the information contained in the ticket and wait for the download to finish:
232+
233+
<Note>
234+
Reusing the same downloader across multiple downloads can be more efficient, e.g. by reusing existing connections.
235+
In this example we don't see this, but it might come in handy for your use case.
236+
</Note>
237+
238+
```rust
239+
let filename: PathBuf = filename.parse()?;
240+
let abs_path = std::path::absolute(filename)?;
241+
let ticket: BlobTicket = ticket.parse()?;
242+
243+
// For receiving files, we create a "downloader" that allows us to fetch files
244+
// from other endpoints via iroh connections
245+
let downloader = store.downloader(&endpoint);
246+
247+
println!("Starting download.");
248+
249+
downloader
250+
.download(ticket.hash(), Some(ticket.endpoint_addr().id)
251+
.await?;
252+
253+
println!("Finished download.");
254+
```
255+
256+
<Note>
257+
The return value of `.download()` is [`DownloadProgress`](https://docs.rs/iroh-blobs/latest/iroh_blobs/api/downloader/struct.DownloadProgress.html).
258+
You can either `.await` it to wait for the download to finish, or you can stream out progress events instead, e.g. if you wanted to use this for showing a nice progress bar!
259+
</Note>
260+
261+
As a final step, we'll export the file we just downloaded into our in-memory blobs database to the desired file path:
262+
263+
```rust
264+
println!("Copying to destination.");
265+
266+
store.blobs().export(ticket.hash(), abs_path).await?;
267+
268+
println!("Finished copying.");
269+
```
270+
271+
<Note>
272+
This first downloads the file completely into memory, then copies it from memory to file in a second step.
273+
274+
There's ways to make this work without having to store the whole file in memory!
275+
This would involve setting up an `FsStore` instead of a `MemStore` and using `.export_with_opts` with `ExportMode::TryReference`.
276+
Something similar can be done on the sending side!
277+
We'll leave these changes as an exercise to the reader 😉
278+
</Note>
279+
280+
Before we leave, we'll gracefully shut down our endpoint in the receive branch, too:
281+
282+
```rs
283+
// Gracefully shut down the endpoint
284+
println!("Shutting down.");
285+
endpoint.close().await;
286+
```
287+
288+
289+
## That's it!
290+
291+
You've now successfully built a small tool for peer-to-peer file transfers! 🎉
67292

68-
- Keep message sizes appropriate for your transport; publish references to
69-
large payloads and transfer the data via the blobs protocol.
70-
- Use HashSeq to represent sequences of blobs (e.g., chunked content).
71-
- Use the downloader utilities to aggregate sources and store results locally.
293+
The full example with the very latest version of iroh and iroh-blobs can be [viewed on github](https://github.com/n0-computer/iroh-blobs/blob/main/examples/transfer.rs).
72294

295+
If you're hungry for more, check out
296+
- the [iroh rust documentation](https://docs.rs/iroh),
297+
- Video streaming with [iroh-roq](/protocols/streaming),
298+
- Writing your [own protocols](/protocols/writing-a-protocol.mdx),
299+
- [other examples](/examples), or

protocols/rpc.mdx

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,13 @@
22
title: "RPC"
33
---
44

5-
## What is RPC?
6-
75
Remote Procedure Call (RPC) lets a program invoke functions hosted on another
86
process or machine as if they were local, abstracting the transport so arguments
97
and results travel over the network transparently.
108

119
## IRPC
1210

13-
[irpc]() is a lightweight rpc crate for iroh connections
11+
[irpc](https://docs.rs/irpc) is a lightweight rpc crate for creating RPC protocols over iroh connections.
1412

1513
### Installation
1614

protocols/streaming.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: "Audio/Video Streaming"
2+
title: "Streaming"
33
---
44

55
Streaming protocols in iroh enable efficient transfer of large or continuous

0 commit comments

Comments
 (0)