Skip to content

Commit 4ad0eee

Browse files
committed
Import sbot's vision docs
2 parents e7bd4f7 + 24a4eb4 commit 4ad0eee

File tree

16 files changed

+658
-262
lines changed

16 files changed

+658
-262
lines changed

docs/api/vision/index.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# Vision
2+
3+
4+
![An arena with Fiducial Markers](../../assets/img/api/vision/arena_marker.jpg)
5+
6+
Your robot is able to use a webcam to detect [Fiducial Markers](https://en.wikipedia.org/wiki/Fiducial_marker).
7+
Specifically it will detect [AprilTags](https://april.eecs.umich.edu/software/apriltag), using the `36H11` marker set.
8+
9+
Using [Pose Estimation](https://en.wikipedia.org/wiki/3D_pose_estimation), it can calculate the orientation and position of
10+
the marker relative to the webcam. Using this data, it is possible to determine the location of your robot and other objects around it.
11+
12+
## Searching for markers
13+
14+
Assuming you have a webcam connected, you can use `r.camera.see()` to take a picture. The software will process the picture
15+
and returns a list of the markers it sees.
16+
17+
```python
18+
markers = r.camera.see()
19+
```
20+
21+
!!! tip
22+
Your camera will be able to process images better if they are not blurred.
23+
24+
## Saving camera output
25+
26+
You can also save a snapshot of what your webcam is currently seeing. This can be useful to debug your code.
27+
Every marker that your robot can see will have a square annotated around it, with a red dot indicating the bottom right
28+
corner of the marker. The ID of every marker is also written next to it.
29+
30+
Snapshots are saved to your USB drive, and can be viewed on another computer.
31+
32+
```python
33+
r.camera.save("snapshot.jpg")
34+
```
35+
36+
![An annotated arena with Fiducial Markers.](../../assets/img/api/vision/arena_marker_annotated.jpg){ width="50%" }
37+
38+
## Markers
39+
40+
The marker objects in the list expose data that may be useful to your robot.
41+
42+
### Marker ID
43+
44+
Every marker has a numeric identifier that can be used to determine what object it represents.
45+
46+
```python
47+
markers = r.camera.see()
48+
49+
for m in markers:
50+
print(m.id)
51+
```
52+
53+
### Position
54+
55+
Each marker has a position in 3D space, relative to your webcam.
56+
57+
You can access the position using `m.distance`, `m.cartesian` and `m.spherical`.
58+
59+
```python
60+
markers = r.camera.see()
61+
62+
for m in markers:
63+
print(m.distance) # Distance to the marker from the webcam, in metres
64+
print(m.spherical.rot_y) # Bearing to the marker from the webcam, in radians
65+
```
66+
67+
!!! note
68+
`m.distance` is equivalent to `m.cartesian.z` or `m.spherical.dist`.
69+
70+
For more information on position, including how to use `m.cartesian`, `m.spherical`, and the coordinate systems,
71+
see [Position](./position.md).
72+
73+
It is also possible to look at the [Orientation](./orientation.md) of the marker.
74+
75+
!!! tip
76+
You can use the [`math.degrees`](https://docs.python.org/3/library/math.html#math.degrees) function to convert from radians to degrees.
77+
78+
### Size
79+
80+
Markers can come in different sizes.
81+
You can access the size of a marker using `m.size`.
82+
Check the rules to find out how big the different marker types are.
83+
84+
```python
85+
markers = r.camera.see()
86+
87+
for m in markers:
88+
print(m.size)
89+
```
90+
91+
### Pixel Positions
92+
93+
The positions of various points on the marker within the image are exposed over the API. This is useful
94+
if you would like to perform your own Computer Vision calculations.
95+
96+
The corners are specified in clockwise order, starting from the top left corner of the
97+
marker. Pixels are counted from the origin of the image, which
98+
conventionally is in the top left corner of the image.
99+
100+
```python
101+
markers = r.camera.see()
102+
103+
for m in markers:
104+
print(m.pixel_corners) # Pixel positions of the marker corners within the image.
105+
print(m.pixel_centre) # Pixel positions of the centre of the marker within the image.
106+
```

docs/api/vision/orientation.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Orientation
2+
3+
![Yaw Pitch and Roll (Image source: Peking University)](../../assets/img/api/vision/yawpitchroll.png)
4+
5+
Orientation represents the rotation of a marker around the x, y, and z axes. These can be accessed as follows:
6+
7+
* `rot_x` / `pitch` - the angle of rotation in radians counter-clockwise about the Cartesian x axis.
8+
* `rot_y` / `yaw` - the angle of rotation in radians counter-clockwise about the Cartesian y axis.
9+
* `rot_z` / `roll` - the angle of rotation in radians counter-clockwise about the Cartesian z axis.
10+
11+
Rotations are applied in order of z, y, x.
12+
13+
```python
14+
markers = r.camera.see()
15+
16+
for m in markers:
17+
print(m.orientation.rot_x) # Angle of rotation about x axis.
18+
print(m.orientation.rot_y) # Angle of rotation about y axis.
19+
print(m.orientation.rot_z) # Angle of rotation about z axis.
20+
```
21+
22+
!!! note
23+
In our use case the z axis always faces the camera, and thus will appear as a clockwise rotation
24+
25+
## Examples
26+
27+
The following table visually explains what positive and negative rotations represent.
28+
29+
!!! example
30+
0 in all axes:
31+
32+
![m0x0y0z]{ width="35%" }
33+
34+
| | π/4 | -π/4 |
35+
|---:|:---:|:---:|
36+
| **`rot_x`** | ![m45x0y0z] | ![m-45x0y0z] |
37+
| **`rot_y`** | ![m0x45y0z] | ![m0x-45y0z] |
38+
| **`rot_z`** | ![m0x0y45z] | ![m0x0y-45z] |
39+
40+
[m0x0y0z]: ../../assets/img/api/vision/m0x0y0z.png
41+
[m-45x0y0z]: ../../assets/img/api/vision/m-45x0y0z.png
42+
[m0x-45y0z]: ../../assets/img/api/vision/m0x-45y0z.png
43+
[m0x0y-45z]: ../../assets/img/api/vision/m0x0y-45z.png
44+
[m0x0y0z]: ../../assets/img/api/vision/m0x0y0z.png
45+
[m0x0y45z]: ../../assets/img/api/vision/m0x0y45z.png
46+
[m0x45y0z]: ../../assets/img/api/vision/m0x45y0z.png
47+
[m45x0y0z]: ../../assets/img/api/vision/m45x0y0z.png

docs/api/vision/position.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Position
2+
3+
Your robot supports two different coordinates systems for position:
4+
5+
* Cartesian
6+
* Spherical
7+
8+
The latter is a [Polar Coordinates system](https://en.wikipedia.org/wiki/Polar_coordinate_system).
9+
10+
## Cartesian
11+
12+
![The cartesian coordinates system](../../assets/img/api/vision/cartesian.svg){ width="40%" }
13+
14+
The [cartesian coordinates system](https://en.wikipedia.org/wiki/Cartesian_coordinate_system) has three
15+
_principal axes_ that are perpendicular to each other.
16+
17+
The value of each coordinate indicates the distance travelled along the axis to the point.
18+
19+
The camera is located at the origin, where the coordinates are ``(0, 0, 0)``.
20+
21+
```python
22+
markers = r.camera.see()
23+
24+
for m in markers:
25+
print(m.cartesian.x) # Displacement from the origin in millimetres, along x axis.
26+
print(m.cartesian.y) # Displacement from the origin in millimetres, along y axis.
27+
print(m.cartesian.z) # Displacement from the origin in millimetres, along z axis.
28+
```
29+
30+
!!! tip
31+
The `y` axis decreases as you go up. This matches convention for computer vision systems.
32+
33+
## Spherical
34+
35+
![The spherical coordinates system](../../assets/img/api/vision/spherical.svg){ width="40%" }
36+
37+
The [spherical coordinates system](https://en.wikipedia.org/wiki/Spherical_coordinate_system) has
38+
three values to specify a specific point in space.
39+
40+
* `distance` - The _radial distance_, the distance from the origin to the point, in millimetres.
41+
* `rot_x` - Rotation around the X-axis, in radians, corresponding to "theta" on the diagram.
42+
* `rot_y` - Rotation around the Y-axis, in radians, corresponding to "phi" on the diagram.
43+
44+
The camera is located at the origin, where the coordinates are `(0, 0, 0)`.
45+
46+
```python
47+
markers = r.camera.see()
48+
49+
for m in markers:
50+
print(m.spherical.distance) # Distance from the origin in millimetres
51+
print(m.spherical.rot_x) # The angle from the azimuth to the point, in radians.
52+
print(m.spherical.rot_y) # The polar angle from the plane of the camera to the point, in radians.
53+
```
54+
55+
!!! tip
56+
You can use the [`math.degrees`](https://docs.python.org/3/library/math.html#math.degrees) function to convert from radians to degrees.

0 commit comments

Comments
 (0)