|
| 1 | +# Vision |
| 2 | + |
| 3 | + |
| 4 | + |
| 5 | + |
| 6 | +Your robot is able to use a webcam to detect [Fiducial Markers](https://en.wikipedia.org/wiki/Fiducial_marker). |
| 7 | +Specifically it will detect [AprilTags](https://april.eecs.umich.edu/software/apriltag), using the `36H11` marker set. |
| 8 | + |
| 9 | +Using [Pose Estimation](https://en.wikipedia.org/wiki/3D_pose_estimation), it can calculate the orientation and position of |
| 10 | +the marker relative to the webcam. Using this data, it is possible to determine the location of your robot and other objects around it. |
| 11 | + |
| 12 | +## Searching for markers |
| 13 | + |
| 14 | +Assuming you have a webcam connected, you can use `r.camera.see()` to take a picture. The software will process the picture |
| 15 | +and returns a list of the markers it sees. |
| 16 | + |
| 17 | +```python |
| 18 | +markers = r.camera.see() |
| 19 | +``` |
| 20 | + |
| 21 | +!!! tip |
| 22 | + Your camera will be able to process images better if they are not blurred. |
| 23 | + |
| 24 | +## Saving camera output |
| 25 | + |
| 26 | +You can also save a snapshot of what your webcam is currently seeing. This can be useful to debug your code. |
| 27 | +Every marker that your robot can see will have a square annotated around it, with a red dot indicating the bottom right |
| 28 | +corner of the marker. The ID of every marker is also written next to it. |
| 29 | + |
| 30 | +Snapshots are saved to your USB drive, and can be viewed on another computer. |
| 31 | + |
| 32 | +```python |
| 33 | +r.camera.save("snapshot.jpg") |
| 34 | +``` |
| 35 | + |
| 36 | +{ width="50%" } |
| 37 | + |
| 38 | +## Markers |
| 39 | + |
| 40 | +The marker objects in the list expose data that may be useful to your robot. |
| 41 | + |
| 42 | +### Marker ID |
| 43 | + |
| 44 | +Every marker has a numeric identifier that can be used to determine what object it represents. |
| 45 | + |
| 46 | +```python |
| 47 | +markers = r.camera.see() |
| 48 | + |
| 49 | +for m in markers: |
| 50 | + print(m.id) |
| 51 | +``` |
| 52 | + |
| 53 | +### Position |
| 54 | + |
| 55 | +Each marker has a position in 3D space, relative to your webcam. |
| 56 | + |
| 57 | +You can access the position using `m.distance`, `m.cartesian` and `m.spherical`. |
| 58 | + |
| 59 | +```python |
| 60 | +markers = r.camera.see() |
| 61 | + |
| 62 | +for m in markers: |
| 63 | + print(m.distance) # Distance to the marker from the webcam, in metres |
| 64 | + print(m.spherical.rot_y) # Bearing to the marker from the webcam, in radians |
| 65 | +``` |
| 66 | + |
| 67 | +!!! note |
| 68 | + `m.distance` is equivalent to `m.cartesian.z` or `m.spherical.dist`. |
| 69 | + |
| 70 | +For more information on position, including how to use `m.cartesian`, `m.spherical`, and the coordinate systems, |
| 71 | +see [Position](./position.md). |
| 72 | + |
| 73 | +It is also possible to look at the [Orientation](./orientation.md) of the marker. |
| 74 | + |
| 75 | +!!! tip |
| 76 | + You can use the [`math.degrees`](https://docs.python.org/3/library/math.html#math.degrees) function to convert from radians to degrees. |
| 77 | + |
| 78 | +### Size |
| 79 | + |
| 80 | +Markers can come in different sizes. |
| 81 | +You can access the size of a marker using `m.size`. |
| 82 | +Check the rules to find out how big the different marker types are. |
| 83 | + |
| 84 | +```python |
| 85 | +markers = r.camera.see() |
| 86 | + |
| 87 | +for m in markers: |
| 88 | + print(m.size) |
| 89 | +``` |
| 90 | + |
| 91 | +### Pixel Positions |
| 92 | + |
| 93 | +The positions of various points on the marker within the image are exposed over the API. This is useful |
| 94 | +if you would like to perform your own Computer Vision calculations. |
| 95 | + |
| 96 | +The corners are specified in clockwise order, starting from the top left corner of the |
| 97 | +marker. Pixels are counted from the origin of the image, which |
| 98 | +conventionally is in the top left corner of the image. |
| 99 | + |
| 100 | +```python |
| 101 | +markers = r.camera.see() |
| 102 | + |
| 103 | +for m in markers: |
| 104 | + print(m.pixel_corners) # Pixel positions of the marker corners within the image. |
| 105 | + print(m.pixel_centre) # Pixel positions of the centre of the marker within the image. |
| 106 | +``` |
0 commit comments