README.md

Computer Vision Tutorial

Computer Vision Tutorial includes classical theories and techniques and also recent ML/DL-based methods for computer vision. The tutorial covers classical theories and techniques such as image processing, camera projection models, camera calibration, and pose estimation. It also covers recent ML/DL-based methods including object categorization (and backbone networks) and its extensions such as object detection and instance segmentation. It also explains about further topics such as multi-object tracking, structure-from-motion, NeRF, and so on.

This tutorial has been initiated and maintained to teach undergraduate CSE students in SEOULTECH as the course of Computer Vision (109079).

This tutorial contains concise code examples written in Python with OpenCV and PyTorch.

• 💡 Some code examples help readers understand how algorithms work internally.
• 🔧 Some code examples demonstrate how to use OpenCV functions in practical applications.
• 📷 Some code examples are from my 3D Vision Tutorial, 3dv_tutorial.

Lecture Slides

• Section 1. Introduction
• Section 2. Image Editing: Learning OpenCV
• Section 3. Image Processing
• Section 4. Color
• Section 5. Image Formation
• Section 6. Image Geometry
• Section 7. Solving Problems
• Section 8. Image Correspondence
• Section 9. Image Classification: CNN Backbones
  • Acknowledgement) Many slides about CNNs are adopted from Stanford CS231n.
• Section 10. Image Segmentation
• Section 11. Object Detection
• Section 12. Vision Foundation Models
• Section 13. Object Tracking
• Section 14. 3D Vision
  • Acknowledgement) Many slides about NeRF are adopted from NeRF Tutorial (ECCV 2022).

Example Codes

• Section 1. Introduction [slides]

  • Note) How to install prerequisite packages in Python: pip install -r requirements.txt

• Section 2. Image Editing: Learning OpenCV [slides]

  • OpenCV Image Representation
    • Image creation: image_creation.py 💡
  • OpenCV Image and Video Input/Output
    • Image file viewer: image_viewer.py 🔧
    • Image format converter: image_converter.py 🔧
    • Video file player: video_player.py 🔧
    • Video format converter: video_converter.py 🔧
  • OpenCV Drawing Functions
    • Shape drawing: shape_drawing.py 🔧
  • OpenCV High-level GUI
    • (Handling keyboard events) Video file player with frame navigation: video_player+navigation.py 🔧
    • (Handling mouse events) Free drawing: free_drawing.py 🔧
  • Image Editing
    • Negative image and flip: negative_image_and_flip.py 💡
    • Intensity transformation with contrast and brightness: intensity_transformation.py 💡
    • (Image addition) Alpha blending: alpha_blending.py 💡
    • (Image addition) Background extraction: background_extraction.py 💡
    • (Image subtraction) Image difference: image_difference.py 💡
    • (Image subtraction) Background subtraction: background_subtraction.py 💡
    • (Image crop) Image file viewer with the zoom window: image_viewer+zoom.py 💡
    • Image resize with backward value copy: image_resize.py 💡
    • Image rotation with backward/forward value copy: image_rotation.py 💡

• Section 3. Image Processing [slides]

  • Intensity Transformation
    • Image histogram: histogram.py 💡
    • Contrast stretching with min-max stretching: contrast_stretching.py 💡
    • Histogram equalization: histogram_equalization.py 🔧
  • Image Segmentation
    • Thresholding: thresholding.py 🔧
  • Image Filtering
    • Image filtering with various kernels: image_filtering.py 💡
    • Median filter: median_filter.py 🔧
    • Sobel edge detection: Sobel_edge.py 💡
    • Canny edge detection: Canny_edge.py 🔧
    • Bilateral filter: bilateral_filter.py 🔧
  • Morphological Operations
    • Morphological operations with various operations and kernels: morphology.py 🔧
    • Application) Background subtraction (foreground extraction): background_subtraction.py 🔧

• Section 4. Color [slides]

  • Color space conversion: color_bgr2hsv.py 🔧
  • Color histogram equalization: histogram_equalization+color.py 💡

• Section 5. Image Formation [slides]

  • Getting Started with 2D
    • 3D rotation conversion: 3d_rotation_conversion.py 📷
  • Pinhole Camera Model
    • Object localization: object_localization.py 📷
    • Image formation: image_formation.py 📷💡
  • Geometric Distortion Models
    • Geometric distortion visualization: distortion_visualization.py 📷
    • Geometric distortion correction: distortion_correction.py 📷 [result video]
    • Geometric distortion correction with virtual PTZ fisheye_virtual_ptz.py 📷
  • Camera Calibration
    • Camera calibration: camera_calibration.py 📷
  • Absolute Camera Pose Estimation (a.k.a. perspective-n-point; PnP)
    • Pose estimation (chessboard): pose_estimation_chessboard.py 📷 [result video]
    • Pose estimation (book): pose_estimation_book1.py 📷
    • Pose estimation (book) with camera calibration: pose_estimation_book2.py 📷
    • Pose estimation (book) with camera calibration without initial $K$: pose_estimation_book3.py 📷 [result video]

• Section 6. Image Geometry [slides]

  • Planar Homography
    • Perspective distortion correction: perspective_correction.py 📷
    • Planar image stitching: image_stitching.py 📷
    • 2D video stabilization: video_stabilization.py 📷 [result video]
  • Triangulation
    • Triangulation: triangulation.py 📷

• Section 7. Solving Problems [slides]

  • Solving Linear Equations in 3D Vision
    • Affine transformation estimation: affine_estimation_implement 📷 💡
    • Planar homography estimation: homography_estimation_implement 📷 💡
      • Appendix) Image warping using homography: image_warping_implement.py 📷 💡
    • Triangulation: triangulation_implement.py 📷 💡
  • Solving Nonlinear Equations in 3D Vision
    • Absolute camera pose estimation: pose_estimation_implement.py 📷 💡
    • Camera calibration: camera_calibration_implement.py 📷 💡

• Section 8. Image Correspondence [slides]

  • Feature Points and Descriptors
    • Harris corner: harris_corner_implement.py 📷 💡
    • SuperPoint [Github]
  • Feature Matching and Tracking
    • Feature matching comparison: feature_matching.py 📷
    • SuperGlue [Github]
    • Feature tracking with KLT tracker: feature_tracking_klt.py 📷
  • Outlier Rejection
    • Line fitting with RANSAC: line_fitting_ransac.py 📷 💡
    • Planar homography estimation with RANSAC: image_stitching_implement.py 📷 💡

• Section 9. Image Classification: CNN Backbones

• Section 10. Image Segmentation

• Section 11. Object Detection

• Section 12. Vision Foundation Models

• Section 13. Object Tracking

• Section 14. 3D Vision [slides]

  • Structure-from-Motion
    • COLMAP [Homepage] [Documentation] [Github]
  • 3D Representations
    • NeRF [Homepage] [Github]

Authors

• Sunglok Choi

Acknowledgements

The authors thank the following contributors and projects.

• Youngjin Hong: He reported two bugs in image_rotation.py and object_localization.py.
• ImageProcessingPlace.com for test images (lena.tif, baboon.tif, and peppers.tif)
• MOTChallenge for test images (PETS09-S2L1-raw.webm)
• Wikipedia for a test image (salt_and_pepper.png)
• OpenCV for a test image (sudoku.png)