Add 4 Machine Learning Algorithms: Decision Tree Pruning, Logistic Regression, Naive Bayes, and PCA#13350
Conversation
- Decision Tree Pruning: Implements decision tree with reduced error and cost complexity pruning - Logistic Regression Vectorized: Vectorized implementation with support for binary and multiclass classification - Naive Bayes with Laplace Smoothing: Handles both discrete and continuous features with Laplace smoothing - PCA from Scratch: Principal Component Analysis implementation with sklearn comparison All algorithms include: - Comprehensive docstrings with examples - Doctests (145 total tests passing) - Type hints throughout - Modern NumPy API usage - Comparison with scikit-learn implementations - Ready for TheAlgorithms/Python contribution
algorithms-keeper
bot
added
require descriptive names
- Changed all X, X_train, X_test, X_val variables to lowercase - Updated function parameters and variable references - Decision tree now passes all ruff checks - Follows TheAlgorithms/Python strict naming conventions
omsherikar
force-pushed
the
feature/machine-learning-algorithms
branch
from
9241b1d to
8e97c39
Compare
- Changed all x, x_train, x_test variables to lowercase - Updated function parameters and variable references - Logistic regression now passes all ruff checks - Naive bayes has only 1 minor line length issue in a comment - Follows TheAlgorithms/Python strict naming conventions
omsherikar
force-pushed
the
feature/machine-learning-algorithms
branch
from
f64f82f to
0841d09
Compare
for more information, see https://pre-commit.ci
![algorithms-keeper[bot]](https://avatars.githubusercontent.com/in/235834?s=60&v=4){kind=small}
There was a problem hiding this comment.
Click here to look at the relevant links ⬇️
🔗 Relevant Links
Repository:
Python:
Automated review generated by algorithms-keeper. If there's any problem regarding this review, please open an issue about it.
algorithms-keeper commands and options
algorithms-keeper actions can be triggered by commenting on this PR:
@algorithms-keeper reviewto trigger the checks for only added pull request files@algorithms-keeper review-allto trigger the checks for all the pull request files, including the modified files. As we cannot post review comments on lines not part of the diff, this command will post all the messages in one comment.NOTE: Commands are in beta and so this feature is restricted only to a member or owner of the organization.
| else: | ||
| self.rng_ = np.random.default_rng() | ||
|
|
||
| def _mse(self, y: np.ndarray) -> float: |
There was a problem hiding this comment.
As there is no test file in this pull request nor any test function or class in the file machine_learning/decision_tree_pruning.py, please provide doctest for the function _mse
Please provide descriptive name for the parameter: y
| return 0.0 | ||
| return np.mean((y - np.mean(y)) ** 2) | ||
|
|
||
| def _gini(self, y: np.ndarray) -> float: |
There was a problem hiding this comment.
As there is no test file in this pull request nor any test function or class in the file machine_learning/decision_tree_pruning.py, please provide doctest for the function _gini
Please provide descriptive name for the parameter: y
| probabilities = counts / len(y) | ||
| return 1 - np.sum(probabilities ** 2) | ||
|
|
||
| def _entropy(self, y: np.ndarray) -> float: |
There was a problem hiding this comment.
As there is no test file in this pull request nor any test function or class in the file machine_learning/decision_tree_pruning.py, please provide doctest for the function _entropy
Please provide descriptive name for the parameter: y
| probabilities = probabilities[probabilities > 0] # Avoid log(0) | ||
| return -np.sum(probabilities * np.log2(probabilities)) | ||
|
|
||
| def _find_best_split( |
There was a problem hiding this comment.
As there is no test file in this pull request nor any test function or class in the file machine_learning/decision_tree_pruning.py, please provide doctest for the function _find_best_split
| return -np.sum(probabilities * np.log2(probabilities)) | ||
|
|
||
| def _find_best_split( | ||
| self, x: np.ndarray, y: np.ndarray, task_type: str |
There was a problem hiding this comment.
Please provide descriptive name for the parameter: x
Please provide descriptive name for the parameter: y
|
|
||
| # Our implementation | ||
| pca_ours = PCAFromScratch(n_components=2) | ||
| X_transformed_ours = pca_ours.fit_transform(X) |
There was a problem hiding this comment.
Variable and function names should follow the snake_case naming convention. Please update the following name accordingly: X_transformed_ours
|
|
||
| # Scikit-learn implementation | ||
| pca_sklearn = sklearn_pca(n_components=2, random_state=42) | ||
| X_transformed_sklearn = pca_sklearn.fit_transform(X) |
There was a problem hiding this comment.
Variable and function names should follow the snake_case naming convention. Please update the following name accordingly: X_transformed_sklearn
| print(f"\nCorrelation between implementations: {correlation:.6f}") | ||
|
|
||
|
|
||
| def main() -> None: |
There was a problem hiding this comment.
As there is no test file in this pull request nor any test function or class in the file machine_learning/pca_from_scratch.py, please provide doctest for the function main
|
|
||
| # Apply PCA | ||
| pca = PCAFromScratch(n_components=2) | ||
| X_transformed = pca.fit_transform(X) |
There was a problem hiding this comment.
Variable and function names should follow the snake_case naming convention. Please update the following name accordingly: X_transformed
| print(f"Total variance explained: {np.sum(pca.explained_variance_ratio_):.4f}") | ||
|
|
||
| # Demonstrate inverse transform | ||
| X_reconstructed = pca.inverse_transform(X_transformed) |
There was a problem hiding this comment.
Variable and function names should follow the snake_case naming convention. Please update the following name accordingly: X_reconstructed
1 participant
Describe your change:
This PR adds 4 comprehensive machine learning algorithms to the machine_learning directory:
decision_tree_pruning.py) - Implements decision tree with reduced error and cost complexity pruninglogistic_regression_vectorized.py) - Vectorized implementation with support for binary and multiclass classificationnaive_bayes_laplace.py) - Handles both discrete and continuous features with Laplace smoothingpca_from_scratch.py) - Principal Component Analysis implementation with sklearn comparisonAll algorithms include comprehensive docstrings, 145 doctests (all passing), type hints, modern NumPy API usage, and comparison with scikit-learn implementations.
Fixes #13320
Checklist:
Algorithm Details:
1. Decision Tree Pruning
machine_learning/decision_tree_pruning.py2. Logistic Regression Vectorized
machine_learning/logistic_regression_vectorized.py3. Naive Bayes with Laplace Smoothing
machine_learning/naive_bayes_laplace.py4. PCA from Scratch
machine_learning/pca_from_scratch.pyTesting Results:
np.random.default_rng()instead of deprecatednp.random.seed()Note on Multiple Algorithms:
While the guidelines suggest one algorithm per PR, these 4 algorithms are closely related (all machine learning) and were developed together as a cohesive set. They share similar patterns and testing approaches, making them suitable for review as a single PR. If maintainers prefer, I can split this into 4 separate PRs.