Home Credit Default Risk

This repository contains a complete data science workflow for predicting credit default risk using machine learning.

🔗 Original Notebook on Kaggle: View here
📓 Notebook on GitHub: View here

📈 Performance

• Model: XGBoost Classifier
• Approach: Gradient Boosting with feature engineering, hyperparameter tuning and threshold optimization
• Optimization: Optuna — 50 trials maximizing AUC-ROC
• Threshold: Tuned on Precision-Recall curve with minimum Precision constraint of 20%

🛠️ Key Features

• Data Cleaning:

  • Removal of irrelevant columns and columns with excessive missing values
  • Handling of DAYS_EMPLOYED = 365243 placeholder (retirees/unemployed → NaN)
  • Median imputation for numerical features, mode imputation for categorical features
  • One-Hot Encoding with drop_first=True to avoid multicollinearity
  • StandardScaler fitted on train set only to prevent data leakage

• Class Imbalance Handling:

  • Dataset heavily imbalanced: ~91.9% non-default vs ~8.1% default
  • Comparison of Class Weight, SMOTE and Undersampling on Logistic Regression
  • Class Weight selected for best AUC-ROC without altering the data

• Model Comparison:

  • Logistic Regression (linear baseline), Extra Trees, XGBoost
  • Extra Trees discarded due to poor probability calibration
  • XGBoost selected as final model for its non-linear capacity after feature engineering

• Feature Engineering from 6 External Sources:

  • bureau.csv — credit history with other institutions
  • previous_application.csv — past loan applications with Home Credit
  • installments_payments.csv — payment punctuality on installments
  • POS_CASH_balance.csv — active and completed POS and cash loans
  • credit_card_balance.csv — credit card usage and delays
  • bureau_balance.csv — monthly behavior on third-party credits
  • Each source validated with incremental AUC-ROC before proceeding

• Hyperparameter Tuning:

  • Optuna with 50 trials on a separate validation set (test set untouched)
  • Parameters tuned: n_estimators, max_depth, learning_rate, subsample, colsample_bytree, min_child_weight

• Interpretability with SHAP:

  • TreeExplainer — exact, non-approximated for XGBoost
  • Bar chart for global feature importance
  • Beeswarm plot for feature impact and direction
  • Waterfall plot for individual client explanation
  • Business report with risk factors, protective factors and operational recommendation

🚀 How to use

1. Clone the repo: git clone https://github.com/lucalullo/Home-Credit-Default-Risk.git
2. Install dependencies: pip install pandas numpy scikit-learn xgboost imbalanced-learn optuna shap matplotlib seaborn
3. Download the dataset from the Kaggle competition: Home Credit Default Risk
4. Run the home-credit-default-risk.ipynb notebook

---

Author: Luca Lullo
Data Scientist | Machine Learning Applied