The Kaggle Book

Table of contents

1. Preface
2. Part I: Introduction to Competitions
3. Introducing Kaggle and Other Data Science Competitions
   1. The rise of data science competition platforms
      1. The Kaggle competition platform
         1. A history of Kaggle
      2. Other competition platforms
   2. Introducing Kaggle
      1. Stages of a competition
      2. Types of competitions and examples
      3. Submission and leaderboard dynamics
         1. Explaining the Common Task Framework paradigm
         2. Understanding what can go wrong in a competition
      4. Computational resources
         1. Kaggle Notebooks
      5. Teaming and networking
      6. Performance tiers and rankings
      7. Criticism and opportunities
   3. Summary
4. Organizing Data with Datasets
   1. Setting up a dataset
   2. Gathering the data
   3. Working with datasets
   4. Using Kaggle Datasets in Google Colab
   5. Legal caveats
   6. Summary
5. Working and Learning with Kaggle Notebooks
   1. Setting up a Notebook
   2. Running your Notebook
   3. Saving Notebooks to GitHub
   4. Getting the most out of Notebooks
      1. Upgrading to Google Cloud Platform (GCP)
      2. One step beyond
   5. Kaggle Learn courses
   6. Summary
6. Leveraging Discussion Forums
   1. How forums work
   2. Example discussion approaches
   3. Netiquette
   4. Summary
7. Part II: Sharpening Your Skills for Competitions
8. Competition Tasks and Metrics
   1. Evaluation metrics and objective functions
   2. Basic types of tasks
      1. Regression
      2. Classification
      3. Ordinal
   3. The Meta Kaggle dataset
   4. Handling never-before-seen metrics
   5. Metrics for regression (standard and ordinal)
      1. Mean squared error (MSE) and R squared
      2. Root mean squared error (RMSE)
      3. Root mean squared log error (RMSLE)
      4. Mean absolute error (MAE)
   6. Metrics for classification (label prediction and probability)
      1. Accuracy
      2. Precision and recall
      3. The F1 score
      4. Log loss and ROC-AUC
      5. Matthews correlation coefficient (MCC)
   7. Metrics for multi-class classification
   8. Metrics for object detection problems
      1. Intersection over union (IoU)
      2. Dice
   9. Metrics for multi-label classification and recommendation problems
      1. MAP@{K}
   10. Optimizing evaluation metrics
       1. Custom metrics and custom objective functions
       2. Post-processing your predictions
          1. Predicted probability and its adjustment
   11. Summary
9. Designing Good Validation
   1. Snooping on the leaderboard
   2. The importance of validation in competitions
      1. Bias and variance
   3. Trying different splitting strategies
      1. The basic train-test split
      2. Probabilistic evaluation methods
         1. k-fold cross-validation
         2. Subsampling
         3. The bootstrap
   4. Tuning your model validation system
   5. Using adversarial validation
      1. Example implementation
      2. Handling different distributions of training and test data
   6. Handling leakage
   7. Summary
10. Modeling for Tabular Competitions
    1. The Tabular Playground Series
    2. Setting a random state for reproducibility
    3. The importance of EDA
       1. Dimensionality reduction with t-SNE and UMAP
    4. Reducing the size of your data
    5. Applying feature engineering
       1. Easily derived features
       2. Meta-features based on rows and columns
       3. Target encoding
       4. Using feature importance to evaluate your work
    6. Pseudo-labeling
    7. Denoising with autoencoders
    8. Neural networks for tabular competitions
    9. Summary
11. Hyperparameter Optimization
    1. Basic optimization techniques
       1. Grid search
       2. Random search
       3. Halving search
    2. Key parameters and how to use them
       1. Linear models
       2. Support-vector machines
       3. Random forests and extremely randomized trees
       4. Gradient tree boosting
          1. LightGBM
          2. XGBoost
          3. CatBoost
          4. HistGradientBoosting
    3. Bayesian optimization
       1. Using Scikit-optimize
       2. Customizing a Bayesian optimization search
       3. Extending Bayesian optimization to neural architecture search
       4. Creating lighter and faster models with KerasTuner
       5. The TPE approach in Optuna
    4. Summary
12. Ensembling with Blending and Stacking Solutions
    1. A brief introduction to ensemble algorithms
    2. Averaging models into an ensemble
       1. Majority voting
       2. Averaging of model predictions
       3. Weighted averages
       4. Averaging in your cross-validation strategy
       5. Correcting averaging for ROC-AUC evaluations
    3. Blending models using a meta-model
       1. Best practices for blending
    4. Stacking models together
       1. Stacking variations
    5. Creating complex stacking and blending solutions
    6. Summary
13. Modeling for Computer Vision
    1. Augmentation strategies
       1. Keras built-in augmentations
          1. ImageDataGenerator approach
          2. Preprocessing layers
       2. albumentations
    2. Classification
    3. Object detection
    4. Semantic segmentation
    5. Summary
14. Modeling for NLP
    1. Sentiment analysis
    2. Open domain Q&A
    3. Text augmentation strategies
       1. Basic techniques
       2. nlpaug
    4. Summary
15. Simulation and Optimization Competitions
    1. Connect X
    2. Rock-paper-scissors
    3. Santa competition 2020
    4. The name of the game
    5. Summary
16. Part III: Leveraging Competitions for Your Career
17. Creating Your Portfolio of Projects and Ideas
    1. Building your portfolio with Kaggle
       1. Leveraging Notebooks and discussions
       2. Leveraging Datasets
    2. Arranging your online presence beyond Kaggle
       1. Blogs and publications
       2. GitHub
    3. Monitoring competition updates and newsletters
    4. Summary
18. Finding New Professional Opportunities
    1. Building connections with other competition data scientists
    2. Participating in Kaggle Days and other Kaggle meetups
    3. Getting spotted and other job opportunities
       1. The STAR approach
    4. Summary (and some parting words)
19. Other Books You May Enjoy
20. Index