kemalseker_24111_ML_HW1_GermanCreditCard_cs412_hw1 (2).ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "kemalseker_24111_ML-HW1-GermanCreditCard_cs412_hw1.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DTfDIhAYYjaX",
        "colab_type": "text"
      },
      "source": [
        "# Load the dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HGi6c-0uZSeB",
        "colab_type": "code",
        "outputId": "e6084587-cb74-4063-cc6e-e8add44a80e5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 126
        }
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly\n",
            "\n",
            "Enter your authorization code:\n",
            "··········\n",
            "Mounted at /content/drive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yFpeyxi8aqIl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# You can find the data under https://drive.google.com/drive/folders/1e550az93U3_kfRBbVY5PZnMKYwGYmHqi?usp=sharing\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from os.path import join\n",
        "%matplotlib inline\n",
        "\n",
        "path = \"/content/drive/My Drive\"\n",
        "filename1 = \"train_data.csv\"\n",
        "\n",
        "train_data = pd.read_csv(join(path, filename1))\n",
        "\n",
        "filename2 = \"train_label.csv\"\n",
        "\n",
        "train_label = pd.read_csv(join(path, filename2))\n",
        "\n",
        "filename3 = \"test_data.csv\"\n",
        "\n",
        "test_data = pd.read_csv(join(path, filename3))\n",
        "\n",
        "\n",
        "filename4 = \"test_label.csv\"\n",
        "\n",
        "test_label = pd.read_csv(join(path, filename4))\n",
        "\n",
        "\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OZXqzZlXBIN0",
        "colab_type": "code",
        "outputId": "1335fbaa-437e-4036-fcfc-7bebe4140873",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 436
        }
      },
      "source": [
        "# show random samples from the training data\n",
        "train_data\n",
        "# One line of code"
      ],
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>duration</th>\n",
              "      <th>credit_amount</th>\n",
              "      <th>installment_commitment</th>\n",
              "      <th>residence_since</th>\n",
              "      <th>age</th>\n",
              "      <th>existing_credits</th>\n",
              "      <th>num_dependents</th>\n",
              "      <th>f_worker</th>\n",
              "      <th>checking_status_&lt;0</th>\n",
              "      <th>checking_status_&gt;=200</th>\n",
              "      <th>checking_status_no checking</th>\n",
              "      <th>credit_history_critical/other existing credit</th>\n",
              "      <th>credit_history_delayed previously</th>\n",
              "      <th>credit_history_existing paid</th>\n",
              "      <th>credit_history_no credits/all paid</th>\n",
              "      <th>purpose_domestic appliance</th>\n",
              "      <th>purpose_education</th>\n",
              "      <th>purpose_furniture/equipment</th>\n",
              "      <th>purpose_new car</th>\n",
              "      <th>purpose_other</th>\n",
              "      <th>purpose_radio/tv</th>\n",
              "      <th>purpose_repairs</th>\n",
              "      <th>purpose_retraining</th>\n",
              "      <th>purpose_used car</th>\n",
              "      <th>savings_status_500&lt;=X&lt;1000</th>\n",
              "      <th>savings_status_&lt;100</th>\n",
              "      <th>savings_status_&gt;=1000</th>\n",
              "      <th>savings_status_no known savings</th>\n",
              "      <th>employment_4&lt;=X&lt;7</th>\n",
              "      <th>employment_&lt;1</th>\n",
              "      <th>employment_&gt;=7</th>\n",
              "      <th>employment_unemployed</th>\n",
              "      <th>personal_status_male div/sep</th>\n",
              "      <th>personal_status_male mar/wid</th>\n",
              "      <th>personal_status_male single</th>\n",
              "      <th>other_parties_guarantor</th>\n",
              "      <th>other_parties_none</th>\n",
              "      <th>property_magnitude_life insurance</th>\n",
              "      <th>property_magnitude_no known property</th>\n",
              "      <th>property_magnitude_real estate</th>\n",
              "      <th>other_payment_plans_none</th>\n",
              "      <th>other_payment_plans_stores</th>\n",
              "      <th>housing_own</th>\n",
              "      <th>housing_rent</th>\n",
              "      <th>job_skilled</th>\n",
              "      <th>job_unemp/unskilled non res</th>\n",
              "      <th>job_unskilled resident</th>\n",
              "      <th>own_telephone_yes</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>6</td>\n",
              "      <td>1169</td>\n",
              "      <td>4</td>\n",
              "      <td>4</td>\n",
              "      <td>67</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>48</td>\n",
              "      <td>5951</td>\n",
              "      <td>2</td>\n",
              "      <td>2</td>\n",
              "      <td>22</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>12</td>\n",
              "      <td>2096</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>49</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>42</td>\n",
              "      <td>7882</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>45</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>24</td>\n",
              "      <td>4870</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>53</td>\n",
              "      <td>2</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>788</th>\n",
              "      <td>789</td>\n",
              "      <td>48</td>\n",
              "      <td>6224</td>\n",
              "      <td>4</td>\n",
              "      <td>4</td>\n",
              "      <td>50</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>789</th>\n",
              "      <td>790</td>\n",
              "      <td>40</td>\n",
              "      <td>5998</td>\n",
              "      <td>4</td>\n",
              "      <td>3</td>\n",
              "      <td>27</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>790</th>\n",
              "      <td>791</td>\n",
              "      <td>21</td>\n",
              "      <td>1188</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>39</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>791</th>\n",
              "      <td>792</td>\n",
              "      <td>24</td>\n",
              "      <td>6313</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>41</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>792</th>\n",
              "      <td>793</td>\n",
              "      <td>6</td>\n",
              "      <td>1221</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>27</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>793 rows × 49 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "      id  duration  ...  job_unskilled resident  own_telephone_yes\n",
              "0      1         6  ...                       0                  1\n",
              "1      2        48  ...                       0                  0\n",
              "2      3        12  ...                       1                  0\n",
              "3      4        42  ...                       0                  0\n",
              "4      5        24  ...                       0                  0\n",
              "..   ...       ...  ...                     ...                ...\n",
              "788  789        48  ...                       0                  0\n",
              "789  790        40  ...                       0                  1\n",
              "790  791        21  ...                       0                  0\n",
              "791  792        24  ...                       0                  1\n",
              "792  793         6  ...                       0                  0\n",
              "\n",
              "[793 rows x 49 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wQIVU4-zXpE_",
        "colab_type": "text"
      },
      "source": [
        "# Train Decision Tree with default parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oLznLl_lXYZf",
        "colab_type": "code",
        "outputId": "25421698-d73c-42d5-f1c7-d61c776d9bf6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.tree import plot_tree\n",
        "\n",
        "\n",
        "\n",
        "# Train decision tree using the whole training data with **entropy** criteria\n",
        "\n",
        "# One line of code\n",
        "tre=DecisionTreeClassifier(criterion='entropy')\n",
        "# One line of code\n",
        "tre = tre.fit(train_data , train_label)\n",
        "\n",
        "\n",
        "# Estimate the prediction of test data\n",
        "test_pred = tre.predict(test_data) # One line of code\n",
        "\n",
        "# Calculate accuracy of test data\n",
        "from sklearn.metrics import accuracy_score\n",
        "TestAcc = accuracy_score(test_label, test_pred)# One line of code\n",
        "print(\"Testing Accuracy = %.5f%%\" % (TestAcc * 100))"
      ],
      "execution_count": 47,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Testing Accuracy = 67.63285%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cqgNZYUMXv8X",
        "colab_type": "text"
      },
      "source": [
        "# FineTune Decision Tree parameters\n",
        "\n",
        "1- Spliting dataset into train and validation "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nWJxk-zjy0Kv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Split training data to 70% training and 30% validation\n",
        "from sklearn.model_selection import train_test_split\n",
        "x_train, x_val, y_train, y_val = train_test_split(train_data, train_label, test_size=0.3)# One line of code"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cqws-kTZYHoG",
        "colab_type": "text"
      },
      "source": [
        "2- FineTune minimum sample split"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O1DvpmCCJXTb",
        "colab_type": "code",
        "outputId": "6e9cfe38-5ac9-421f-dcc0-39d4c3913221",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "min_samples_splits = range(2, 100)\n",
        "\n",
        "train_results = []\n",
        "val_results = []\n",
        "for min_samples_split in min_samples_splits:\n",
        "  \n",
        "  # Fit the tree using the 70% portion of the training data\n",
        "  # One line of code\n",
        "  dt = DecisionTreeClassifier(min_samples_split=min_samples_split)\n",
        "  # One line of code\n",
        "  dt.fit(x_train, y_train)\n",
        "  \n",
        "  # Evaluate on Training set\n",
        "  train_pred = dt.predict(x_train)# One line of code\n",
        "  train_acc = accuracy_score(y_train, train_pred)# One line of code\n",
        "  train_results.append(train_acc)\n",
        "   \n",
        "  # Evaluate on Validation set\n",
        "  val_pred = dt.predict(x_val)# One line of code\n",
        "  val_acc = accuracy_score(y_val , val_pred  )# One line of code\n",
        "  val_results.append(val_acc)\n",
        "  \n",
        "# Ploting\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.plot(min_samples_splits, train_results, 'b')\n",
        "plt.plot(min_samples_splits, val_results,'r')\n",
        "plt.show()"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxV1bn/8c+TMCMKSIrIzBVkcGBI\noRbbWhFFsaJWfwbQ4nCFtqI41YtDr4q9jlVre6mKljohONQhLbQ4QbVeB4KCihYZVEjEioJahcqQ\n5/fHOmkOIcNJck72Gb7v12u/ztnT2c/JTp6svdbaa5u7IyIi2Ssv6gBERCS1lOhFRLKcEr2ISJZT\nohcRyXJK9CIiWa5Z1AFU1alTJ+/Vq1fUYYiIZJSlS5d+4u4F1a1Lu0Tfq1cvSkpKog5DRCSjmNkH\nNa1T1Y2ISJZTohcRyXJK9CIiWU6JXkQkyynRi4hkuToTvZnNNrOPzeytGtabmf3azFab2RtmNjRu\n3SQzWxWbJiUzcBERSUwiJfp7gDG1rD8a6BubJgO3A5hZR+BKYAQwHLjSzDo0JlgREam/OhO9uz8P\nbKplk3HAfR68DLQ3sy7AUcDT7r7J3TcDT1P7P4xG2bYNLrkEPqixJ6mISG5KRh19V2B93HxpbFlN\ny3djZpPNrMTMSjZu3NigIEpL4c474YQTYMuWBn2EiEhWSovGWHef5e6F7l5YUFDtHbx16tMHHnwQ\nli2DyZNBz1MREQmSkejLgO5x891iy2panjJjx8I118CcOXDrrak8kohI5khGoi8GfhTrffMt4HN3\n3wAsBI40sw6xRtgjY8tS6rLL4Ic/hJ/9DP7851QfTUQk/dU5qJmZzQUOAzqZWSmhJ01zAHe/A1gA\nHAOsBrYAZ8TWbTKza4AlsY+a4e61NeomhRnccw+sWQPHHw9z58KJJ6b6qCIi6cvS7eHghYWFnozR\nKzdtClU5r74Kd98NZ5yRhOBERNKUmS1198Lq1qVFY2wqdOwIzzwDo0bBmWfCHXdEHZGISDSyNtED\ntG0Lf/wjHH00nH8+rFwZdUQiIk0vqxM9QMuWMHs2tG4NZ58N5eVRRyQi0rSyPtED7LMP3HwzvPAC\n3HVX1NGIiDStnEj0EBpjR40KwySUpbQ3v4hIesmZRG8Gs2bB9u1w4YVRRyMi0nRyJtFDGCbhjDNg\nwQLYuTPqaEREmkZOJXqA4cPhyy/h3XejjkREpGnkXKIvjN1OsHRptHGIiDSVnEv0/ftDmzaQhJtv\nRUQyQs4l+vx8GDJEiV5EckfOJXqAYcPg9dfVICsiuSEnE31hYXgK1d//HnUkIiKpl5OJftiw8KoG\nWRHJBTmZ6PffPwx4pnp6EckFOZnoKxpkVaIXkVyQk4keQj3966/Djh1RRyIiklo5m+iHDYOtW9Ug\nKyLZL6FEb2ZjzGylma02s+nVrO9pZs+a2RtmttjMusWt22lmy2JTcTKDb4yKO2RVTy8i2a7ORG9m\n+cBM4GhgIDDezAZW2eyXwH3ufhAwA7gubt1Wdx8cm45LUtyN1q8f7LGH6ulFJPslUqIfDqx297Xu\nvg2YB4yrss1A4LnY+0XVrE87eXkwdKhK9CKS/RJJ9F2B9XHzpbFl8ZYDJ8benwC0M7O9Y/OtzKzE\nzF42s+OrO4CZTY5tU7Jx48Z6hN843/xmaJD97LMmO6SISJNLVmPsxcD3zOx14HtAGVAxwEBPdy8E\nJgC/MrP/qLqzu89y90J3LywoKEhSSHUbPx6+/hoeeKDJDiki0uQSSfRlQPe4+W6xZf/m7h+6+4nu\nPgS4PLbss9hrWex1LbAYGNL4sJNj2LAw3XknuEcdjYhIaiSS6JcAfc2st5m1AIqAXXrPmFknM6v4\nrEuB2bHlHcysZcU2wEjg7WQFnwxTpsBbb8FLL0UdiYhIatSZ6N19BzAVWAi8Azzs7ivMbIaZVfSi\nOQxYaWbvAp2B/4ktHwCUmNlyQiPt9e6eVol+/Hho1y6U6kVEspF5mtVZFBYWekkTd4X5yU/gnnvg\nww+hQ4cmPbSISFKY2dJYe+hucvbO2HhTpsC//gX33Rd1JCIiyadEDwweHB4arkZZEclGSvQx55wD\n77wD998fdSQiIsmlRB9z6qlwyCFwwQXw8cdRRyMikjxK9DF5eXD33fDllzBtWtTRiIgkjxJ9nIED\n4fLLYd48+NOfoLw89K+/8UZowpEZRESSqlnUAaSb6dPhkUfg9NOheXP46KOw/KOP4JZbIg1NRKRB\nVKKvokULmD0b9twTvvMdmDMHjjsO7r03dMEUEck0KtFX45vfhLVrK+c7d4biYnj00dBoKyKSSVSi\nT8D3vw/77adhEkQkMynRJyAvDyZPhr/9DVasiDoaEZH6UaJP0Omnh/r7WbOijkREpH6U6BNUUAAn\nnhjGw9m6NepoREQSp0RfD1OmhMcO3nVX1JGIiCROib4evvc9GDUKzj9fDbMikjmU6OvBDP74Rzjm\nGPjxj+H666OOSESkbkr09dS6NTz+OEyYAJdeCldfHXVEIiK10w1TDdC8eRjOuGVLuOqqMEbOySdH\nHZWISPUSKtGb2RgzW2lmq81sejXre5rZs2b2hpktNrNucesmmdmq2DQpmcFHKS8Pbr8dvv3t0PXy\njTeijkhEpHp1JnozywdmAkcDA4HxZjawyma/BO5z94OAGcB1sX07AlcCI4DhwJVmljVPZW3ZMgyL\n0L49HH88fPpp1BGJiOwukRL9cGC1u691923APGBclW0GAs/F3i+KW38U8LS7b3L3zcDTwJjGh50+\nunSBxx6DsjIoKoIdO6KOSERkV4kk+q7A+rj50tiyeMuBE2PvTwDamdneCe6LmU02sxIzK9mYgQO/\njxgRqnGeeSYMcywikk6S1evmYuB7ZvY68D2gDNiZ6M7uPsvdC929sKCgIEkhNa0zzwzPnb355jC0\nsYhIukgk0ZcB3ePmu8WW/Zu7f+juJ7r7EODy2LLPEtk3m9x6K3z3u/Cf/wklJbB9e5jKy6OOTERy\nWSKJfgnQ18x6m1kLoAgojt/AzDqZWcVnXQrMjr1fCBxpZh1ijbBHxpZlpebNw9OpOnUKY9q3aBGm\nnj1h3bqooxORXFVnP3p332FmUwkJOh+Y7e4rzGwGUOLuxcBhwHVm5sDzwDmxfTeZ2TWEfxYAM9x9\nUwq+R9r4xjdg8WJ4+OFQki8vhxtuCHfSzp8f7q4VEWlK5u5Rx7CLwsJCLykpiTqMpLrttjA+zgMP\nwMSJUUcjItnIzJa6e2F16zQEQhOYOjX0zJk2DTKwU5GIZDgl+iaQnw+/+x188QVccEHU0YhIrlGi\nbyKDBoVB0ObMgeeeq3t7EZFkUaJvQpdeCt27h5uq0qxpRESymBJ9E2rVCmbMgCVL4A9/iDoaEckV\nSvRN7LTTQjXOZZeFm6lERFJNib6J5efDddfBqlUwe3bd24uINJYePBKBY4+FkSPDQ0u6dAk3UZnB\n4MHQrVudu4uI1IsSfQTM4MYbw8PGx1UZ8LmwMIxtv//+1e87aBAMGJD6GEUkeyjRR+Tb34a1a+Hj\nj8P8tm1h6IQnn4Qrrqh5v5Yt4YUXwlg6IiKJ0BAIaegf/6j+Dtp//QtOOik83GTpUujcueljE5H0\nVNsQCCrRp6HOnWtO4k88Ea4GfvjDcONVixZNG5uIZB71uskwgweH3jovvhjGzhERqYsSfQYqKoKL\nLoI77ghVOCIitVGiz1A//zm0awc33RR1JCKS7pToM9Ree4WHmTzyCLz3XtTRiEg6U6LPYNOmhTtt\nb7kl6khEJJ0p0Wewrl3h1FPDWPeffBJ1NCKSrhJK9GY2xsxWmtlqM5tezfoeZrbIzF43szfM7JjY\n8l5mttXMlsWmO5L9BXLdxRfD1q3w299GHYmIpKs6+9GbWT4wExgNlAJLzKzY3d+O2+wK4GF3v93M\nBgILgF6xdWvcfXByw5YKAweGsXN+/Ws46CA48kho0ybqqEQknSRSoh8OrHb3te6+DZgHVBmhBQf2\njL3fC/gweSFKXa6+OryecAJ06hTGz/n97/V8WhEJErkztiuwPm6+FBhRZZurgKfM7FygLXBE3Lre\nZvY68AVwhbu/UPUAZjYZmAzQo0ePhIOXYOhQ2LABnn8+jJXzxBNQXAx5eXDIIaEuH8JgahMnwg9+\nEG28ItK0ktUYOx64x927AccA95tZHrAB6OHuQ4ALgQfNbM+qO7v7LHcvdPfCgoKCJIWUW5o3h1Gj\nQhXOBx/Aa6+Fvvbbt8Mbb4Tpr38NI2PefXfU0YpIU0qkRF8GdI+b7xZbFu8sYAyAu79kZq2ATu7+\nMfB1bPlSM1sD9ANye9SyFDODIUPCdNVVlcu3bAmDop19NmzeDD/7WWQhikgTSqREvwToa2a9zawF\nUAQUV9lmHTAKwMwGAK2AjWZWEGvMxcz6AH2BtckKXuqnTZtQrXPKKXDJJWFwtFGjwvTTn6pOXyRb\n1Zno3X0HMBVYCLxD6F2zwsxmmNlxsc0uAs42s+XAXOB0D+Mffxd4w8yWAY8CP3b3Tan4IpKYFi1g\nzhy4/PJws9W2bfD116E6Z//9w2t5edRRikgyaTx6AeCdd8KQCs8/D337Qvv2Yfnee8NDD8Geu7Ws\niEg6qW08et0ZK0B4POHixaFbZr9+oZvmXnvBX/4CDzwQdXQi0hgq0UuN3GHYMNi5E5YtC428IpKe\nVKKXBjGDKVNC18xXXok6GhFpKCV6qdWECbDHHnDnnVFHIiINpUQvtWrXLiT7hx6Czz6LOhoRaQgl\neqnTlClhhMz77486EhFpCCV6qdPQoVBYGKpv0qztXkQSoEQvCZkyBVasCAOlme065eWFPvg7dkQd\npYhUJ5GxbkQ47bQwPs6XX+6+bv36UNrfuBEefBBatmz6+ESkZkr0kpCWLWsfBO3gg+H888NDUO67\nr/pk36YNtGqVuhhFpHpK9JIU06ZBhw5w5pmw777Vb9OiRRhAbdw4OO446NKlaWMUyVW6M1aS6qWX\nYMmS6te9/354MMra2PilI0aE8fGPPx7692+yEEWyUm13xirRS5Nyh7feCgn/ySeh4lT36xdK+j/4\nAXTuHJbl50OfPhp6QSQRSvSStkpLK5P+okW799w599zw1CwRqZ0SvWSEzz8PI2h+9VWY//Ofw8iZ\nf/0rfPe7kYYmkvaU6CUjffUVHHhgeB7u8uXqsSNSG41eKRmpbVuYNQvefRdmzIg6GpHMpe6VktaO\nOAJOPx1uvBEGDQoPRKmqT5/wVCwRqV5Cid7MxgC3AfnA3e5+fZX1PYB7gfaxbaa7+4LYukuBs4Cd\nwHnuvjB54UsuuPlmWLgQTj21+vXNm4cB1045pWnjEskUdSZ6M8sHZgKjgVJgiZkVu/vbcZtdQXho\n+O1mNhBYAPSKvS8CBgH7As+YWT9335nsLyLZq2PH0CVz5crd15WXw/TpMH48fPEFnH1208cnku4S\nKdEPB1a7+1oAM5sHjAPiE70DFY+P3gv4MPZ+HDDP3b8G3jOz1bHPeykJsUsO6dgRDjmk+nULF8JJ\nJ8HkybBmTWjArY1ZeETi/vsnP06RdJRIou8KrI+bLwVGVNnmKuApMzsXaAscEbfvy1X27Vr1AGY2\nGZgM0KNHj0TiFvm3Nm3giSdg0iS44YbE9+vfP9ykVVPCLyiAww8Pny+SyZLVGDseuMfdbzazQ4D7\nzeyARHd291nALAjdK5MUk+SQFi3CyJnXXQfbt9e+7bZtob/+E0+E+v/ahldu3RqOPBK+//3qB2pr\n1iz08e/Xr1Hhi6RUIom+DOgeN98ttizeWcAYAHd/ycxaAZ0S3FckKcygV6/Eth00CM45J9Trb95c\n/Tbvvlt51+6TT9b+ef37h4Hauu52vdowrVuHK5QWLZLzeZLbEkn0S4C+ZtabkKSLgAlVtlkHjALu\nMbMBQCtgI1AMPGhmtxAaY/sCryYpdpFG23PPMFWnZ08YPRp+8xv4+OPqn671z3+GNoInn4Rbbknu\nw1fatIGJE5P3eZK76kz07r7DzKYCCwldJ2e7+wozmwGUuHsxcBFwl5ldQGiYPd3DLbcrzOxhQsPt\nDuAc9biRTGNWOdBaVfvsE/rwT50anqu7dWvjj+cOAwfC/PlK9JIcGgJBJA2dfjoUF4criWa6rVES\noCEQRDLM2LGh7eDll+veVqQuSvQiaejII0NJfv78qCORbKBEL5KG9toLDj1UiV6SQ4leJE2NHQtv\nvgnr1kUdiWQ6JXqRNDV2bHhdsCDaOCTzKdGLpKn+/cMNYKq+kcZSohdJU2ahVP/ss8npny+5Sz10\nRdLY2LEwc2Z4AEtNd/CmwuTJcMIJTXc8SS0lepE0dvjhYQydjz6CTZua5pgbNoQ7ct96Kzy9SzKf\nEr1IGmvZsu4B1ZKttDQMwTBlCjz1VKhCksymOnoR2UW3bmFc/2eegXvuiToaSQYlehHZzZQp8J3v\nwIUXhmojyWyquhGR3eTlwV13wcEHw+DB0L59WN6pExx7LBx/fOj+KZlBo1eKSI0efxweeqhyfs0a\nqPjz7NIFWrVK7fHbtw/j/RcUpPY42aC20StVoheRGp1wwu7dLEtLwxDKr74K5eWpO/b27TBvHsyd\nC+edl7rj5AKV6EUkbQ0dGkbxfFXPpauTxqMXkYw0cSIsWRKe3ysNp0QvImlr/PjQj3/OnKgjyWwJ\nJXozG2NmK81stZlNr2b9rWa2LDa9a2afxa3bGbeuOJnBi0h223dfGDUKHnig+oezS2LqTPRmlg/M\nBI4GBgLjzWxg/DbufoG7D3b3wcBvgMfiVm+tWOfuxyUxdhHJARMnwtq1eqxiYyRSoh8OrHb3te6+\nDZgHjKtl+/HA3GQEJyJy4omhG6eqbxouke6VXYH1cfOlwIjqNjSznkBv4Lm4xa3MrATYAVzv7k9U\ns99kYDJAjx49EotcRHLCnnvCuHGhq+X06ZCfH3VEqdO8ebgpLdmS3Y++CHjU3XfGLevp7mVm1gd4\nzszedPc18Tu5+yxgFoTulUmOSUQy3Kmnhhu3unePOpLUGjEiNVVUiST6MiD+x9sttqw6RcA58Qvc\nvSz2utbMFgNDgDW77yoiUr1jjoEHH4Qvvog6ktT6xjdS87mJJPolQF8z601I8EXAhKobmVl/oAPw\nUtyyDsAWd//azDoBI4EbkxG4iOSOvLzQ1VIaps5E7+47zGwqsBDIB2a7+wozmwGUuHtFl8kiYJ7v\neqvtAOBOMysnNPxe7+5vJ/criIhIbTQEgohIFtAQCCIiOUyJXkQkyynRi4hkOSV6EZEsp0QvIpLl\nlOhFRLKcEr2ISJZTohcRyXJK9CIiWU6JXkQkyynRi4hkOSV6EZEsp0QvIpLllOhFRLKcEr2ISJZT\nohcRyXJK9CIiWU6JXkQkyyWU6M1sjJmtNLPVZja9mvW3mtmy2PSumX0Wt26Sma2KTZOSGbyIiNSt\nzoeDm1k+MBMYDZQCS8ysOP4h3+5+Qdz25wJDYu87AlcChYADS2P7bk7qtxARkRolUqIfDqx297Xu\nvg2YB4yrZfvxwNzY+6OAp919Uyy5Pw2MaUzAIiJSP4kk+q7A+rj50tiy3ZhZT6A38Fx99jWzyWZW\nYmYlGzduTCRuERFJULIbY4uAR919Z312cvdZ7l7o7oUFBQVJDklEJLclkujLgO5x891iy6pTRGW1\nTX33FRGRFKizMRZYAvQ1s96EJF0ETKi6kZn1BzoAL8UtXghca2YdYvNHApc2KmIRaTrFxfDQQ5Xz\n++0H06dD69bRxST1Vmeid/cdZjaVkLTzgdnuvsLMZgAl7l4c27QImOfuHrfvJjO7hvDPAmCGu29K\n7lcQkZS58UZ47TXo2hXc4cEH4bHHYO5cOOCAqKOTBFlcXk4LhYWFXlJSEnUYIgLQuzeMHAkPPBDm\nFy6ESZPg88/h8suhe/fa96/Nd74DffokJ07BzJa6e2F16xKpuhGRXOQOH34YSvMVjjoKli+H00+H\nn/+8cZ9fUABvvQXf+EbjPkfqpEQvkixbtoSSbjK0agUdOtS9XSp98gls27Zrogfo3BkWLIDSUtix\no2GfvX49jB4NkyfD44+DWePjlRop0Yskw8qVcMghsDlJN32bQUkJDB2anM9riNLS8Fo10UOIrzHV\nNr17w7XXwsUXw733hisESRklepHG2r4dTjstJL/f/hbyGnl7ymefhZ4ty5ZFm+jLYj2hq0v0yXDB\nBfDHP8J558Fhh0GvXqk5jijRizTatdfCkiXw8MNw8smN/7zt2+Gyy+D99xv/WY1Rkei7dUvN5+fl\nhdL8gQfCiBGhzr6qDh3gySehY8fUxJAjlOhFGmPJErjmGpg4MTlJHqB585Bc0yHR5+XBPvuk7hg9\ne8If/gCzZoXG33jbtoUS/8MPw49/nLoYcoASfab64AOYPx9+8hM1ZDXGQw/BK680fP/iYujSBf73\nf5MXE4RqjHRI9J07Q7MUp4nRo8NUlTsMGgTz5inRN5ISfab61a/ClJenP4KGcg8/u6++Cr1cGqJd\nO5gzB9q3T25svXrBokXJ/cz6KitLXf18IsygqAiuuir6WDJc9j5h6r334Msvo44idV58MbxedBGs\nWhVtLJlq06bQ8HnDDfDFFw2byspCQ2Ky9eoVPnvbtuR/dqJKS6NPrkVF4R/yww9HG0eGy85E//XX\nMGRI42/oSFdbtsDrr8OPfgQtW4bXhvZnzmVr1oTX//iPaOOoTq9eUF4e+ptHpawsdQ2xierXL/Q8\nmjcv2jgyXHYm+r/9Ldy4EvWlb6q8+mpI7CefHLrzvfxyKJVK/aRzou/dO7xGVU+/ZUu42om6RA+h\nVP/qq5XnS+otOxP9U0+F1zffDJfXiZg/P9wAkowHn9xySyiJ/PWvjf+s6vzf/4XXb387/BGcckqo\nx7z99t17LkjNVq8Or+k43kpFn/KoEn2q+9DXxymnhNf4UTSlXrIz0S9cGBrJystDaTcRt98e6iQf\neaRxx3aH3/wm1Jt///uh+ijZ1SovvggDBlT2Lb7jDhg1Cn76UzjxRPj00+QeL1utWRMSWToOudut\nG+TnK9ED9OgRBlZT9U2DZV+vm48+CoMuXXYZXH99SIpHHln7Pps2hX8OEH6ZfvrThh//lVfCH+fM\nmeEW9l/8Ap59NgzvWtOdf+XlMG0aTJgQbqOvTXl5KNGfdFLlsvbtw9gjv/pVuKPygANg4MDd9zUL\nx/nBDxr67bLLmjXpWW0DoUtjlH3paxv+IApFRXDuuaHhOz8/6mhSZ8CA5HfVJRtL9BXVNiedFO64\nq+idUpvHHgul7uOPhxdeqPwlb4h580ID6cSJMHt2GLd7xQoYPLjmS8+XXgon97rr6v78d94Jdacj\nR+66PC8PLrwwXMEMHRp6a1SdVq2C8eNV11lh9erwII10FWVf+nQq0UMoBB1zDOzcWf3vdrZM27en\n5MeXfSX6p54Kw54efHBIhvfeG5J4bTd9zJ0LffuGhyw88URIyBddVP9j79wZ9j3mGNhrr7CsqCjc\n3j1hQnj/1FOhqqV588r9Ki5J//KXcHVR2+3eFf+4qib6CkOHhvaG6qxfH/75TZoU2g+qKxmtWhV+\nBuXlYb59+zDgVMuWNceUib76Klz9pWuJHkKif/bZaI5dVgZ77hmqQNNBx441/15LnbKrRF9eHhLp\n6NGhhDtyZPiDfuONmvfZsCH0zhk/PiT7YcMaXhf4/PMheYwfv+vy3r3Duv/6r1DKv//+ynU7doQ+\nwgMGhP/mjz9e+zFefDGMCdKQkmj37uHK4cUX4Ze/3HWdO/z+9+HK45JLQhXQ9OnhhqIRI8KVRDZZ\nuza8pnuiLysL3YWbmm5QyirZleiXLQu9Zo46KsxXlHprq7555JGQ5Cpa9ouKQt16Q25CmjsX9tgD\nxo7dfV3z5qFqZvBguOmmyhLz4sXw8cdhvJT99qv7n8yLL4bv1dBhDyZODNVaP/95KCG99lr4vhMm\nwJlnwvDh4WazLVvCVFwc/uiHDQtdOV97bfdp2bJwNZNJKnrcpHvVjXs0femV6LOLu9c5AWOAlcBq\nYHoN2/w/4G1gBfBg3PKdwLLYVFzXsYYNG+YNdt117uC+YUOYLy9379rV/ZRTat7nkEPcDzqocn7d\nuvAZ11xTv2N//bV7x47uEyfWvt2cOeHzn3wyzJ95pnu7du5btrhfcYV7Xl5l/FV99FHY96ab6hdb\nVZ984r7PPuGzKqb8fPf/+R/3HTt2376szH3UqF23rzqdfHL4eWeKm24KcW/eHHUkNVu8OMT49NNN\nf+yuXd0nTWr640qDEZ7hXW1erbOO3szygZnAaKAUWGJmxe7+dtw2fYFLgZHuvtnM4p8NttXdBzfy\n/1FiFi4MdfMVo+2ZhdJvTSX6998PDaHXXlu5rHt3OPTQUDq//PLES85PPx3q14uKat/u5JNDj6Cb\nbgpXHo89FhqBW7cO+/7iF/DoozB16u771lU/n6i99w4l8SVLKpf16wf9+1e//b77hiqxRYtCVVhV\nixfDrbeG3jynnda42JrKmjWh3jfZY9QkU1R96XfuDFWQKtFnj5r+A1RMwCHAwrj5S4FLq2xzI/Cf\nNez/ZV3HiJ8aXKL/5z/dmzd3v+SSXZffdlsoFX3wgfunn4bS/d57h2mPPcK6tWt33WfmzLC8Y8ew\nXdeu7vPn77rNjh3uP/xh5We1bu3eoUMo2delIqbp08PrggWV6w480H3kyN33+eIL96OOcm/Z0v1f\n/0rsZ9JUduxwP/RQ9z33DD/nhpo/332//Sp/pgUF7tOmuW/dmrxYKxxxhPvw4cn/3GTavj1caV1+\nedMet6ws/F7+9rdNe1xpFBpToge6AvGVhKXAiCrb9AMwsxeBfOAqd/9LbF0rMysBdgDXu/sT9f93\nlICtW+Gcc8INQ/EqSr833xxKz//4B5x6KrRpE5b37195u3mF004LpagtW8L8s8+Gnipvvll5tXDj\njWEc7QkTKp/tefjh0KJF3bGedRZcfXXo57/33nDEEZXriorClcS6deFGEQgl7wkTQgPiTTelXw+Y\n/PzQu+ngg+GMM8LVTX2esvT116Gh+rbbwj0AFVdFGzeGZYsWhbaLAQOSF/OaNfCtbyXv81KhWbNw\nhdnUJfp061opjZas7pXNgP/DAvEAAAdDSURBVL7AYUA34HkzO9DdPwN6unuZmfUBnjOzN919l47c\nZjYZmAzQoyK51VdBQag+qOrgg6FtW/j1r0OvmpdeCg2LtWnXLiTyCm+/Hbotnn12aJxcvhyuvDJU\nwzzwQP0bRtu2Df+UrrkmNIzGd7WsSPTjx4eHMmzbFp6w06VL6BJ56KH1O1ZT6dMn/PzPPhuOPjr8\nA0vU8uXhZ3zeeWHMnvghg3/0o9C9c9gwOO646v+BHHRQ6CGUqG3bwnj+Eycmvk9UouhLr0SfdRJJ\n9GVA/FOAu8WWxSsFXnH37cB7ZvYuIfEvcfcyAHdfa2aLgSHALone3WcBswAKCwuTO1hLs2bh2ZSb\nN4cS9B571P8zBg4M+15wQeh5cvvt0KlTeG1o75dzzw117lXvwu3TJ5SK//a3ynF3xo8PJduKK4d0\nddZZIWkvXBh67iSqbdvwJKFjj9193dixoXvs1KmhXaGqr74K7SlHHRVGLE3EBx+EXk/p3OOmQq9e\n4QqpKaXbXbHSeDXV6XhlHXszYC3QG2gBLAcGVdlmDHBv7H0nQlXP3kAHoGXc8lXAwNqO16heN6m0\nc6f74YdX9jKJr1eX6GzeHHotjR+f+D5//nM4hy+8kLq4kuWqq9zNmrZdZvp092bNwu+8ZAxqqaOv\nsyLV3XcAU4GFwDvAw+6+wsxmmNlxsc0WAp+a2dvAIuBn7v4pMAAoMbPlseXXe1xvnYySlxduKCoo\nCFUMRx8ddUQCodfMlCnhprNEqzjSeXjiqnr3DkWLdeua7phlZaGnVX3aWSStJVRH7+4LgAVVlv13\n3HsHLoxN8dv8H3Bg48NMEz16hD+4hj52TlJj2rRQtXXrreG1LqtXh8b4VD70OlkqulguXVrZgSDV\n3ntP1TZZJvvGukk1Jfn0061baFi9+2747/+uuyG4YtTKTHioesVVR9VhNVKtrvtBJKMo0Ut2uPhi\nuOee0Fhe1yMk16wJN4hlgq5dQ0P1hg1Ne9zRo5v2eJJSSvSSHQYNCj10rruu7vGC/v73MMJopqiu\nN5JIPSjRS/a4/vowhERdA6wdeGDmDNUgkgRK9JI9DjhAj5sTqYb6T4mIZDklehGRLKdELyKS5ZTo\nRUSynBK9iEiWU6IXEclySvQiIllOiV5EJMtZGHgyfZjZRuCDOjbrBHzSBOGkI3333JSr3z1XvzfU\n/7v3dPeC6lakXaJPhJmVuHth1HFEQd9d3z2X5Or3huR+d1XdiIhkOSV6EZEsl6mJflbUAURI3z03\n5ep3z9XvDUn87hlZRy8iIonL1BK9iIgkSIleRCTLZVyiN7MxZrbSzFab2fSo40klM+tuZovM7G0z\nW2Fm02LLO5rZ02a2KvbaIepYU8HM8s3sdTP7U2y+t5m9Ejv3D5lZi6hjTAUza29mj5rZ383sHTM7\nJIfO+QWx3/W3zGyumbXK1vNuZrPN7GMzeytuWbXn2YJfx34Gb5jZ0PocK6MSvZnlAzOBo4GBwHgz\nGxhtVCm1A7jI3QcC3wLOiX3f6cCz7t4XeDY2n42mAe/Ezd8A3Oru+wGbgbMiiSr1bgP+4u79gYMJ\nP4OsP+dm1hU4Dyh09wOAfKCI7D3v9wBjqiyr6TwfDfSNTZOB2+tzoIxK9MBwYLW7r3X3bcA8YFzE\nMaWMu29w99di7/9J+IPvSvjO98Y2uxc4PpoIU8fMugFjgbtj8wYcDjwa2yRbv/dewHeB3wG4+zZ3\n/4wcOOcxzYDWZtYMaANsIEvPu7s/D2yqsrim8zwOuM+Dl4H2ZtYl0WNlWqLvCqyPmy+NLct6ZtYL\nGAK8AnR29w2xVR8BnSMKK5V+BVwClMfm9wY+c/cdsflsPfe9gY3A72PVVnebWVty4Jy7exnwS2Ad\nIcF/DiwlN857hZrOc6NyX6Yl+pxkZnsAfwDOd/cv4td56B+bVX1kzexY4GN3Xxp1LBFoBgwFbnf3\nIcBXVKmmycZzDhCrjx5H+Ge3L9CW3as2ckYyz3OmJfoyoHvcfLfYsqxlZs0JSX6Ouz8WW/yPisu2\n2OvHUcWXIiOB48zsfUL13OGEeuv2sUt6yN5zXwqUuvsrsflHCYk/2885wBHAe+6+0d23A48Rfhdy\n4bxXqOk8Nyr3ZVqiXwL0jbXCtyA01BRHHFPKxOqlfwe84+63xK0qBibF3k8Cnmzq2FLJ3S91927u\n3otwjp9z94nAIuCk2GZZ970B3P0jYL2Z7R9bNAp4myw/5zHrgG+ZWZvY737Fd8/68x6npvNcDPwo\n1vvmW8DncVU8dXP3jJqAY4B3gTXA5VHHk+Lveijh0u0NYFlsOoZQX/0ssAp4BugYdawp/BkcBvwp\n9r4P8CqwGngEaBl1fCn6zoOBkth5fwLokCvnHLga+DvwFnA/0DJbzzswl9AWsZ1wJXdWTecZMEKP\nwzXAm4SeSQkfS0MgiIhkuUyruhERkXpSohcRyXJK9CIiWU6JXkQkyynRi4hkOSV6EZEsp0QvIpLl\n/j/KPwBiMeOdlwAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "akVAE3MbL7bE",
        "colab_type": "code",
        "outputId": "e4a42d8c-70c2-4fed-cb58-85bdd01bf00b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "# Choose the best minimum split sample based on the plot\n",
        "max_val_results = max(val_results)\n",
        "Best_minSampl = min_samples_splits[val_results.index(max_val_results)]  # One line of code\n",
        "\n",
        "\n",
        "# Train decision tree using the full training data and the best minimum split sample\n",
        "# One line of code\n",
        "tree_dt = DecisionTreeClassifier(min_samples_split=Best_minSampl)\n",
        "# One line of code\n",
        "tree_dt.fit(train_data , train_label)\n",
        "\n",
        "# Estimate the prediction of the test data\n",
        "test_pred = tree_dt.predict(test_data)# One line of code\n",
        "\n",
        "# Calculate accuracy of test data\n",
        "TestAcc = accuracy_score(test_label, test_pred)# One line of code\n",
        "print(\"Testing Accuracy = %.5f%%\" % (TestAcc * 100))"
      ],
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Testing Accuracy = 70.53140%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VciE0lfYKhya",
        "colab_type": "text"
      },
      "source": [
        "# Now, apply the same procedure but using KNN instead of decision tree \n",
        "\n",
        "# For finetuning, find the best value of K to use with this dataset."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EdBaXNKoLBcL",
        "colab_type": "code",
        "outputId": "70e0db45-6cb4-4007-b8f8-93165b6f258a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 90
        }
      },
      "source": [
        "# Write your code here\n",
        "\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "\n",
        "# initialize the values of k to be a list of odd numbers between 1 and 30\n",
        "kVals = range(1, 30, 2)\n",
        "\n",
        "# Save the accuracies of each value of kVal in [accuracies] variable\n",
        "# hint: you can use accuracies.append(...) function inside the loop\n",
        "accuracies = []\n",
        "\n",
        "# loop over values of k for the k-Nearest Neighbor classifier\n",
        "for k in kVals:\n",
        "  # Follow what we did in decision tree part\n",
        "  model = KNeighborsClassifier( n_neighbors = k)\n",
        "  model.fit(x_train, y_train.label.ravel())\n",
        "\n",
        "  score = model.score(x_val, y_val)\n",
        "  #print(\"For k = %d, validation accuracy = %.5f%%\" % (k, score * 100))\n",
        "  # Update the accuracies list\n",
        "  accuracies.append(score)\n",
        "\n",
        "# Train KNN using the full training data with the best K that you found\n",
        "i = np.argmax(accuracies)\n",
        "print(\"best k = %d with %.5f%% validation accuracy\" % (kVals[i], accuracies[i] * 100))\n",
        "\n",
        "\n",
        "model = KNeighborsClassifier( n_neighbors=kVals[i])\n",
        "model.fit(train_data , train_label)\n",
        "\n",
        "# Testing\n",
        "test_pred = model.predict(test_data)\n",
        "TestAccuracy = accuracy_score(test_label, test_pred)\n",
        "print(\"Testing Accuracy = %.5f%%\" % (TestAccuracy * 100))\n",
        "\n"
      ],
      "execution_count": 51,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "best k = 25 with 71.42857% validation accuracy\n",
            "Testing Accuracy = 70.04831%\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:28: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zMEdOIlJQBin",
        "colab_type": "text"
      },
      "source": [
        "# Bonus\n",
        "\n",
        "# Apply gridsearch using decision tree on any hyperparameter(s) of your choice, you have to beat your previous obtained accuracies to get the bonus\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J-9TxVbXQCw7",
        "colab_type": "code",
        "outputId": "eebe6614-955e-4192-dbed-d9149f9caf84",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "# Write your code here\n",
        "from sklearn import tree\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "\n",
        "parameters={'max_leaf_nodes': list(range(2, 100))}\n",
        "clf_tree=tree.DecisionTreeClassifier()\n",
        "#clf=grid_search.GridSearchCV(clf_tree,parameters)\n",
        "clf = GridSearchCV(clf_tree, parameters)\n",
        "clf.fit(train_data,train_label)\n",
        "\n",
        "print(\"best max leaf nodes:\" ,  clf.best_estimator_.get_params()['max_leaf_nodes'] )\n",
        "\n",
        "clf_tree = tree.DecisionTreeClassifier(max_leaf_nodes=  clf.best_estimator_.get_params()['max_leaf_nodes'])\n",
        "\n",
        "clf_tree.fit(train_data , train_label)\n",
        "\n",
        "test_pred = clf_tree.predict(test_data)# One line of code\n",
        "\n",
        "# Calculate accuracy of test data\n",
        "TestAcc = accuracy_score(test_label, test_pred)# One line of code\n",
        "print(\"Testing Accuracy = %.5f%%\" % (TestAcc * 100))\n"
      ],
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "best max leaf nodes: 15\n",
            "Testing Accuracy = 73.42995%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1VCH10cwnAU-",
        "colab_type": "text"
      },
      "source": [
        "# Report: Write a summary of your approach to this problem; this should be like an abstract of a paper or the executive summary (you aim for clarity and passing on information, not going to details about known facts such as what decision trees are, assuming they are known to people in your research area).\n",
        "\n",
        "Must include statements such as:\n",
        "\n",
        "\n",
        "*   Include the problem definition: 1-2 lines\n",
        "*   Talk about train/val/test sets, size and how split.\n",
        "*   State what your test results are with the chosen method, parameters: e.g. \"We have obtained the best results with the ….. classifier (parameters=....) , giving classification accuracy of …% on test data….\"\n",
        "*   Comment on the speed of the algorithms and anything else that you deem important/interesting (e.g. confusion matrix)\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "72KDasqHnt1T",
        "colab_type": "text"
      },
      "source": [
        "# Write your report in this cell\n",
        "The dataset contains information about clients of a German Bank. Each entry is a person with its attributes. And each person has a label indicating whether a person good(not risky) or bad(risky).  For training and testing my machine learning models, we have to datasets. Firstly, for training, we have 793 entries.  Secondly for the test, we have 207 entries. \n",
        "Then, I use the Decision Tree algorithm with default parameters. My testing accuracy is 67.63285%.  Then I split the training data to %70 train data and %30 validation data. Then I test the training data with 'min_sample_splits'. The parameters range is from 2 to 100. And I tune every number in this range to find the best accuracy score in the validation set. And then I took that the best 'min_sample_split' and use it as a parameter before fit data. The testing accuracy becomes 70.53140%.\n",
        "After I use KNeighborsClassifier. I tune the values of k odd numbers between 1 and 30. I find the best k parameter as 25.  Using k=25 gives 70.04831% testing accuracy. Then I use the gridsearch to improve the test accuracy. I choose 'max_leaf_nodes' as the hyperparameter. The best 'max_leaf_nodes' parameter is 15. When I tune it, testing accuracy 73.42995%.  \n",
        "..\n",
        "\n",
        "..\n",
        "\n",
        "..\n"
      ]
    }
  ]
}