chapter1/4_cifar10_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "训练一个分类器\n",
    "=====================\n",
    "\n",
    "上一讲中已经看到如何去定义一个神经网络，计算损失值和更新网络的权重。\n",
    "你现在可能在想下一步。\n",
    "\n",
    "\n",
    "关于数据？\n",
    "----------------\n",
    "\n",
    "一般情况下处理图像、文本、音频和视频数据时，可以使用标准的Python包来加载数据到一个numpy数组中。\n",
    "然后把这个数组转换成 ``torch.*Tensor``。\n",
    "\n",
    "-  图像可以使用 Pillow, OpenCV\n",
    "-  音频可以使用 scipy, librosa\n",
    "-  文本可以使用原始Python和Cython来加载，或者使用 NLTK或\n",
    "   SpaCy 处理\n",
    "\n",
    "特别的，对于图像任务，我们创建了一个包\n",
    "``torchvision``，它包含了处理一些基本图像数据集的方法。这些数据集包括\n",
    "Imagenet, CIFAR10, MNIST 等。除了数据加载以外，``torchvision`` 还包含了图像转换器，\n",
    "``torchvision.datasets`` 和 ``torch.utils.data.DataLoader``。\n",
    "\n",
    "``torchvision``包不仅提供了巨大的便利，也避免了代码的重复。\n",
    "\n",
    "在这个教程中，我们使用CIFAR10数据集，它有如下10个类别\n",
    "：‘airplane’, ‘automobile’, ‘bird’, ‘cat’, ‘deer’,\n",
    "‘dog’, ‘frog’, ‘horse’, ‘ship’, ‘truck’。CIFAR-10的图像都是\n",
    "3x32x32大小的，即，3颜色通道，32x32像素。\n",
    "\n",
    "![](https://pytorch.org/tutorials/_images/cifar10.png)\n",
    "\n",
    "\n",
    "训练一个图像分类器\n",
    "----------------------------\n",
    "\n",
    "依次按照下列顺序进行：\n",
    "\n",
    "1. 使用``torchvision``加载和归一化CIFAR10训练集和测试集\n",
    "2. 定义一个卷积神经网络\n",
    "3. 定义损失函数\n",
    "4. 在训练集上训练网络\n",
    "5. 在测试集上测试网络\n",
    "\n",
    "\n",
    "1. 读取和归一化 CIFAR10\n",
    "------------------------------\n",
    "\n",
    "使用``torchvision``可以非常容易地加载CIFAR10。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torchvision.transforms as transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "torchvision的输出是[0,1]的PILImage图像，我们把它转换为归一化范围为[-1, 1]的张量。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz\n",
      "Files already downloaded and verified\n"
     ]
    }
   ],
   "source": [
    "transform = transforms.Compose(\n",
    "    [transforms.ToTensor(),\n",
    "     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n",
    "\n",
    "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n",
    "                                        download=True, transform=transform)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n",
    "                                          shuffle=True, num_workers=2)\n",
    "\n",
    "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n",
    "                                       download=True, transform=transform)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n",
    "                                         shuffle=False, num_workers=2)\n",
    "\n",
    "classes = ('plane', 'car', 'bird', 'cat',\n",
    "           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们展示一些训练图像。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " ship plane plane   cat\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztfWmQHdd13ne73/7e7IMBMIMdBECQ4E5x0S5KoilZFuVYtiU7NmOpwpTLLtspV8Vy9MNhlX/YlZQdp0pRQtmO5ZRiSZElU5YUSxQpiaIs0QQJ7iBB7DsGg1nevHn765sf59w+5828GQwGJAYzul8VOY3b/W7f2327+5zzncVYa+Hh4eHhsfIRLPcAPDw8PDzeGPgXuoeHh8cqgX+he3h4eKwS+Be6h4eHxyqBf6F7eHh4rBL4F7qHh4fHKoF/oXt4eHisElzWC90Yc58x5jVjzEFjzKfeqEF5eHh4eFw6zFIDi4wxIYADAN4P4CSApwF83Fr7yhs3PA8PDw+PxSJxGb+9A8BBa+1hADDGfBHA/QDmfaHncjnb29t7Gaf08PDw+OnDmTNnxqy1ay523OW80EcAnFD/PgngzoV+0NvbiwcffPAyTunh4eHx04eHHnro2GKOuxwbuunQNsd+Y4x50Biz1xizt1wuX8bpPDw8PDwWwuW80E8C2Kj+vQHA6dkHWWsfttbebq29PZfLXcbpPDw8PDwWwuW80J8GsMMYs9UYkwLwMQBff2OG5eHh4eFxqViyDd1a2zTG/DaAbwMIAfy1tfblS+3n+nf/BgBg3bq1cVs+ScNqZaO4Ld1sAACCgL5BQSgWn0Tg/kqbbbUAALVmM25rWepvfPScm0S8r6eP+AYTpqVfQx0bdZzzCjJ8rkZd+k+E1H8+I8e3WjRuo76diUQSABBFdHzTyr6oQb9NKoNW1dI5WjVp+8oXPwONze/+9Xi7WaMDjelkFVOIbNs4bGRn72pr64RgEU5SkdH90nYraqq2qO147XnljrcXmcqCY+TzB8pKGJVnAADTh56Yc/zv/O5vAgCazdYlnUdfb7et2zp5lFm+9gZzj++EEO541cesfiPI9Wzxtj7GbenrYXgNGjeeWkM6rPKzpJ455GgNB6lU3PTZz3yubRzfeOTReHugb4DG1pB+a/VpAMDG4fVxW1cqpH6bdH9ePfBavO/gqfMAgHJD5rJmcBAAsG6t8IXHz54FAIyX6TloICmD4uubDuUa2TqZgsOW3O90SPPKZOi32zaPxPs2DtP20FoZ9/DGbQCAseJU3NaMqD+3jirVSryvwdfB6vdTvQoAGNm2DUvF5ZCisNZ+C8C3LqcPDw8PD483Bpf1Qn8jUGXpqaREvWZAX7QW5IuZN7TNwjuCUKTagGWOhDYgBfQFrgfyJe5hKaV09DAA4MKhw/G+zfe9n869Sb7Edf7Cqo85LJ/LySqtpIzRGifJSJuTTlstJRE0aLtR5y94S0mOvC9U8yux5BDJB34ObEskH8sSvVGSvxP82uQ/12ZYilMaThAF3CbjjmIpT+6VmXU90EHCNG3bTuLWUqXTFJxaMHef1pKi+K8ah3V/TdvvACCKL4OWlkUymo0gYClxMeqHQifp+mJtbi6xhH6xcyzqOCV5u+Pt3L1WH8f3vnWBJOMj3/lxvC84dAEAUCtV47b+n70bADDy3rfMO4pGvR5vRw263lFLabR8zkBrMe5+h6yl65nycQl1X669ZisAIJkXfu65w0cBAJUEtRWG5Jnu7u6mjcnzcdvksUM0tqas9ewg/XYbvw8G+rrjfekUSe1rh4bithS3NZsy55bl9xi/R6yd+4zyUiMkLv917EP/PTw8PFYJ/Avdw8PDY5Vg2U0uxQtkRwiyos7ls0RIBJGoQEGL1fIGk6LKvuLU8agpKo1T7dNKjYmS1NZ7w3YAwOANG+J9id4eAEBSHR+ymohAk3btKn2o2Esbkf6UMEIURaweNpVJJEjwcfw9rdcUicVEbF0RNHVWBdPIYj5oA0KdNdKkNqG4v8qUIxwXt0VaVefrrDVeNhBEmiidY5WQ/h3X2YkUDVuiazqVNOK/RnXa4PM39Ylazgyjz8umHFZzQ6Wqh0EHgtLML8t0IjTj3y0yVcZizS+O5DeLtO50Mim5fh253cke02aNdJYtdXOjEj2Ho9/fR/9+TAK+owNknmj0yPrr+UgBAJDL5ucd6/iYmDUqJTLl5DLybKxfO8ADkfXvHjnDz0gm1xXvM8E4ACChzBQ5flck8zKO/rXkYNHqor99m4VkzKTp+U6MFeK2fIVI0aSRZ27D1k3UxhfLqHdRbw+ZX3J5uR6VOl2/4vRk3JbKkINFhv8a9V6I7596lgKjyNslwkvoHh4eHqsEyy6hVyP6Kk3VRFqo8FcrUBJjsUJfz2qVXJ20q1smQcfl1Aewi92Negry1WuxNJ1I09c5k5QvrA3oUhSnxTew6ly49GePP6gBN7YUodliF6SEYlHd3sjK1z+WMBJMRirxLEjRL0KIGJJgN62E8C1zUK3JuJ1LVKQGbvhahqH064R1JxEH6viQ2Zp2d8G5LnMidXYSC12/2j2O/rYRckyABXwPoK8Vj62h3BxbxpGocqpW5MhZGncSmhB2RKySSNvYqIvDzbOThH5R99A3GW5M7q92W1QLMG5yEno0Iyz7uSeeAwCUf0iSeXhWororayj/0qbfuT9u63/PTQCAVmr+6zgzU4q3ixMkuQ7298Vt64doO6m0xnq8jmktDK0XLbrMy6I8MxO3hSmSfru6hbS8ac8eOi6k5zzJLpMAkM3QePv7RfJPrqP5VZV0Xamz6y+vu3Vr+uN9/X10fD4n74/zp8kVulKRa9rNWn9PDx1fq4kVIpGg41rKbbFRV66iS4SX0D08PDxWCfwL3cPDw2OVYNlNLuUKqRz1otgTGhy51UqJKptlzYRdtxEm5Vs0UCBbS06CPNGqUR9lK2rU9DipPAnuI5+X/rt6qb9SVdT9sSrtb1ptVuH9sQYr+5oNmkNaaaEBE091FSGXTtNAEzEBK2pXMiZRRUUuctRevtEeUakxskZUznKlyn3J7U3EJiLlD+/IRTNrbhDiM6HIZ2eSCZQY0IrJWyY7zVwVvNkWaUt/80lFALH5rBGRmhspOcNZCnKhHC/+6lBtHHXbcH9VfIAjW9W4o/DisszFIj9n+94v1vTS0Wzj9nUah/5tp7bY5MImvDb/fDv3eCbZz+zbH7cVf/giACA8TETmpLo+w7/yXgDAhnvF59zm6D6HC7C5mbTYQKvOx1uZSl0UdV4Rmk1mRcu85rMq/9OePTfQOdXaSTDhmFDj3TREvynxIxcox4W+QgYAkFbPdCJDz86EFXNJfYzMRUMDFIk62Cdpvx0pGikz4Pg4EbaplJhhCnkyuWQzND+DNqdzAECjocK/zQKBJouEl9A9PDw8VgmWXUIHR0yFisjJFugrp6M8M0weZPkjp8k95wmVS8mX++yhUwCA5/f9S9z22v4DAIA7b7sVALBtm+R/2Hk9uTJGRsiSxgydY6Ym0p6RkET6o6VVlj6qdR2RSH+rVdFAwpDnzGSgjqhLJ2gydSUtjxVJyyhE89+ut+0Zjren2QUtraTabJLmovOTOO7RSVktRZw1OKIvqTQhF0EZWU3k0JwTvC/VIdqtWpdzunM411QC9VHhXBYzdR1RR3+1tNfJPc9Jog3WkqZUVGOzOVeurU2TZDf66pzhLui2qInjeHBuSUDvYkJY5xfiIzQ5G5PsmEuQx4K2ivhtwbk5Km2Nr0fEkdWRcnkNW46Ylvsy+hw9B2OPPR+3JQ+ShNniezX8kbvjfVt+5X0AgBRLt4BodbYTGc7YsH5dvD0+NgEAKJen47YK59Np1OVe5ZhobJa5TZHX7plPZmQcYYaOD5TLcpLX7gDnTEoYWa+papHOXSvGbVMVGlOo3jebOb9MPk/E6oAic4fW0HvjldcPxm0TkzS/tSOb47ZslknZBLstZuUeJJM0h2pNpRR/A7h1L6F7eHh4rBL4F7qHh4fHKsGym1wKnIYzUxBG05EH2nc76VRMVouM0b7KTNAosqmQ4ak1hXTo6yGSYk0/+ZRu2iBmin72GZ0uKyKWCUFH1ACA4fBKw6aFSGvPrNrrhEK1Wp2Pk0vdYOIu4fyBI+Urz5qm1X7UiQzPb36f36deOyt9sHkqpcxSKY5S0yaAJM/FEaYpFYKXDB1hqmNQ+fhQrnOCGdJkwvl6K/WWM6llMzI/Z7IwoSbTqK03RWRWj1HmhKDdpENj6pCAi80OyYBUcBtJJKDlZR4qNvfCGCWcmps8d65f93znXChqNJ5nh8RuUGvX9deKE5SpGAZn+lFkmrMK6GRlzoxlndkmEvNDxCT1xE/EtjT1yE8AAJkj43FbZYaek8J7bwYAbP34vfG+BEeILmRe6YQE9DPK/afUPauQ2aM4Jms320XPYW8fmThsUswrDX4H6Jxp+SSt65ZORAe37jjBl3qPxCYtlYhrsEDn6ioIOdtdoPXTw0m/Umn1fmJTzoVxSZXbYhI+bIu5cHEbdM7256uTgwE7BzQXCDi5CLyE7uHh4bFKcFEJ3Rjz1wA+BGDUWruH2/oBfAnAFgBHAfyStXZiKQPIcXhnmJjr/ue+sACQa7LE48hT9ZnOcMRlJiXS4cAWcjfase0jcVu92f517u7SUZNMuqovcS8PKan4u4pLfctuTzWVbL/GhFJDkahOujc6KYpLhxtHNyoXKpaokmmVUyZLUkK6Nn8k2YHjcvnr7pzqlAFL921Rmy5VqZMm1T6Xkrg9SpYlJNXmvBpdZGla7Uww6Ztoi05lzSZQUipvJ+PiJbLLkdChun6SCli5YEZOkjfcl9ZEeGzqRtZKIl3NxkKkaFtq2lm7W4qMHDs/BgDo75EoxWSGUxIrN1XLInyTVb1UQtaf01CbqrJJiolx7cDa5OtlW9R/qOZZeYZSw4597Z/jtvQBmntjRvrtvnsnAGD4Ex+iuWyX1LAZpxVcovhXUdc4iOiZzimf3pA1iaipIihDkpadtFpXc08yMa4JcifpRuq+5Avk2JDlcxVnxHXZ5c7JZcS9sDdH0nhe5WZx6avXMTnaUFr6y68RGXpuVHLVRM6qoNaAS6VrLUn0SXVfXLrdRF3lmuL3wWTxzZXQ/wbAfbPaPgXgMWvtDgCP8b89PDw8PJYRF5XQrbVPGGO2zGq+H8C7efvzAL4P4A+WMoAUyxpNo13beLukJFIWBfOcOyKTUtnPOEBosFsFHLA0mU+JG+I4uwi5UlCNtHyRGxW2YSqJwNmKswkZW7lCY5qeac0ZomUpOKmCFpy7oO1gx8uwG1Ok7KHuCxsmpc3Z5dLB/LdLB1ZEkZOClMTd0dzLbnRxmTKdjZC3lSujk4h1ZTbnkej2lduKK9T5rxZlufwetITuyvTxufW4eVrtmoWzTSp30lmug0FbpkmW2lXQU1SZX0JfCG1281k2dO1K+/JLFLQTNOW+3PP+d9K51W9c3pUEyH47PSpubCmWMNNdItk1uEhCTQdJ8VovBHTczIHj8b7RLxJLkD4s8y2xNhPetCluu/bfkSabvZbc7ho6v9DsWLpFwqg1n2WJNK1LR3KPdeXSm++ioJ0EB+VcKIqbo5N4W8pFsTRNdnjT9ioju7t7Jpoqz1E351XpYls9AIR8/VIJne2Q2koluh+9fZLLpcSZI0slyVXjtKK6ytfSjIt5OBdWuWlOWjfqmW64vC7iUXnJWKoNfa219gwA8N+hixzv4eHh4fEm400nRY0xDxpj9hpj9pbL5Yv/wMPDw8NjSViq2+I5Y8x6a+0ZY8x6AKPzHWitfRjAwwAwPDw8R2tLMSlUUcRIxIRI2mqfHo6udKRDWY43rGpWjBBK5y6QW9r4eUm1WWIVMFmhtnRK1K6U+21SuTmOkLkmFYrK60wXBSZPtWrf4Ki8UI3baVnaHQ1O9Xd1BVUfCY7u1K5WDTfn+vx1MLWZwm23RS46InEB00s7Cej6sLNaZp3X5QrhibZU1GuDXTbTKh9HwGptpHLKBM69Ky72MJe4ba9jSvsDtXxN4HLJzFVvnbkmoeSXZnBxWabtnsWN86fP1ddv144dAIBvffWf4rbNm8nEcc3OrdIdX6/9zxF5+cNvPRnv6+om08Et77w+btt6A/VhdZ6jFl2H5iFK4Tr6hR/G++wJWutVlQeouY0cBjb+5s9JH7dSpLRzpU3qZ2+x1TdmQbuaOktLWjk/OKJbRy8nOR1uD7stKp8DTE0x8R/J8TP8LKcSKnqUU+km2dypI7FdJHOoXAhTgSMtxeSS5P0zZXofNCJxOshwUY9CQcy5k8UiHy9mmP5W+/OqSWUTr1Np67jeLhFLldC/DuAB3n4AwCOXPRIPDw8Pj8vCYtwW/w5EgA4aY04C+CMAfwLgy8aYTwI4DuAXlzqAgV6S3kxGpOuJCn0NE4F8MdNMUAYt+iK3qiJ5NwP6Ik+pxGW1Bn+x89JHrkm/Of/sCwCAvv71ck4mJM5PnY7bbnnP22hsgZAwNc6amMxwngZFerWcS6OanyOsNGHmpMcmu08aRWiaJPeh3J/KnOsi3+okI/Pv2nbNzV1iOx/YBp2lzxGloW5zEoSSvII0XefSOSLi/vkbX4r3nT9LEuPOG26K225/z4cBAKmuwbit2Sxz/45E1cE7Lv+JjDN2s+yQh9C5AbbaMjw66V1lzbQXl2Vshz7aSWLe16H4xcgIVYvfNCwFGr737e8DANYMyLp7+ikqLPH9R34EAKieFgmv3qTtc2eF5Pxo988CAIY3SSV7nCW3vGNf+wGN41Vxp0uU6V5VhkSa3PbL9wAAhm7bJXPhObT4HuvMm0kOEDMddbT50aUk2CAuDShSqJPaA0VGlqscgMTableX9DE9xflmGiJxl5mYDAuyJtNJVyiF3ZnTKqdRltZrTfVh+fnrTqtyepHLL0RP87kJcX10onZXl2Q4Hef9FWVWdgRpnZ/fbFa5pLL0Xq3JOKrVyzdJL8bL5ePz7HrvZZ/dw8PDw+MNg48U9fDw8FglWPZcLi7KsyujqsCzmSSlXJXTfFzTpd8siX9qin06xyclN0WTSZJ0KAaQnCVf3MnxEwCAiZqYbc5zfcWZc5JXorT9OgBA/4jkfFnD9RVLrLJZRWS4aLE2n3PmRXQUYY0jPl3UpPYRr3C/TdWHqxeaTYjf/FJhdXSl8902rt6oIhlZrdRETYuJ68rUhbhtcozSFD/16D8AAF768ffkeFYrD78qKYyPHqJCCjfc/q64bdNOKlzQvYaqtCdUyl533VqRHrdLFzvX5KLz/zg4U0tDmXI0ETcHcUpb7ZdPf3VuFmeWiJyvvCLaWhx5fO2ua+K2HzzyXQDA/5n8Qty2/+UzAIDKFN33ZF2KHJg8bQ+sFVNAnZP9ROfENHPiH56i+T1Hppl8Q0j8CyEdP3SPmL3WvYdqblr1zLncMPUKqf015bsddtOaDxdBJLchkHFI7IDOVeNMjnJci00tzic7TMqadIVKGvWaOt7FMGjillM0Mxl/7bXXxntOc3TnkePH4ra+XjKd9PaJk4RxJhc2iWh/8TTHjzRVPdA6x7bUVCrgEr+jMmxO7uqS/ELOFFutyv3Wv10qvITu4eHhsUqw7BK6I8LQlC9VjkeVUFFljlRZM0DSQqC+9M//iN20EvIFTPZSrFOrJpJ8ENF2Y5Qk+ddbItFPl6i/7LgQE08+8jgAYHD3trjthjt3AwD611OOjmRWpJxKhUudKSnOFZmIFKsXtVyOB9P2FxCpr6kk0pCLXiTbItlmoS2C0ZUi035SnOdDFaCIJXI+vjItIWrjp4/x3yNx29HXSLo+fuDFuG1slCTMUtG5dSlJmiXcRku0pJ889k0AwL4fPRq3bdpCLn63vp0KKVz7lrfH+wZHyMVPZ7tz7pBQbmFOeHQ5c9pcMDtkRbQL8Htul+lQiEK7noW8Pl0pMh3x2OIBnTsmBGX9HF2byYxoOK3zdM2jGZLOTL+cc9OdWwAA17/thrit39J1OPbNp6TfZ0kyT3FekFJGufVxOba+t+yM24IukvgbitxO8KwnzlEOGh0F2XOjENizsVDZvVJFrkeGta5MUhHqvP4CnWOHXQwPHaJ8KX0DEqFZ4FJ1E+NjegQAgKRa165s4qaNREhn87J2vvcjIp9nlDR86x23AQCaSsp3eYVq7Cqc7xYC1EWg6ria4jS9W7JlIXEvXBjj8dCcu7tln5PQSzOq4Ae7YAahqqV5ifASuoeHh8cqgX+he3h4eKwSLLvJZZp9sftUulhHek0pYqY2QappinXeTFPUnX379gEAnj4hZpv1Gyi6LlSpStcUyS86OEVmhFGV4GumyoUGpsTsMMzE1qlpUfH+8bvfAQDcfQsRLe+4V8i93MhGAEBFOaK7tJvVqqqTaVxye45kU+SK839NKILN3aVggfRIgao2HvJ32ipTR41Vu/FJMQGcP30UAHDuGKm3R/c/F+87fZwiFyvTktSpXnXVNzqMQ+wUMg72ve8fkIrpZU5sNDMthPTr+18GABw59BoAYOhRiVPbddPtAIDdt741btuyi0i9gkpN62wulpOrtRefMG5A0hTOfy0ddB9CDit7misCwsel1T2b4qrxrzx9NG5LNOg6FBISOdvHkckmS2PccL1EkV73Vqp9O1yQ6zfxBF2jaO+JuK2LrSOTKSbrrlVmih302+7tym+dScjAapMmbZ84TP3WVAKsnTc709Lc4hsLweras7yZSipzAhONFWV2mDlGz/UEmzOmy/JM37TnRhqHcjBIu5S6GTHbZPhd4uqAllQaX5e+98SZc9LGtVJdgRUA6OHteotI/7J6F0V8nzVxXOft8oScK8XPd55T9U4oX/YEmxC1aasyQ89EocebXDw8PDx+6rHsEvrpM0ymqQ/+un5OodktX6pyRNJhrUrHnz51Kt6XGaAq3Du75AtbPkuSaE2RnIdnSPru5gIUF8ZFGj/NuTF6MvKNu5FPX7ogX3PX73Oj5N44fkii+O79N5QNoW/LxritwQSojjBMpkiasFyWzuqcD9aVIpMmJ8kHqhTZbJw/sj/ePnaIts8ceS1uO3vsdQDAhXNn4rbiBZpDZaakT01wbno6bW2AucfNih4MFWuYSbM2pdIPz/B1CDQ5NstF8vTRo/E+t/3ME4/FbTuvIwn9pjuU1H4dSbO964gIS6jIY8OucC11D1rR/G6LbhydIkW122fEBHbGkIRXnhCt41tfIk3ulb2yPkJL931iXFIfTUzQ/AYyJE2aCdEGu0+TtFd7Re7Z9DNEVveU5LpXWMNL7t4CAKjvEPKt7wZyBQ0HhNSL4MrjqfnxXIrn6JlYs2Et5qCDUL5QGb7uvMqvErgCHrI+LEvaLvoaALoGSesaHKFo2qeefjred+oUXcuNG+T5clHXrmAEAOS5bJzbd+BVeTa2bKR3RZiR+1icome6b5306+6zIzJbU6JFFIt0X6pVIVbdM9pUc3Gam9MoiioVcK6bI3OV+6z+7VLhJXQPDw+PVQL/Qvfw8PBYJVh2k0uSox9dPU4AOHeOVKAoJ2p5gdWnCkdZulS1AHD99eQb/p5rbozbZo4TuXN0n5gdXmdzQ6pGalT/hJAahR5Sh6/fIlGhwzPkp378JamYPjJIqnGO1dXTRyWZ1+gJ2l63VRIy1ZjgbSlCzrCppcJzrqm0uGk+LlLqcI1TeOZSosLOxuce+u14u8hqe01FoXVSjZ05JeBztsXaxf/Qybns7KY5ariu/blhLV2r4aG+uG109NCc8Tif3Hg8mhBmsq6oCKW9HHfwyj6JQF2/kdLKbr+eqtbvuP7WeN/GbRSt2bdGkmIFiyL1Olwz1eRSLp89RH7l3/qS+Na/+DStmUxTyOoZS6aWKBRVvWcz+XgPtOjedk/IPZt6nJLIhZFKUlelcU81pI9TZbrfZ1+kv9s23xbv27mWngnbIQ2yvgaNMpkGyxfIbNR7rdyzpSKvkmK5SE4dPxLxxcxmZF1v2LgFANC/lkwvUyVJWzt5geNGrJgm8jkyJeWyuh4orR9HQhZVVHmBaxi/9c7b47YyO2ZMqTiM6QaZIRMhjS2lUuu6dZFStU1zhTz/FbNvmhOBuVTR5RkxyZlwronGLmP6XA8PDw+PqwzLLqGHnPA+q6pwZzkCb1JJIefG6OuZZQHmgiIjK1xrMNy2I247Nk2/LSXkqzeUo6/h+SmSZLq7xB2sZ4ak5P5JkQgKOa5jqlyt3nrPvQCAE6+Qq93BY5ITYopJxnMnZWzdAxSx2lSpb6fYZa9cpy99QkXKhYYLXOjK8FwRvtWYn4AaPS3nDOJoSfW97lBU1ElNLkeGdjmU36o2V5ygkyjPf1pKAkuxpD1dVLk3Gq3Zp4olk7iOaVuAJrsGhnNlj7JyBT30GpG+Rw7Q3x8/KoUl1m8gjWnXjSKVXbub3FpFntJz6tAUp+CVcZw6QhrZlz/3j/TvF2TtZBMk4VabEhVaz7AEzXU7AWAPu9d2naL1XTghUqJhV9pmc66WdGhM7veLJ0kbmHTk7zpZT7feezefXOUXSvDaUrl7Zpj0q07TA6ZraEaXXE2Uu9cpj909VmS04e1kWu6CSyfrHAfecpvcs9fYvVUXuOjm/CgZJS1X2NXxpVeIDM3mhCS+Zhtp4MWKyqHCbpPNQJ65PKfHLhTobxCIdF1kzWNgjbjNTnJq36RKkesWeZPHGyk3x+bk1Jy5mCVeZw0voXt4eHisEiymwMVGAH8LYB0oquJha+1fGGP6AXwJwBYARwH8krV2Yr5+5sM0fx0LXWJHy+VIWu9T2clKJa74PUl2sT6obHAlsrk3x8Qd7AdPcs4G1XZNN31Zp9ltcVIFDK0N6ZzT4uePfEh2sc294vKVOUOSUViZ5L9in3vym9+ifsvS9oGf/1cAgIoOsmi4sXMuC5XJrRbS2AItXbOkVrHzZ2NLq8x5UcsFguiCFXPh9ocd8p90EIg75klxWRBjaV/JCBemaZ7TRbF/u3HoTIatBSSTDiU6pOyd1ihYG3A27ml1I6f3k6R2YL+4rz03TPb0T/zbB+ee0wlNkRYxucSeOq7IGTrLZS5pqEz/tQqtyVIo6++GOykY7eZhcY/LHiE3vyeuAAAfT0lEQVRJLXmWxpudkXlWXVm9LZJLpd5Fa+XA8X+O207N0DkyzEftf2JvvO+ZG8mGfk/3mrgttZl5q0ieobETtK5bFRpHqk+kZidhau1kMaUubKvTfdW/ZA24W7Icjk2QpNs4SOfctUu07qGBdQCA0TPixplyaoAVzdCVgxtjm3tfr/R/fozu2ZRaH5YL6WRUQY58lp55l7FRF8ko8f1eMyT35QznNNKapFvXzkXWqmI4Tc5cGmmX5QVcaReLxUjoTQC/b63dDeAuAL9ljLkOwKcAPGat3QHgMf63h4eHh8cy4aIvdGvtGWvts7w9DWA/gBEA9wP4PB/2eQAfebMG6eHh4eFxcVwSKWqM2QLgFgBPAVhrrT0D0EvfGDO0lAG4lLPnLyhbR43UkMH1YnIZGiDzx9Qk537ZJKRNapj27TsoaV13DpJaeTa1KW57fC+RKnfcTq5tu7eL6hu1SBXryYhqdZZzi+hcEIVjnHuDuY/N65WL4llS8ZrTYhpplGm7qb6dLk1mIkXj1jlaSpxOWPEzSLMJpZleKOercksL5jQh6PDtjpjAdOlAtcuhIzQjpco6k0u9oXN6tPcZqpSsk1N0T7VaaRKOiNUErNugPwtFHwJAhvNgNFRdyCg2/VAnCRWR6Pp37mMAUFS5ZGbDXbf2qblOpI8t22ht/cKv3Q8AePLbP4n3Hdz/PADg1k2Sm2VXgaIvsy+LZTI3yuQcX4/JUPqvrOPCFlvEFDCTJiJ47bukYEXPLlqD4+zuO/qaFGl5/u8pwrZxQdbwzrvpt5tukpS6jaP0m16+72n1HMRTnj9QuSMyWRl3vcYR27r4CxuwMjkx79SYFD3wOrm31lRiJMvvBb2WpW6nmI9mKjVua/C/ZZ0cPemivmWN9XPOF+0t26jT+hjleq09ymzj8sZ096iI3AHqQxdOSaecyyMTtmrNu3rCOj9OtFDRlUVi0aSoMaYA4O8B/J61tnix49XvHjTG7DXG7NX5gz08PDw83lgsSkI3xiRBL/MvWGu/ys3njDHrWTpfD2C002+ttQ8DeBgAhoeH54he4+eJQCxXFKkxSt+LKZVVbUsfkRRVFrfMoOSa2DFEkvyJJyTQ5NbNdPx4XspPffc5yv/y0gmSkMpGXIyKE/TlHigIOZuokoR5/NjJuG1ngb7UIzdQxfSCkmQyo9RvSiXbP3uGCwYo8SbJfSS4cEWkAoscdxq15gYi2db8EroSrhG50l6BJkrdpdeSvMv6SD9OK6nWtYVKTnV3SBeHcBKxE+R1JjwhZedK9FZL/rO22mp1uH+oCboSdTZSJNOs4CTdv3b7c2jV63Pa4nN2aHNn0m6ZSNJcN+8igjXX9Y541/ZnSXoLX5e1M3CINLiUUg6ciJPdRut55K7r4n2NDbSG63mZZzZDa2H7jbvjtuOvUbbMQpYk3cf/51fifaUT9Fju/75oD8efJXJ4aKdoqDmWFLfcTCSqLv3mqGCzKCpU8K577om3R7m0o8uDAgAzJecuKPNr8PqZnqYrc/KkaBu9abqmfb3yjLrSb9Oq3yY/ay6wJ60Cl7o5T5TOn5Rk1+YNw6L1z3CQUXWGi5Ko8pbOEaCgnDYGOAdNtSqagnE5c/hvS2u2bh2pxZZIXL4X+UUldENPyF8B2G+t/TO16+sAHuDtBwA8Mvu3Hh4eHh5XDov5JLwNwK8BeNEY4xJm/0cAfwLgy8aYTwI4DuAX35whenh4eHgsBhd9oVtrn8T8bqfvvdwB5HJM/CRELZrh2owvHpK0tQfrRwEA/UkiOiYSMqTuHKlP77xLiKK9+yhisBEJ2fo+rs348ouUI+NHj0tBh8Cp7JGoYmGC1Kdeda7zTMxMvE79rw8lwrVZoqjA0eNSh7NngFTpFw6J6r11N6nVu/aQOUjXCm04pUnpTnUmeQIz/+1Ka/MKk5wNRawadq5uapNIy53KkaLK5MJ9GCP9lpmAssq/3dlQbBx1qna58+g0tLw/1MUmnKkl/tvBHKNWoCsmoP12Hbfk8mv09Yvf9dAIkZcjnNMFAHKp+YsIuBwj2vbj6sTWdASv4XqTTPR1l2WtrSvxPZuQMWa5bu0FVXQ2wzVq9/zC+wEAhd0qhWs8RFWnlVX6yVEpVHL6LPlA9w3SnO/6yPvifU99+wkaz7A4B+S5GMpTP/xR3DbExN3N99NvWyrC2hVkuTSDC7BmUCIpk5zOOKNy8hw9Ts/EybOSCvv4Gd7mhZJSz15kuHhJU0UnVOjZaWSVqYPzugwMkgllQEW9xmS/WsPD68lktm5oXdw2ygU+XNGSal3I2RLfg1xGFcQoONJUYlBchG0cF4K5PucNZW4NLvkKz4WPFPXw8PBYJVj2XC6WM6cl0kIwZFhasMpNL8WuR1V2TzrTkNwNr54maWXdJpGWMyyeHn9WMuDZ80ScrK2SlFBrCdHRYOm3XJMvcZnzPfT0ictSncmUiXHqY9MGKe3Vy0nrZ4oiPVUnaPvA8/vith/98PsAgBtuoTweP/+LH5VxryVpoalzb7AwGxfL6ICEygZXYkKpqaTlNBeUSKhCAJk8kUzdrCWVpkV6anAkW0blsUlwRGsrqSNQWTJ3eUSUa6AjJrNZlSWStQA3RgCxO1ySc4tEOmKOT5VKiuyRS1N/A+tFml23lUjq7dfTNd18zfZ43xp2Lc33SgbBBrstntn7A8xGi8et3RzB20kltTfHuY9nDgAAii8cjveZcyStNxTZXx+h673uHaJJbuFcK9lhijqsKF/TwOU6UeMImDjrGRDpd8uWLQCAs68fpXkOy763/eufAwAcPyhjG9hI1+2XbxO3xaPf/jGNu4+eoboS9ULrCPL5M3Z2wvMvS2Tu6Dg5DJxRmsXhY5QRtVgTlti50vZw9sJARV42I3o2q1WlwUUkJTeV22KOyUoXIdqtyEu3jLI5aRvmZzihks+4/EItft4LPB4AKMUas4zNPRttcbBM5LviF1rjq7K7dpuLbniJfqEd4CV0Dw8Pj1UC/0L38PDwWCVYdpNLnUnIRl2iK51/cVXpL07tCwJSCZtKFTsxQ+r7i4fFZJBkgurmfuW/PEEqW4X9v2+74fp43wuc0OfZI+L3mmMVLK/qX25ZTwGx+TQn5lGka+86UvGClpgpsiHNb8dGVeThuaMAgAOvEilbKd8b7wuaRGwVZ1TRC8MVyBeIoHzw038cb7/63DMAgOmixH919dD5t193Q9w2sJZSiWbYf/nU8aPxvh9882sAgP3PiG+/S23alt6Wtd8k+/pu3iWmjpvvInPCWhUtmeE0yQdeFEL61ecpmVSR1fJ0VtTb4a3UX3lazGPbdpOv9FvuvT9uy3XT/LJMdhrlt244+ralzFiajJoNR+Jq9dm6IgUHTsRtY89S2trqIfL1Ds9LSlaXLjl74664bf0H3wIAGLhRTEVpjntosakl1RbdyyNQmrgj2gIV/7B+M/V3nlNKV2ZkHKkCXe8de7bFbc+/SOtjY1KCuzNscgx4zesXQ+AsHPqCLIK/++4PfxxvT0yRuVNf9TqbRRuKSQ/ZJtJkE15KrQUXUZ1MqnqqTFb2KnPFnt1ENPewia1Wk3dLgQtQ9A1KYq0C1yKuluR5SaSov94cpdguKhNhnEpZOyIwyaodHOou/TeT5rWy3JeJcVrPRjkiZIPLl6+9hO7h4eGxSrD8Ejozd4Fi8Jruy62krIiH6iSZalO5FxqSQl6flHS4u1liu3NYIkq78+QSdf4cuRYllcSW4jS4FRWFhhRFlYXKZemVIyQFJZlcLFZFQndRblu5jBYAdLPUWZqWSLZurkru0naOn5Yg2wpXhq80Rcrnw6GaIHIA4aO/8Yl4u176ZfqrJBNXbSBSYZ6tupM06Nq+5c474n3veOfbAQD/8oPH47ZvfOXLAIDzZ2S8t9x5FwDgtre/FQCweceeeN8adgcLlYbjbum9H7wvbjvLbndjY+T2aUJV2quPJKlTRw7Ebb1ryL2sS7kmtvjaR7wuAlUMJOJIxzaJaqF8MXyN7LQQbec4uvL8Uy/FbcmzJNGZKZK8qirF6sjPkHay9cPvittSrKU1jXJVc2X3FsgfEym5y7K0Z7VkN0Ca4ZrNdL33P/tKvK/lIn67FdHMz9fjX5AiIJuy5Np3O+eSaXsxuLHpcORFSOhFlYelxfe0zYWVb0d7niE6f4VdlwNV/MWRnMODUpjGpftdNyJumddx8RIX0TmtJG/DFzpQc6mzi2JxWlwOs+z66KKMSyUhbkMm73W+oNgNUqUMDlirc8/hjEojXa2SxB8k5KEOvITu4eHh4eHgX+geHh4eqwTLbnKpcV3IRFIRIxyd2FLpJOusijXYb92qXJcXWLVLJiRy6/kiTW1KVfwe2kT+yFGek3M980K8b/sMkRTljJxziqPlioGojlVWSafZhHFWVZhJWTpnqSr+4qc5Xefxw1IDsspq3JoRIqVeeEp81MtpSvG768a74raBXWQ2mmoJqTLb5DJVFJXQ1e3MKHUuYmKwVtHkDh8f0nFRQ3odZN/tD338gbjtlrvIfDAxLulft+yk6MsuJpYaqop5hYlpo6o1pdlfPquqtA8PEVHrIkQbTeljcobU1Y3rhFS21lVkmuvzHvG5AqvMFKwOG0W+lSyZU4RqFaTZd/zF70hVoNKTlJq5qyx9NDjBV7iR7uO2D0tyrvXvvAUAkOoVUs8xyEnNcs6y/Ng2UwYnolPHO9OMztMWcjrX/g20TupPSsWibq6nObRZ4iWaozTu42WphzvJEZk2dP7UStaL8wlfWiRjlJi9SgHbbAtf5kadVpbOlXdmSVX3tHcNXefuNRL52cf+4aEa24VxeuZ7+slEk1E+5xH7socJMeW0Wq4Clsy5xhHhU1z7s1IV82WOaxG3VP2qGj/zY6Ni9u0p0HPV4gpHNVXJLAzp2ugKR3o9LxVeQvfw8PBYJVh2Cd0JTToHSNOJjkoKFymLG3TeEa7QPVGRr/9Ek9rOnJF8MGtP0de2j930rJISMUQEaM+UkJzZGknEXapuaIHPUTYk6a5XeTaKLFGViyL3jXM18DAQUaybXdUSLE1OKvfC7msoei/Xo1J58jlr6iIp6hYAMDUjUq3TelIJOWcIamtzD3U5OmKCSCXbL3LNQyU1mDxJyT0FiUScqtA5pmdIam9hbjSrlp7cLZqqSb/JGY68C5mwUhF7TmjKZufKHrZNCqe/iZCujHYfS/B6yqpo05kstQnVKkj3kVQ7cr24HJ48w/ObkLWwZhe5Aq5/x60AgNxOkYKRZQmwLe3N5efqABO7ukJ8wBptvockx1ClcD24j7TQbXdK7ryxM7TGJ6fleVkzQustlZOoaAfr0jG3yX8Xn0uont9mnK5WrbGWK0Aha7Kb53DjTaThpLOiZToXxWnlzJDl+21qsu7OcBrrJpP9SXXfU+zeOK7uYyZDY4sUkT5+gQjMOl/LbEa9K6wroiKvT1cX+Pw5cRjImH7ul4vc1ETDbvB1SKc0KepzuXh4eHh4MJZdQm+5gCGdpc+4zH3K9cwZDfm4prLLJgxLY2lxZ6ryV3c6KxL32AkKCsnNkNTeWxApLs+Z6rZvlACMJhcTOHVc7N+Fs/T1P8cVxUsVkcadpDRTkS9tke2sdSXp1lmynOBAg923S2Xz7TeT66DJSHmrSS5pZ+eaJGMYZWsMmI+wKoyjxuXaImX3c+M18V/5vjvTnlFSsMvU6CQrQLJUWs4z04TO5jhXsjMs7bW0K56p875OM+PiCqZN1OW/qonH4ZLz6WyOCZZ8Mim5Rg1VJnA2gh7S7obfc1vc1ruHgqMaJeEgutklNmB3xVC7rLkslIuUyjtJ751L8Rn1fz4XS+25HtIyk0rqO7Cfyii+fUbme/oMuYeORSLJD3CRmLzKTCindGu3k5Y0v/tnqNZkGPL6aGgOgsakubLt2+lZ6OagIO1yePgoZTE9eFRs/7feejsA4PrtkklzgLMmdnVzkZsJlT/mOP02qYpeuGC3UL1Tknw/0inStFLq+LEJGlOQFKm9t5ffPepyzLCrY8DXuVaR6x0xl5RQmmROleJbKryE7uHh4bFK4F/oHh4eHqsEFzW5GGMyAJ4AkObjv2Kt/SNjzFYAXwTQD+BZAL9mrZ0/v+s8iDUw5c0UhqTS61qQtTqnVmXVuqUKUWRYu87kRAUqDFHU3JRyC7rAboUNdkGqc0EKAKgcI9W0OykueVuvI4Jy3RapS5reSSrY5EtU1f3s00K6ptn006rJ2KaZDKooVazQRar6NXe8DQCw/aZ3xvuiBKlu5ZqoZ8mIztlszu/WlNCECmtxRheiYK1WR0smWe2TGqjKDBLXClVFHuDS26qo3lb7vUqogERxrdMucFzHVLniNfkcUcyQy+GWf6unEsWV47W6z6afWG1WxKpL/6rm3igKQTUbLjuwzp2TX0sRq0kJPI6XbNOlEFZEfci/fbMkJl3f05nKAi64cO27xeX1AtfcTCgTVIWv31RG1umWO8lkURggU5++3phr4VrcGAMxJwTsJthoiXttnU1U6zZKlGeCHRVef53qpHarGr+njlAK4JYq0pJ2kbPT8txmzM0AgOIk/fb0KTGZzpTJMaKnR8jfTJLurXYhdCSuIzsrKg+LW/+q9ga62H1yUBX1mL5ALoxlrk9aa6oU0JxTZg2nywaAnkGJfF4qFrPeagDusdbeBOBmAPcZY+4C8KcA/txauwPABIBPXvZoPDw8PDyWjMWUoLMAHLOY5P8sgHsA/Aq3fx7AfwLw2UsdgPvYqvQWsQ9aUxFhIR8npc5UoQH+UtYUueJIsqBLAlIyIySl1Kc54KVXvqblFH3Ni0WRuMdfJhI1NSU5GCwTHCeLpwEAeYgkvZZJt5SSpM9zIEVUkPJWqREiflK9JJlERsgQp5To5PlOqrUd69ETdFGImIxUbeLxKGJFwJKr4+M6fd111TunBei6D/Vmu/SrC1xE1pWx06XtEm1zov5cMQEmXZUG4EjzNhK1xVK7qtzucoREjuBV8wwjl79Dxq1LrM0B/zSp7wH3oRSQ2B3TZQLV6zVy13SBlDGXhbZ+OdCKz7/5zlviPeu2UbZKqyTjG+6mbJW5XlE33vvRe6gnibdRp1qaO50met3zqvvKFUgbGByScZw6Tc9VjonmsfNCis7wc9vfIw4DU6OUB2jflGjbY2dJIh/sJ4K3UBDNvbvHFcQQiTtgrS6TlOfQ8lp3hVs0yV4oEIEcJkV7cAVm9Nqd5vHWqtS/UXlp3GFN9c5qLbAkF4tFaYTGmJALRI8CeBTAIQCTVvTbkwBG5vntg8aYvcaYveVyudMhHh4eHh5vABb1QrfWtqy1NwPYAOAOALs7HTbPbx+21t5urb39jXDL8fDw8PDojEvyQ7fWThpjvg/gLgC9xpgES+kbAJxeygDqroanZtMil7dF5XdpOH9krtqtiTNX5b7DJ6WhKoSnk0Q4NvOkMs1EEi1WKrCKldFmGJpSKxJSJeEKLaSor6GC9N/r9Ki0jLvA47V94t++bgOlmO0ukKoZhuI37KqLB4olbjLVHCXnV33rKodK0yXW1yYJl1ZW+QZX+NpU2DRjlEEhjCu9K1LURQzqYTA5Hfsjq51h4EwuyswTzp2DSxvq6obaSJtt2JSixhHvtionCpwvvfPT1guE+k2oyMUU8/dSb15NyW2ooQZxv3PhLC1J22HvZQT/LT6ytH1sSVWNPjtC5gYXrQgA9/3Sh3hLTAD53tm5RTQZzvel7fm6NFuSi0ZOqtq33d1kOilOCaFZYjNFmvOwjI1K5GWSa8nm8jK/PBeiyGTkGSpz/qYy70uGisTP0vmDrFzbGa4vm1BEvSM54zgCI/vyabqmVeW04UyIDZWrxhWvSKXpOpdUNKvhlLoVlVspxePu6pd4mkvFRSV0Y8waY0wvb2cBvA/AfgDfA+CqGz8A4JElj8LDw8PD47KxGAl9PYDPGxKNAwBfttZ+wxjzCoAvGmP+GMA+AH+1lAE4UjSK5ItZZ4lRZ5RzkaIuI58mCF2RDC2wOWmyFmnSgaV7di9sZlSi/Bbta7SkzXJei4oREqZ1gUrUpSyVb2vVRMrvZglzsFsy7PWeJvIlCsQ9qWuQyNDMWiJtqqGMscnj0JkBXfZELaXOTrcYqeObLOVr0tDydQjURWoZlsw5TFcHY0rQoypIEBPSqgr9LEkt0ER26HLFyH4Tk2O60UmYbhxKQocjI/Xg5ro3xiSoi3BVbGTgIh2VBG1VRsfZmCujzt138cYrD3cNUpqM5OsQpmVN9gy5bXVvFxK4O2ke7pwLaBHt+7icY17GkWaJ+6ySwtMsabuMnrqwRB9HwmrTbW8XR3kqt91mgxwVLOa6t7rIzFRKtBM396rKKZNx2gBrEbrISIvnUlNrqM4vspKSuF3hDBeRrTnPMkvoukxkyKTp2suQ0Bfj5fICgFs6tB8G2dM9PDw8PK4CXCWyhYeHh4fH5WLZk3MZDhVtWjE7JDjCLKkjEjn9q+0QtmbiaEJdhZv+NpQpx9Ujdbm8tAkjGZDqVuA6ogCQSZNZpar8dYtjpB6OXSD/13Mq8c84p3/tyYo61+wlk0tfQaqM9w6SypjgEFdtfnDFD9qU1RQnCrLzO6omVRRkM3YsV/7fvK1rNBpOsm/dddFRoW5bqc2OaGwLIlxgTK7OoqtZSt21k5fcMY3NkXvab5314baFylNtqwsaD5eTvelsb7P2zT7HTwcWaV651F4X6KyNDOftfKGgjuBiFrnZyaCBcnVu+uYa10I1KqFVmiNLNeFtLPud8zNRV77eDTbXub4AIMcFMAJF2EbG1XNlx4G6EJrTZRpbMiPmI8ePatfsmQo9+0mOktU1WRNs8mkzHVfnTxi3WPy0rWoPDw+PVQuz0Bf2jcbw8LB98MEHr9j5PDw8PFYDHnrooWestbdf7DgvoXt4eHisEvgXuoeHh8cqgX+he3h4eKwS+Be6h4eHxyrBFSVFjTHnAcwAGLtiJ31zMIiVPYeVPn5g5c9hpY8fWPlzWEnj32ytvWgFjCv6QgcAY8zexbC1VzNW+hxW+viBlT+HlT5+YOXPYaWPvxO8ycXDw8NjlcC/0D08PDxWCZbjhf7wMpzzjcZKn8NKHz+w8uew0scPrPw5rPTxz8EVt6F7eHh4eLw58CYXDw8Pj1WCK/pCN8bcZ4x5zRhz0BjzqSt57qXAGLPRGPM9Y8x+Y8zLxpjf5fZ+Y8yjxpjX+W/fco91IXCR733GmG/wv7caY57i8X/JmE613q8eGGN6jTFfMca8yvfi7hV4D/49r6GXjDF/Z4zJXM33wRjz18aYUWPMS6qt4zU3hP/Gz/ULxphbl2/kgnnm8J95Hb1gjPmaq8bG+/6Q5/CaMeZnlmfUl4cr9kLnikefAfABANcB+Lgx5rordf4logng9621u0F1VH+Lx/wpAI9Za3cAeIz/fTXjd0FlAx3+FMCf8/gnAHxyWUa1ePwFgH+y1l4L4CbQXFbMPTDGjAD4HQC3W2v3gBIAfwxX9334GwD3zWqb75p/AMAO/u9BAJ+9QmO8GP4Gc+fwKIA91tobARwA8IcAwM/1xwBcz7/578aYECsMV1JCvwPAQWvtYWttHcAXAdx/Bc9/ybDWnrHWPsvb06AXyQho3J/nwz4P4CPLM8KLwxizAcDPAvhL/rcBcA+Ar/AhV/v4uwG8E1zi0Fpbt9ZOYgXdA0YCQNYYkwCQA3AGV/F9sNY+AWB8VvN81/x+AH9rCT8BFZBfj2VGpzlYa7/Dhe0B4CegAvcAzeGL1tqatfYIgINYgRXZruQLfQTACfXvk9y2ImCM2QIqxfcUgLXW2jMAvfQBDC3fyC6K/wrgP0BKGg4AmFSL+mq/D9sAnAfwv9hs9JfGmDxW0D2w1p4C8F8AHAe9yKcAPIOVdR+A+a/5Sn22PwHg//H2Sp1DG67kC71TNdkV4WJjjCkA+HsAv2etLV7s+KsFxpgPARi11j6jmzscejXfhwSAWwF81lp7Cyh1xFVrXukEtjXfD2ArgGEAeZCZYjau5vuwEFbamoIx5tMgk+oXXFOHw67qOXTClXyhnwSwUf17A4DTV/D8S4IxJgl6mX/BWvtVbj7nVEr+Ozrf75cZbwPwYWPMUZCJ6x6QxN7Lqj9w9d+HkwBOWmuf4n9/BfSCXyn3AADeB+CItfa8tbYB4KsA3oqVdR+A+a/5inq2jTEPAPgQgF+14re9ouYwH67kC/1pADuY2U+BCIivX8HzXzLY3vxXAPZba/9M7fo6gAd4+wEAj1zpsS0G1to/tNZusNZuAV3vx621vwrgewA+yoddteMHAGvtWQAnjDG7uOm9AF7BCrkHjOMA7jLG5HhNuTmsmPvAmO+afx3Ar7O3y10Appxp5mqDMeY+AH8A4MPW2rLa9XUAHzPGpI0xW0EE778sxxgvC9baK/YfgA+CmOVDAD59Jc+9xPG+HaR2vQDgOf7vgyA79GMAXue//cs91kXM5d0AvsHb20CL9SCA/wsgvdzju8jYbwawl+/DPwDoW2n3AMBDAF4F8BKA/w0gfTXfBwB/B7L3N0DS6yfnu+Ygc8Vn+Ll+EeTNc7XO4SDIVu6e5/+hjv80z+E1AB9Y7vEv5T8fKerh4eGxSuAjRT08PDxWCfwL3cPDw2OVwL/QPTw8PFYJ/Avdw8PDY5XAv9A9PDw8Vgn8C93Dw8NjlcC/0D08PDxWCfwL3cPDw2OV4P8DDB0i5Cpe5VYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# 展示图像的函数\n",
    "\n",
    "\n",
    "def imshow(img):\n",
    "    img = img / 2 + 0.5     # unnormalize\n",
    "    npimg = img.numpy()\n",
    "    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n",
    "\n",
    "\n",
    "# 获取随机数据\n",
    "dataiter = iter(trainloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# 展示图像\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "# 显示图像标签\n",
    "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "2. 定义一个卷积神经网络\n",
    "-------------------------------\n",
    "从之前的神经网络一节复制神经网络代码，并修改为输入3通道图像。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "\n",
    "class Net(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(3, 6, 5)\n",
    "        self.pool = nn.MaxPool2d(2, 2)\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5)\n",
    "        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n",
    "        self.fc2 = nn.Linear(120, 84)\n",
    "        self.fc3 = nn.Linear(84, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = self.pool(F.relu(self.conv1(x)))\n",
    "        x = self.pool(F.relu(self.conv2(x)))\n",
    "        x = x.view(-1, 16 * 5 * 5)\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "\n",
    "net = Net()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "3. 定义损失函数和优化器\n",
    "----------------------------------------\n",
    "\n",
    "我们使用交叉熵作为损失函数，使用带动量的随机梯度下降。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "4. 训练网路\n",
    "--------------------------------\n",
    "有趣的时刻开始了。\n",
    "我们只需在数据迭代器上循环，将数据输入给网络，并优化。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1,  2000] loss: 2.180\n",
      "[1,  4000] loss: 1.823\n",
      "[1,  6000] loss: 1.690\n",
      "[1,  8000] loss: 1.569\n",
      "[1, 10000] loss: 1.521\n",
      "[1, 12000] loss: 1.481\n",
      "[2,  2000] loss: 1.402\n",
      "[2,  4000] loss: 1.367\n",
      "[2,  6000] loss: 1.356\n",
      "[2,  8000] loss: 1.345\n",
      "[2, 10000] loss: 1.299\n",
      "[2, 12000] loss: 1.306\n",
      "Finished Training\n"
     ]
    }
   ],
   "source": [
    "for epoch in range(2):  # 多批次循环\n",
    "\n",
    "    running_loss = 0.0\n",
    "    for i, data in enumerate(trainloader, 0):\n",
    "        # 获取输入\n",
    "        inputs, labels = data\n",
    "\n",
    "        # 梯度置0\n",
    "        optimizer.zero_grad()\n",
    "\n",
    "        # 正向传播，反向传播，优化\n",
    "        outputs = net(inputs)\n",
    "        loss = criterion(outputs, labels)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        # 打印状态信息\n",
    "        running_loss += loss.item()\n",
    "        if i % 2000 == 1999:    # 每2000批次打印一次\n",
    "            print('[%d, %5d] loss: %.3f' %\n",
    "                  (epoch + 1, i + 1, running_loss / 2000))\n",
    "            running_loss = 0.0\n",
    "\n",
    "print('Finished Training')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "5. 在测试集上测试网络\n",
    "-------------------------------------\n",
    "\n",
    "我们在整个训练集上进行了2次训练，但是我们需要检查网络是否从数据集中学习到有用的东西。\n",
    "通过预测神经网络输出的类别标签与实际情况标签进行对比来进行检测。\n",
    "如果预测正确，我们把该样本添加到正确预测列表。\n",
    "第一步，显示测试集中的图片并熟悉图片内容。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GroundTruth:    cat  ship  ship plane\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAB6CAYAAACvHqiXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztfWmMHdl13ner6u2vX+/d7ObOITm7NDMajSRblmXJTkayLRmJ7Mgx7EGiYIDAQuzAQCzHPxwB+WEjgR0HcBQMLFmyY1hWJNlSZMWRPFq9jDScVZrhcBmuTTa72Xv321/VzY9zbp3TG9lkU2x2+34A0cVb9aruvXWr6pzzncVYa+Hh4eHhsf0RbHUHPDw8PDxuDfwL3cPDw2OHwL/QPTw8PHYI/Avdw8PDY4fAv9A9PDw8dgj8C93Dw8Njh8C/0D08PDx2CDb1QjfGPG6MOWGMOW2M+cit6pSHh4eHx43D3GxgkTEmBHASwE8AGAPwLICft9a+euu65+Hh4eGxUUSb+O1jAE5ba88AgDHm0wDeD2DdF3qxWLQ9PT2buKSHh4fHPz6Mj49PWWsHr3fcZl7ouwFcVP8fA/CWa/2gp6cHTz755CYu6eHh4fGPDx/96EfPb+S4zdjQzRptq+w3xpgnjTHHjDHHarXaJi7n4eHh4XEtbOaFPgZgr/r/HgCXVx5krX3KWvuotfbRYrG4ict5eHh4eFwLm3mhPwvgiDHmoDEmC+CDAL54a7rl4eHh4XGjuGkburW2Y4z5MID/ByAE8Alr7Ss3ep79818AABibpG3ZDHXLBPK9abWaAIBO3KZjstl0X5zQb20iFh8TxACAIFR9bpdoH2hfJttI94Vw15RzxEkHANDuSN+ShC1NJuL+iOWpyfu0LSrhcRkjra0WjSGOo1VjD7hvrUTaqtQN1Fpx2la67wlofPjDH063O53OqmveCtzw+eyKv7op0G3UGrhGbbgzbv4SdbybZznJtby11uq3O/5jH/vYqn37f5TnNu6kbdNXrwAAmg1ZM4fuOgwA6OmuAAAyofQnm6GFl9VtvJ4jo9ZYpw4AKJcyfA7pa8TboVrEs7MzAICurq60LZPJ8HnpOBPIOTpJCwAQrCG6BUYaa1Uyh0YRrcl8Pp/ua7XoHB1+BgGgkC/wtaRvv/+7v7Ps/Hv2DqXb5YGj9LtQnttKVxkAsNiUdV1dmOb+0v1O1GKIeBCFKJe25UN+hannNn0AuSlO5PyuLVFt7hpu7HR9nss11o7h+2cC/V6I1ziOfpvLUX+zgfQblrZNVuavNn0cAPD1Z76/6lwbxWZIUVhrvwzgy5s5h4eHh4fHrcGmXui3Ai2WsqytSyNLpzmU0qYA9CWLIpa8tcTBX12TkcamkyoS+QJGLAGG3BSpc5iEpGZ0RApx0nKiztEyJLnEIX1hW3pfHPC55GttWMrPq75FLBkFEXU8brdVRzo8JDmHk0jDcH0LWRiG6+67VbhZiV/PRypHKSkycSKV5TFY2ec0JgORhuQsm5fQ10K5SPc2sPJ4NKvUlrSE2M9n6bylAh0Xqcu4tZNTi6yQ5fuuxtKM3XG0rrJqnbgpiiK5t07yD5SU7+Ymx1qrXibVWpuvKXDarYWcN+CLZVhKdVI/ALSbTR6fGgtLnbjGmkisSPmdsJfOlZFnOg5JQg8ySkKvL1Hf4ir3Q87XtHRcW0nGDZ5fJbSj1SYtKuBnol6Td4t7TvT4nMYcBPIcWqfZ8GRqi0CnE/Mxck1j3PtJ1kxvL405V+ji88s9S9y6zkk/4qUyNgsf+u/h4eGxQ+Bf6B4eHh47BFtucrFskoAVU4dlMsrEohImbVKBwgKbNZTa6qwNmpjIskrVsaLSJO1w2XFOdQIAY1cQcwAMEzg2FNWxHpNud2Wa1LNqS9SopSVqC62ctyvP5Jgi9SpFIpQKORpnErTSfUFqXpGxuxG0k/XNBNqE8IOqE7uR8y4zb7jjl+mmbpc2EdGcN9s0H5HWs2P6bWjWunayRtvGcK2xRGz2CpTZKxvStTKBtOUCNqe5fYrQbNbJNBOGisCL6L63m0KsBmATW4farJFHMmbTUjZTkOPdPKg15sjhmM2GOt5j+upVAMDwQK8cz+aVMCvXCvlabp6V5QcRH99UJLEjbNttaVuJwMq+mPsbq+cgNjTmfJf0o3//MP12fhYAUK4tpftaDXpHxGV5HpNuijzvysrcu+sGbJdtNeX5cg4U+bzcl3RK1Zpw69j9DZSNt8NjTvTy48tnI1m7hQITx3BmQzHpJM6cq2XqW+DE4CV0Dw8Pjx2CLZfQo5gl81C+jgFLGrlQff0d48RfykAzP/zTjpZgHcmTFelm14G7AQALc1MAgKlpkWQyEUnjAeTL3erQ9NStBEQdP08Sj831AwDaoZA8LZYcluZn0rZLEyxp5JXkNT4HANi3i67Z36WlOOfKKGN3wkdsV7tGOWjJ+Fa4K94SKT/tt9Ie2LWzo8SbNmtKp86cAQAM7xJ3t4TJ7cE+kTDzTCQlm+jjteYoy1J40hHJLmTpKqMIuQy3BTGto2xGSX0hu8Yq7SsT0L1NjNLIEnbHbTA5qtZTg8deLMoaDh1TqsVDnocqu1Q+99zz6a42awq9lTenbbkcOweoKUhdZ1l7DZS7oLHOOUDWpE0cMbi+hN6BuFYGoLWehIoQZi0tVNpaidnNSpHv8fPPpvtaUyStjzxwt/TtKj1zTSPzVuaBLdaJWM2rseRYYw/6hYAMmBTVr5Rmkc4btVlzactkLZbovuTm59O2aO99AIBaT3falrDWFfM9yydCrKYWgVjawnjz8rWX0D08PDx2CPwL3cPDw2OHYMtNLk4vN5Gk1XXqcEdHUDIB1WI1OKvIpjh26p8ySfA5tF/vW378JwAAz/39PwAALrPpBQCqHRf5KarY+bFJAMDZsUtpW653BACwZ/ggXTMnamWL1cVMWbJcdhqkJk5PSpqbYi+Za8aWKPqwodTn4S5SCYsZUUPjNqnNOhhuJR24Fil6OyJFr22aYfIto6J62ce8viQk+Nw8qcYTU2SqKnSJ+tzPEZE6qtGRgDp6dI3OrujFxpFl855V58i4yY+l3yEceU9tGeXX3XbqdiLnCCs0D8aquAP2d05cNHIs63ppgUxz5aKQgAHPt47ajDiyeo7J0JkFMSUW2E+7pSwjrTZdK8rqNUNtMUdid5S5yUVpZ5WPteU1m8TrmwH1zDsTYqDGHnd4rMrWYdgk0jB03zOJrAUzQKa42qL0rX32JPXXiFkq4emqOv929Xxl2xw/clGR8jwf2tGiwebTsMFzJZdEcxf1sX5FTKtdhp550z0g4+PrtgNHNKvYC57vUJHsUbB5M6eX0D08PDx2CLZcQm8G9CWer6kIMpZuessiVlSYZIpYQtGEVep2pAgaR5rWarNp29e+RHljJuZI4phYku/Z+Ut03PnLkuI9zJO0HoeVtK1UoS9xpkj7orxIBjmWIvOBjGWqRVFqI3v2pW0NJmvOnCEJfWZO5ZTZTec9MCiaQoZd94xyGxP5jMervv42uTGZNA3MXENA0FJ5sIaEHrMUlrA0oqNZXQTe1emFtG2hSmOt6/wdNRpNkCPyuVqXe1suskSq+ubk/Y0qIDeqqeSMc7GT+XZk6JouhwlHJiqXw4g1ykgxj6Gh+bCxvns8PnYEiJVr29IizdsFfc3IRVaLNLm3QvPmXBRfevnldN8b7r8fAJBol8qY5jevXXpZU6jXWAOO5Pwd1hDDSJwD2pwvqNlcPyV2rKT3hNew1TIkOzG0tHsjX7d7kedqcDjdVxjaT/2xQkaCXS/twK60qZ7h3CxXKC8MlAtwlZ9XO9yftmUS6lNDafgl1hJbizS+ps6xU+CI3Krcl6iftAeTUW6ZnK+li38aKg2gY2juTaBcdLH5aG8voXt4eHjsEPgXuoeHh8cOwZabXK7WSc2YaQsp+s2/+wYA4L6jYrr4sfuJbOhlf3VNxrgkPIFSX2ImXxSXhrPnyc95pk6qkC32pfvCMpNvfWIeKHD905ZKmdpiIq7SS32rlKWPk1fIhLIwq8gSVgnzBTHNXJglMjZTIXVyclyqS5WvLAIAdlXk+IJL1ZsoMm0FqjWd3IxVTqVqutTCoUr05LZdOlCVEwtBsvpb76JYta1jic0BjhwtKOKswRF148rkMjlL24kizNpsT6ktEoE8OSXzN3ZpHABw35FDadtdB/ZQ/5VffkrOukhfbWVx3dZhCtegSkM2+SVtMScEbOKrz8tYwOYGy0mdwoKMPcv3Kqvm27TJ1BZrMwVHQ5uUiBVzU7VKpoWJCTm+VCnzNVViMp7z1hIdl1f+8FfniFh9/vtihinl6JqHD8mcRmz6adZo/RUilUiqSWsrVmmkY/eoNdR8rISaYpfCNlkWK8L71LOcYXNX7vQpOv1z3073dd7MpiqVhtZyjEh2UZ6NBmgeyhzvEebk+KRE5zdWEfWcHK+rX95BmUtsrlmiNZkZFucHXKR9UUXMoo2rNL9hUdqSo+Sb3uDEXoEi8bMdmpxI2RLtNTj+jcJL6B4eHh47BNeV0I0xnwDwUwAmrbUPcFsfgD8HcADAOQA/Z62dXe8c1+xAN0kJtWn5trSzRDzO1FTy9xa5EVWy7OaliBQnkYahkDaNFkm4VxX/NLVIX+diDxEivYNCVFYTkjQGoKLymEBpZURqalRJgmks0fH7FblSY2l8siXSsmFpaX5GSWUsrdT56x9mpd8TCzSN4/OiFewfYA3kGl/wuboMtFwkrSFQeSVcsY5lgrcja1wQ7rK0tWt869dwh7wyTi6dfX2k7RTyIvk0GzTmYk7adg2SpmWV+Fat0VhLLMm0GirdKQ96qSnj66R5NpQbXeo+6fatGuYyifFa3pZ5V8BAHeQk9JzSCspMPnczmRWw+yUA5Pge57VAylpU0JC1kBY94EIprQVZa10l2tfbJ5rk2THSAs9cvJK2nTz9NABgdook0qWGnKPWppozEZQbIkv+D959NG17308+DgDYzeu5mZdxNqpV/p1cs8IF6E19EeshE8r6c+mvHTkKSArZSMmV5Vm6VmeM3HwrSttYvEzXb+UlGtOC3gvmymTaVhplQrPCmifkWSqwu2x2TvrdYCK6MzWetmV5DjsLNFe5GXGMaNdZmyqIhjN3lpwpsgWR0LtGiMR1qaCsclFsOjJcreFWsnkRfSMS+icBPL6i7SMAnrbWHgHwNP/fw8PDw2MLcV0J3Vr7LWPMgRXN7wfwTt7+FIBvAPj1m+nA3W94DAAw9syJtK3cTV//x972lrStGJKducUSspY+DWeji63k++gaovrVL758Ss7bQ9Lh7v3kymWVLS7DUnjSnE7bWq1k1bVC/qK+8tJLAICKSlBfLJFkUFJ2tMtXJgAszzMTstTRx+5mc7Ni/5udoe2z4+KaNTpMLllRVkU3rEBUEU0hZum6revvsW0y/Quxa7pgFS2R2jV8GJ0Arzwk0wAXl+8DynW0h12/2m11LpbaimWxSToJ3XCwmFEuYrmCc+9SZdWYGFlmc1zVN7lmZvkhvHt9Ef3iuXPcb5nvxQVad3FbNIVLl0g7meU1UF0Se/JQP0nV5ZIEBYVcnKWlMhRGnGso4FxCVSW9N9xgVKGNC5eJfzk7JjxDtUW/zXez61xJJsatxFJWZLfx8xSMc/nyRNr27W//HQDgXuYqBntEIq0vkeTvysMBQPteyqeyNL++Yp7Lytitk9YTpTKzhhMoN9slDgRcevSNAIBK9KZ0X22R7kFb5X0yOZ4bVZ4xU6DrVtk9U7vbtjlfSkY9G3WeG+00WGe7fm2JrlkqyFgafHyuLM95Xxe9e2L1rljitQt2oyy0VcZG7pP2MG7fgvxJN2tDH7bWjgMA/x26zvEeHh4eHj9g/MBJUWPMk8aYY8aYYzpPs4eHh4fHrcXNui1OGGNGrLXjxpgRAJPrHWitfQrAUwAwOjq6SqcodpOpYP8hIWjqbIHYd/Bw2jbAavvc2XMAgLaOLuuQ6eKxd/xM2rbv0KMAgIMPnkvbnnuBzCS9ZTJhXJ6UXC4RuzHldHEF7u1SVciuuRlSO/vKGX0I9YPNKgODksvFFW2YmhUTiuFoyi52eYxCRYywyv36xbG0bbCX1PIje5Tr1Ap84o//l5yf+5FR6l+5i1TGwweFCH7zG8itypW9tMos5EhGq+0rLseOMqs4wi6bo/NrsjObJRNKf69yn3S1YVWNxjRHSIbO0ejI+eeYJJ5TqUoX58kE0Naumkxk9rPr2ZHDQlhlXDShLgwfLDPALMO3//4ZHq4qsOKI7LqshXNXiLhLa38q8ai3m0wWJUUS5/i4jHJljNilLuCaojVFaEZ8DqvyFl2ZISK9rdjtYpdzt+N8R0vK3ZLvR6Mh/a500Xnf+qYH07Yqp3xusIvuhQtiSnn99ddp7MrF7vw0zX29JueNckLuA0CpJA4GHZ6HdqzvGReaUWSgYRNUYZiIz4WqjOXqPI3dKHfcFtdMzWpycY5+43JB5bLyHCzwGs9n1KvPpTVWkaJNjl4G1wyer8uadGl0iiqatmsPmXhDbQZM6+HyvdK1LNybQy3K5Bb4Ld6shP5FAE/w9hMAvrDpnnh4eHh4bAobcVv8MxABOmCMGQPwWwB+G8BnjDEfAnABwM/ebAfCHBELlyeOp20PvYmS8Ze65YsfLhIBFbOUEKnyWWcuEnHx9t6DcuIiBZ90lVSV9oiuVWA3wXxWlQrnr/Pu0ZG06VWWTLKK3FlgYubgXtIojt5zX7pvZoaLWVQkQOEyu1MZRcL09JJUO8/Sp85/UijSb+uL0u9TFzjYQxFbw5K6go6vqeCnOm1nVJDPIgu4RdUW33sPAKBhmTxSEnqOJSUt1bpCFToLYXcfaSMp8aTcHZ0bVqikcRfppWWRhKWVcxz4dWlSFL6ZadKI6nWR7OImS6Iq54vLKbJnLwVr7du7J91XSteKJn3Xl9BfPEX9KBZEI7KsETY7cl+6OWumI/9aSgq+ukT3IFRz1ZUnjawTCwlumAQM2bfNRBKolquSZNlqC9k6M+PIUF0ujf62OEfMYlXmqsXurHsHxfWxv5cWjwtcAoCZWcoD099D/Xj0jfen+8bYNXW+Lmv4tTG6L4Fa1wcl7QoAIFKZTgtd9MwtqZJyEas0scoyGHHwTcBrMlHuloYL3kTqmm6r3VIZJlnLjljy1hqRI0NjpQW60nYdtSozBSYt49VZW13ul0xHaQrsMaAzNuZjl6GTr6WWnAusW+5FvPnsqBvxcvn5dXa9e9NX9/Dw8PC4ZfCRoh4eHh47BFueyyWTJ4Km0dDqM9dvVBGUxZIjmcgUoOuNliNSmT751MfTtp/+Fx+mc6jotizXUnTFMg4e2p3um5whgquxJGrzriHyW9cFA5pc5/HQYSJs7zosZO78C1TLsbooaqUjdToqQq7OJpEerj8YW4la6+4ldbGjKhKEAY1v7LKYIobfgGX4uX/2z6WPTBaWVP4YR8IUlKnKpZZYWOD8Kh0xBWSYpIuU/61l1bWu/LNtQudzVdE1ERvx8ZmMjkBdbbZx/rcNzn9SUjkyejmfTtySvuVDGtfctJgMxi6dAwAcZiI9DJRpybqK9irF8DVcfhfYrGc18cixBYVQ5mPP3ruo/y5N8BVZa1NsKhoeFo/e3ACZgapz4s+dcCRsdy/ZK3I5iaVo8JBrHTG55Pk5iNuyxkImF13Rl0xWFdrI0/Zjj4gJ5ej+UTp/S9b62ddpXK+feBUA8LY3C2G6dy8df+FlyTnUjl1OpfVrimZVP7JcUzexYuYsMAneUWmKFzlSNmbiM98tpqLhEpvAFHno1rU2V4RwNVPpry7MsRYsP5va5BKzr7tLUxyoa2adoUclimryO0XnjorY5BiD88fooiv83Oi6rtr0erPwErqHh4fHDsGWS+iGI8hqSjJusISZ0XkcptmliPO1ZDCX7hvpoS/mqeMSFXp57DRt1KT02/mxcwCAh3dRdOru/cIsjk6ShFQ9LVJIX46kw64eKSv1+utn6ZqjJN3PLYj01OYv/cRVJYE5skS5JtZYQjec20FTISWXvTGRyM+sofloTV3BekjaIkGkEoraX87SeQt5mdM6Z8qrtakf586ck2syKbrv4P607exFmssv/fXTaVubM1zmOV9LUZ3fRdd1VyTqsKebpKyHHxYVY3CApNK79tCcBspd0ElZjrgChOyqD4n0NjpC92p0N5HaOoNfjV3blmks1xBlMkzUDw6Npm15JqSnpsSdtMpRyy7cr6EiQLsHaW3tVq63Xd00zsqASO3TTKTHLLG1VUU35yJZU0Riq+0IT9FYsi6jZ47uccaKBjXEcz/YK/cgzwTfYK+wmBV27Zu+cAEAcP71c+m+XX20/ucnnknbMkyGt8L1XyGRyl0SchbJvMrvMjdJBO/MkuRQuTpO89vbRev/gftEU8iwdt5UhHCbNQRN6Lv174q+BIqod1KyLp0Yp0SsZi2X5wbSmVyRnkOeuYiP12vX/SbjNCf9oPPpA+WCGV/DlXaj8BK6h4eHxw6Bf6F7eHh47BBsucklTX2r1JeRAVK3tPr+tZfJJ7yXk+wf6RMVKJ9jUigSX+yrk+fo9E2JeNt3F/mph3zeYkUIqIFhIqymZ0S9nWcyVBc2HxoidTlic1BDkZcu6VJdmQc6/OOOOkmjyak5O/Q97VcquOFag1kjY8kxaRTb5ZF4Gn/5f76SbiecsD9QPrxlJpi7lPnjwBEa82A/mRj6RySKtI/7lFfJpeaOkznqe8el7mrdumIa9P9IqcMV/u3hfWK2edtjj9C1SuLjXWK13Wm8LTWnHfatrs2Lia3NftyFovStp4fMDROcDG1KFckocMTi8C6Z52JRxSCsQC+b2EJlTmhyIQ+jZKCZaerTwgKnQVYmwpAjDM9fkgRYlQUyl3R3S5yC8z9vslOAUQRhzkUzluS+F6yLLNW5gOmZKBXYHGnFHLOnn+alqAjK6gL1u6NMOa74x0E2ER1/7Uy67+hRSsQFRYBevky+6fleMXsBens5CeiKrSTK/LHIMR1Xr4opcW6Wznvy5e8CAF576R/SfYcPU8zHgcP3pm29A2w2UuYKlyraFTvRhoww9WFXfUsLvUibq5ErhXQU6crHa149jaxeg21PSddlye/4rOp+63fJzcJL6B4eHh47BFsuobsoru6yEFY9XbRtVM6QBUuSxtQsfSkHuqTrJSZ04kAkk3OXzwEAhnslGf5+/sI7d7DvPifRqZfGSZLvKovUnmG3qldOX1A9dpGO9LepvqpLHKHXowoSdFjsHJ9QCfi7qE8Ru0YViyKBufwnaAuxGlepb8ND6+dyefaF76fbhQwRlM2mELZZJvXe8tY3p23nL5GkPc2c1AP3i2tblgnNWlOk/AxrNo88IoRmgyMRsyxNHjkk0br3c4rV0QGRSCtFureJclO9eIWiFCdnubjH1NV0X5XJ8rk5kdBbnMI2o1wwXS4ZF0ncVgRlsYfm7QHI+Lq7159LJ2nXVCRqaFwJP9EKYk7FGnEEcmJFPsrm6PwDAxJ5XOY1nleuoN3c74jvmXbntOwa2FHupN3s0hmo6MqE08RGLrqyKZJ3NyeQsR3RGmPWeloq0rHO96PIa/P8FVl/r75O2l+zKRGo7QbNrw019b4+nFSbz8vY77mbIpUP3yvuw7VFktZfeZ5cgF84JkTst79FGuLxV2WtH733IQDAkbtFau/ppfXmyOJwWR/d/K6Re1mTra5kXmd12UcXPRorEjVJ3SfXx7L01MaVzZQ1rFNs3yy8hO7h4eGxQ+Bf6B4eHh47BFtucnHRe7uGxCfc1RhMFLk4sodU+WNsSpkzkqLWhqSWdw8I8dhdYR/QvKjWB9jkUuaUvX/0iT9J99X4Wgt1IdNq7AesM23u4kjOxgypf9WcviaZhV47If7wExNkPlhQ0aM9PXTCSonU51CRWBmO3gtrl9K2wRLt786LQqeSkAIArl5U/vN9ZDbas0dIwPvecITOn5NzvPIiEU/DrAaXVTWjSa6vWKqIyaq/Qse97/F3pG0BO3R3d9NxA/3iPz/DqYbPnpf5mJ8jM9DCvETHLjL5PMdpimcWJAK0wwRvRqU1znKFoEBF1nVXaFw9HFnaq8xTOTZpZQti2lqqC+m8Ev3sQ659+8tcfSZR6V8zAc3HEPurGxUlm2WfaWcKAoA8R0uGKs+uM7GkVZqUycX54NeqsnZcxGJOLUrL5pfaPM33pXMy3zPs/NxTkOOHOcVwPq9r8LIJJSJzU1QU8vwq1/fcOyLPXBdX81pork/kJSotrkviZQPdRn0LlW96Tz+loX37O2ntHj4sJry//eY3AABnz8qzUX2Bn9sFMck9+AaqdrR3L51Lp6eOO7TGY9W3hE27y6p0pfVz3V/Z5ertaoLcWUu0z7sjSNNrLSNF+R2nzDbahHOz8BK6h4eHxw7BlkvojgSs9IqE3ompW7lI3MCOcmGGY8+R5LWQkQi8xJC0N7xbvvSvHid3px/60X+Vtv0DFy6oVklKbLekwMXkFeeKJ9+4Ja4BGKmovN6AJPjdBTrH/FWRhjohScbDQ0KsxuzqVVcSYaNOEmmVybdOIhJYu0GRckMZkQRHyyRJNTvStlJCv3TylXR7gYmzn/4n/zZte/xxSo75N18T98YhJguHihxFqlzh8hw9N9wtkloXb+eVu2CHpRonieqcNVdOkCR1YVJc91pcqCTKS5rYri4ikYdYYmy3VhNRGVWkwOW80LkvurpoLJVKF+9TdSo5n87EhNzvRmP96llFlk7birgtsAtmT0W0niRN5UyEZkHVSU1JLyUdJpbbtBzliou4v4qs6/D97sTS14VpGoN+cDMsoS/NkzY4flmio4f7aCw9JYl2rrF0nShNocNndETsbi7YAAB3c53Rh+6ToiEnz9Dz8sL3xLFgJXTK6IALUASRaN0ZdgqIVXSlSz8bMEl85KgQ8Am7+Y6Pfy5tm52isZ5qilY3cYnqE991hEjXe++XcwwNE0kdqXdLp83FN1RK3Zhr5Lr7uGZBlGU5ZVbvT1M08zzoU6TFZJTovywa9SbhJXQPDw+PHYKNFLjYC+CPAewC+fo8Za39fWNMH4A/B3AAwDkAP2egHt4UAAAgAElEQVStXb8E+DpwuUt6B0SC6PDXvBFIYYR8mSUNzlB44aIEI7z9zeSO1liSL2axi9wExy9J7o3TJ6naecdVA1feTFW223b1i5vZ/DxJRt1lkUjvPkq5JZ596TUAwPPHz0o/fuy9AJZniTxzmiT4OZWx0bk8Nuokme8fFsmuwEEkfX0iGduIJIdOa323poYqBfbgG6mP73r3u9K2/h6ybf/wW5T9myW7LtYUKmWRmkMu2uCq0gNiq9VFB+ZnyW5bYYknURlkDt39AABgaI9kpJyZJc2mq0dcGV3mPmNXV2R3dlhXGg0AltimbFXJMFc44eI42f6dFgQAbS7+ofO7FEvrBxZVWZvqUgUuXJDRpMrTs8DBTglnZTzsAnAA9HD+kzCjpU/a1lpMi+uZ1Zg7aTSl350WzZVRBTFsk44vKY2lp4c0nEKWbNyRkXXSw9pdd5esyRafo6aySbY4w2nAgS69SjMrcpbSMcXTsHCN++8+krZdVe6mdC7NB7C9XPUty7sT/SCy5OpszC2lre3ZewAAcODAgbTt2Qm63x1VHu/q5Bz3h6T348dfTve5wKm77pJ+Dw+T22RXl/BF4AC/Rott7urZy7BGpoOInNuijiuyRrtG0qjS06cFMQThLShwsREJvQPg16y19wJ4K4BfNsbcB+AjAJ621h4B8DT/38PDw8Nji3DdF7q1dtxa+zxvLwI4DmA3gPcD+BQf9ikAP7P2GTw8PDw8bgduiBQ1xhwA8DCA7wAYttaOA/TSN8YMXeOn6yLhGo3dfVLUoFonNacWi4riCDBXK/LkK8oVrkaqTbkkuUi49gDOnxQ18RKTRW97G6XP1WlJuzgdbt+ouEldmCGzSr2pktuXSL2tDBJp9HCX1K68yur4ufMvylhqZJ6Ym5drDQ2SatxtqT/7y+LqN1ThohBGTCguZWpJqbDi9Ec4dM9D6fYHf+nf0PhiUctPnCZiMjEqBw6Tp21W/2bmVNKaxOWxEfrVFVZPIMTW4gL1JJwg1fiyqgfqCpUkDSGbSkzAnjklprCznLLVuf31Dch8OPPA/LyQXtNTRAxaZUIJ2B3OBC6viYo8ZgI2r1MHL62klQU5dpGcnpKxvD5L13RRlgDQ00vk98gI5RNpqajCdovMNomVPi6wWayuzEExR3CGbM7StSudWSVfkrEU2F2xodZuwkRiqcxusGqdZDlKUhPIjmBuKBLQ8HGOlGyrIiZj02RJrakapI5U3DUi638lQmVySLfVNWF4vpa587nfmFX7XJRpV5eYg1KyclnxEmfCo2stzsp9fIFTUL/y0rNpW18/3cddu4QI3jVygK9JZph+ZYod5IK+RhHv7j53lBmww6Rp6raoXR/Z3GWV+c0mK000N44Nk6LGmDKAzwH4VWvtwvWOV7970hhzzBhzrFZb37PAw8PDw2Nz2JCEbigF4OcA/Km19vPcPGGMGWHpfATA5Fq/tdY+BeApABgdHV3F6i1yIpGCylSXZp5LVLk0JlMG+kh6OxlINrjJGZJ8pkP5wnWX6St6zwNCdJw5R5KgKyKgicojR4gkOXLwrrTt/DhJJK+88r20bXqKg1S4CEKvclUbe4Uk+vEp+d4ZJnZDFeA0spfcv/bzF3tfl0hgeS5l1WzowAeSqLRb1Up84Bf+Zbrdu4ukppe+L1KwI5daSgqImaRzpdY0KeNKe8VaguC2YJkYwLlTOAvm1LS4KDq3OxVLgp5KD/dHJN2ZadZGWEqcmhICtMnaSUe5fcZcBjBUuVyKeZrnnHNp1BXZXfIeiPRUUFkkV2KOid7Ll8T9r8Rk9T2q4ILLSFnk/DSNumhVs7Pk3tpuyzhrnGulqNw+uyu07ks5+ltQZGfEUmesSNFOp8XnVdk7XfmztBiDKprAWm5bPXlRyKReolxpOZvk9FXSRKamxcXTZUWcVfl0nKaV6xJtaiWM1RI6/dVEoWGpVuc4SSVt/usISACoL1E/rlyRghiXL9P2fFGOy/A6ciR/SeWPKUZ0nCbIL3FRjVPn5J1Sr1MRl05M5xoYlGInDz5IAYpHDotEPzhIa6HSLc4duQJpEhZ8ffXsddIkjoqYvh2kqKGckh8HcNxa+7tq1xcBPMHbTwD4wqZ74+Hh4eFx09iIhP7DAH4RwPeMMc44/B8B/DaAzxhjPgTgAoCf/cF00cPDw8NjI7juC91a+7dYPyvkuzfbgTOnSc3Zd0TSX+YDTgPaEuIqYrVJiBEhUctctOGee8QP+G++8mUAQG1e/NWL/URenR4j69DePUKiHrybCi/klBp/aB/tn5sR9/pXuW5pwoTL2KyQRwtM5jZiMR8tzJFZZ0gRLuenqa1vL5kfpnPKJzphElWZV2zEtRQTUd9XelG/8OKxdPvl79F310BMOS5fRqSLMKSpYDN8jKjqEafb1elOXT6VrOpvwH7qoaV9laxEyQZslmqHyjzAkbPKbRhZzrXSrrF/dFVMVi0mDU1bRY+yzaelSPOYo0Gri3R8Ud3HwW7qR6RMHc6ysRY12jdI66RXFR5xBRoiNR+LS0RMLi1Rf3M5MZc4UlGnXx0dJjI8lxfzgCNDLecTqTakRw0mnOdmJb/Q9Az5eteVeedeTlOcYd/+5QUduN6pWk9NroU6lkZHiw95i81Ztaqcf36OTI9ZFfXqxv70176Wtr3jLQ9jGVTxhsT5l3dUhCabZJQ7PExqDqJ9oYqcfen55wAAS7Pi797P/vUXx6Wtwj70WX5uEhVhXSmzP7yKD8hGXBgkp+IwAjbjzpKZ6dxZicSem6V5e/6Yyt3DcRt790o07SgXjBkZpWd/dFjeNyVO020Kqt5psH5sxEbhI0U9PDw8dgi2PJfLi6dJWt73wGNpWwL6OhpNAvIXfoEJmrk5IW36+8hl772P/1ja9tAbKY/DZz7/F2mb4bwM3Vx9ffeouFyVmawLOyKZ9O2i6Rk5KFLWPBcneP5FkoLHl5S7VIYI2O4RIYoGDlPbssII7CZ4got2nL4iEmyW2aO6ioys8jR0EpEq3rPCSfTb3/xqul3jzHPZjCpdVnSkrNzy0HL+DlclPaMldOpHPqcIW3b7y6osfVGJxprP0jhzKh+FSxViVJZIR263VeGMBhOeqVSrI+z4eF3aLg3xVRJxT4m2u0s0pnJBpOBchs6XMXIfjXI/XIk2k3TazTFil8p4GdHnyu/x/CnROM9SeL0q46xzhsm68jl1mlCQcW5ssuZPHH8VAHD+3Lm0zUU5W+UOOTpCDgB9nPGyrrzJ3PbcrBCa00z61pUG7HIOOU+0uQXRkgKe+2Ika8fli7lyRTTglRJ6WxXVcKS86cg5XFSqdtazoDZHoi4tyWS5Yip3HxVt/pGHHgUAPPeyFL145lnKIjrHxVHijtyDoREiN9/+9renbRHf53PnxcX5mWcoF9QD91EUeqVbnCsmeMwTE+IA4NburmFxbzx48ABdnx0Lqovi9ukcDDKRaAWNNXIY3Si8hO7h4eGxQ+Bf6B4eHh47BFtucjk5Tyr9VKxSj2ZIBQ9aSkVJXA0++js6IjaHH/khIjTzGVFDD+6nyM+f/MAH07bP/sVf0bWu0HnH50XZazROAwCyEJV3pk7bp8+LWglWi+wgmXR6h8X8kNYVVNGYCZsnEiMmAJeMap4jOfMZlYSMU9hWjUouxWSkTbRKtlw9Gx6U6LnxOhFEcSxqdoXrnEaqbwtTRPYuLlS5X6KaJk5dXit6TZlVMgW6DzZD13eJ1QAgYJtLUSUrc5Xp4/Zqcxo4CZTJiu0iz+RmQZk/+rpITd2rYgD2jJD/r+M9mw1R1QNL6ylSkX09FVp3Ncm1leLkSUoJe//996VtBTah6OkImH5MODpwQkXJumRvzboya7AJMVZmlUOHDwAABoeo/7rwQobNPD0qUZYjVHWZTOdD/toJShu7pApiuH06hiFhk1J1Ueaoxv2scTRrS5nEXDGNCxNCPLoar/E16mDaZRGg1m2kcFGeKogViSNS+VYVVL3dH3nnu3mX/MAVrzj6kJhsH3gT1c11ZVcDRRO7AiyHDkm8ScRzeuCIpNkd3UdEc4EjjruVycWNyxVwAcSsMjQoacBdsq+QTVWBYn9jdnBoKztdYtafy43CS+geHh4eOwRbLqGfmKNvyhf+VqIxH9pP0squrBAGRZYSRnbRF3BkQKSWuw4xuWlFqhjnvCqf+PRfpW3PvUgkk4tEXRZ4aR0pJeeIc3SNWBN97ArYYYK1EyjS0M2mKiXVaPF51Zc4YoI0ZGnMqlwnHaaIMupr7kqRtdrrR5LZtkj03SWSOBYVsdqOSWq7594H5DejJK1McnTgpIoOXOK8Ljpdg5MsbSznLUUkhdzzRkpLelmVlru6QBpAvSUSY50LS+io1By7UpZYE+lRuUsGuYL7yKhIPod3k1vhUE7E1CV2dZxht74wK/NXLBEJXlYRuf2cv+PyWSHCHNos3TeWRMMJHBmpRExXvCJm18RTp06m+xbnHTEtj5grAhIp8TrhkMGAI22hXDH7WavSZGuNUy7X6zKnFy+OLTtOBR/CsotnrSX3zEnX1SnRgDPcT1fyr6MiKavstthRrpISabm+VFlX2knILpiRVRG8/Lx2VARvh+fBnV+XsXMCf0dpOK4cXEvlUBndx/mYEk5Rm6giEvycn70grqD1lssDpAqmdB9cdv3ZeblmxBJ3qXJABuvyIc3LmC9PzPA5qOM5lQ7cBcCasqyPxuz6ZRE3Ci+he3h4eOwQ+Be6h4eHxw7BlptcllgN+ZvnRV09+TpFj77nTUJK3TVKqv3ZMxSp+Y43i+kgz6r6YkvUuc/8NaXHfP5VSbBUc1FqbPIIVKpSpxYFKrrNmUlipc412RTSZpXQKN/mJkdcajIoilbXvyxyIqEsXAXydBdiJhV1UqwOE4jZLqnyszIX2vRlScQVt0l1qyt1uHaREpP1qQrrg5xWNsNVcgoqi1Y9dBVYtF1qtZpdq5OZ5h1cNer+eyV51YULZM6YnpNI26Yj2xSZFjHRXWAWa0ARoD2lEl9Z7sGVKRrLiSlJ0mSY2KoMkRmpUBHCtMgkqk7LW1Yk10oU+J61lFnDkdXL6mQ6/3M2V1QqEr2cZ5/+cklIvZDHVVTRps7Eceo1Suw2PyOmgHmO6IyVz3kmyxGraj3lWH83PH81FW06ycRdrSnqfMhj6O2W9dRi81yNneQ7KvlXkppXdP5Xng+zvkz4rW99XcbSoapBpUjmI+Z111ZmFUfMu4Rk+llqs2lLP4+OcGw0pS1OK2BxKmpVP7Svh8y55bKumEVj0PyuScfnEp6piE4ec6BMKBEn/QrM6uPcEJaFVxh+fxTl+KDB5kJFeN8ovITu4eHhsUOw5RJ6/wDlt5iZlc/jOEe1/T3X7QSAuL2ft+hLOLhLojxNSF/g7x6TaLG/+hpFejUTkQjAX+ogWP0di1lytOoz7dzRtJTgojwzLBkY/TnlPBSa9HK1KHXumZCvH1qWOKzSFFjK12L7yC6SJrsqSqqsLZfQd430pdtjF8Z4TLqYAG2fPXkibZpnd0J39apyi6yyNJTEy5hjOl4VE2g1SaJ7/m+/AgB4Z0nG+QCPs94t0rIjAXUUcIMJu3mO3tTk7PnXKBpvqi6Ri40MXb8wJGPu3UUSV65CYwpVpGiR3f5yRSHZTbj+0neusXFH7oGLMk46SlvjsTtStKAiKQPWGusqJ0pzhrTFC7o4Bc+DSyHr8uUAQp5n8kor4Eu0WjJ/i7MkkTcaS/xXiGx3p/JqzbfrnIJX1X91BKb7q8lI517YUdqJZak2m1mfqM+rSOV2yPdFpcTOsdNBolxdndtmwNfUJHTC+W60VuAiZhOrooB51NbV7TSKhObbF6i6uFHIKaubEtmaEqQ8PF2ztM0as9a63Zox6tlY+Z5pqahXy+doqNdHLiRtanR0P24WXkL38PDw2CHYcgndSbMZlQWw0yDp6uyESGXNKgV7vOMRqiBf6JGcCfNcDOKb35GMg3W2/bZVtrscu4056WOtCkqhkhbSj62yreVYsjNOVArU8TmSQgqq/JlzcWqrQJpFltpcUEZTSYLdveyyOSKJ8svsD1lXgSArP8X7jkomtwV24auOTakjOOueckeb4etmecwtZS8Xu+1qt7RlBQkYp16m/BkXF0XyGQxoPpZpOCy1LCl7/RVLUuFptqmOqRwgtSJrOPukwMDwQZJg8j3iupreB5aaymXRFIpsTw/UGrPXsP0ucJ6g2qK4LU5epjXZaEjfXPk4l8dD32On6QUqmCnDgW+OVwEkw2XENnftothmO7LOB9Ns0tpZVO5x7raVKuwOqyRD26Z5bi7JWndFMuaVROokc2efNspentjVwWUut41J1i+6kqj7uFQlHqUY6ntAf2O1mF0AVIvdcDsd5crHhTysksYlq6U8hx22ocdOG1T32gVVaeHZWupns6Fz28TLjteau035nFi1uaBCXSRm+TXDlu43587p1YVvaHsUXkL38PDw+EcP/0L38PDw2CG4rsnFGJMH8C1QTYUIwGettb9ljDkI4NMA+gA8D+AXrVWhmhtESjJpYjAk1bGlSJuJJVKLnj9BxNJ7a6ICLVoyRVyaFZNEnlXuTk3O0WAV09WAjFQUn9u3zC3NOLcnOc4Gy1POZnLigrbErl4tlYLXmV+02cGZWKocsVruEfNKL+eCaKmUn6+xS1tGuWu9aYVWVukVgnBwmPKrjCuTS6r+qd802azi6k1q18D4GhGAy/bwidusslenJN9HkOOUxMpl7jJf40WIOn464vkokxpf2itFMgZHKSdPPxedAIAcuwK2VE8smwVyEVe5jzQx7doUaXkN37Ar58iFVldhdyq40RG/nL7XVX/X6naWzTs6j43brwnHDpsYlpa45mtT51xhlzmjXQhpXWRVMYbh3aN8DoroXJgVN9EOF6ywioR25pRaS5thnDnD+dhh1fEZNXZXeKJWU2bAFbh4UZwUTo1TP0qqRmjEtqJ4WUkOmlMXDZoooj7LuX50mzPRxDq1Ec+zIy2NypHiyFZt23L5YPR9ce61SeyiSBXZySbKZTmbXAEPuzqy1f2yrfJExX20LnY/KK7Z3e6WbiKly0Yk9CaAd1lr3wjgIQCPG2PeCuB3APyetfYIgFkAH7r5bnh4eHh4bBYbKUFnATg/qwz/swDeBcCVmv8UgP8E4GM33ANHNujCARz8kqi8Dy6fytlJkgg+8Zkvp/ve9U5Kcn/2skiHVRcsoL5ZGZepjqWEonI7ynLhivqiSNeOuLCKtMwwQekkQE2EOUkwUQRKnV3UdJs7roel6n6VFP/qNAWWzE1Jhse58xRMdfjQQayHQl4kthwHsGRUPpOYyTH98e+kkguPT++8hpSwjCJjaWiJx/eakvq6uTzdaw0pBPAKay/TFZFc+/fSuEYOkjTeo1wwc+wGGah8HG1eK2GkSrmxRBylQTZyfCpda5eya5CiYcKue8p1NHUv1OdlbS2wTmKTczTZBbPTlvXkJG5dcd7BkeeZrC4RyGUDNanMazGfU+5/BfrNzDRdU2dRzLDGGerq8qyNdrQ0uYLUWxZI4wp+KK1niYuo1KqSD2YlAqvKFzppNRap1mkDy4KTQnZbtM41UGlaLBmrOKt07q1yTXQ3woqPYgonhWvX4g5fv62cAhJ+B1lXIlA9D2leJtURg9VjsUx+dziAsaLyEe15kJw7IiP3e+4k57PaI9rojWJDNnRjTMgFoicBfBXA6wDmrIQRjgHYvc5vnzTGHDPGHFvLq8TDw8PD49ZgQy90a21srX0IwB4AjwG4d63D1vntU9baR621jxZVbmMPDw8Pj1uLG/JDt9bOGWO+AeCtAHqMMRFL6XsAXL7mj9dBP1cqb6iCBFWOZMuG4s/t0mo6X+JvfvfldN9Zrm84VxVmZGaJ1GbFLaLE6nuH1a6cql7vVPV8QeWJCJyPsKj2zme2wyYGo/1TWQWLVYX6FvvJFlT+Dpdkv2+ATC0tRQg3uaBDPSfXTDh6UFeEX4m2iuiscj6Orh65ZqNKarYuoBCzephmbFWpW81qq0AKq9IDWyaUquwj/G1VlOR8jdqmVb6KaJgqoI/sGUzbDg7Sdn83zUugok2rLCc0FLEVseqva37mOQo04urr+YIIDzmeex2FeS0ka+QRccqoVaYfy2xyatJR53CRhrE2GfA60uvOrTFH0i6zeiVuPQmpHDP53MrIva1zWltnakk0Acq5XxpKO3bjstoX2x3vzBWqHxGPxbaEyJ6dJjNau7X+muwoP/SYj2sFmhB2eX10URRu4mcpUPfApchNtGmEzWKJSjftCGln/dDHO5OZtvIkzj9cmdicmSk1zWj/cjYLQRO2zmyj3gdtTmPddzcV09h9YG+6r8H1SF9/TWJnCm22bEsQ/A3juhK6MWbQGNPD2wUAPw7gOICvA/gAH/YEgC/cfDc8PDw8PDaLjUjoIwA+ZSghQgDgM9baLxljXgXwaWPMfwbwAoCP30wHGix15tSnpckSUiYUKbXDH0qXsD8oiBR3jsnQQJE2HZaeOorQbHBGuSpHamrix0lNpaxIcQUmSgMlVTjCsVCk6+ucGlc5U16i3JMiJkR6K0Ja7uojrWTXLiL/5qoiySxwZsKleYlS7OFCB1NXdeTnADTaqop9mKWx9w7KNdtlmstOW2W2S9xfJkyVhO6GrCMGU+lNs3+OuONshG2VQ6XZTf2+q0dInt4+iu4sV2TplYt033JMODdUvpQWuzlaJV2Hzt1U94O3M6xpabdFV7xBE2z2Gqxvg139Iu2u6lzhtOsjj90VutDraaXkzR2grupITp575zYYq8jLNs9DqDSzNucDiZV7balJmo2TzHWunWadpfs1SsUla0T8un5Eer653zMTkj+ozRGr+hasgh4653wJsnLNjMt2Gi+ryME/5blSp7MuQ6HSEPOsgfRWhEh3JedcQRY9pyG7mOaUBuzytCyLjuX74iJnFxdUHhZenkkkczTPqRSjAenH/qNEfPZy9Pel106n+6ZOU0bZSPUtf428OBvFRrxcXgbw8BrtZ0D2dA8PDw+POwA+UtTDw8Njh2DLk3M5lTCnkhgVHTHSFlXTuZkm7AWtEwYlrJ51WorEil0KTU1s0XaSpuiU79nsDJk6ZtQ1K1wYoVtFYVbYdz0PMse46t0AELFKGKpal01O5uQKJOjjOjWu1VhTSYzmpnnswubmOSKxcY3oxlCpaz39ZA4ql5QfepNNUMrk0omdb7rzPVaJxvhbHyxLB8pmBJVcKmIVusgmjq4uFcHIRQTKOSG3S+ybns2JutrizSX2m68rgtcRt3ml3mZD57MtanOwwpyh73uLSa9sVpFYmfXn0kX/BsqskXGmPm0u4b65GVpWtD2NHFTJq+LVxLSLlHaFLlotue91NrXEdRXRyaRoSZmlCt2k0nd4nO2GnCNYwyaS+uNrgtyFg7ApqqRiNKpcG3ZhQcyAzmKl18xKhB01x1y3M1ERwhbU3xAqZTBvS1StIjSNXfYXABJOvleLJJGfRHu79Ndqvjmau9GWvrm1bpb5sqed5DOpUFS+via8K5zKefCoxIoE/K468ex36JqTYjIN+f7pQiVrmcBuFF5C9/Dw8NghMPYWfBU2itHRUfvkk0/etut5eHh47AR89KMffc5a++j1jvMSuoeHh8cOgX+he3h4eOwQ+Be6h4eHxw6Bf6F7eHh47BDcVlLUGHMVQBXA1PWOvcMxgO09hu3ef2D7j2G79x/Y/mPYTv3fb60dvN5Bt/WFDgDGmGMbYWvvZGz3MWz3/gPbfwzbvf/A9h/Ddu//WvAmFw8PD48dAv9C9/Dw8Ngh2IoX+lNbcM1bje0+hu3ef2D7j2G79x/Y/mPY7v1fhdtuQ/fw8PDw+MHAm1w8PDw8dghu6wvdGPO4MeaEMea0MeYjt/PaNwNjzF5jzNeNMceNMa8YY36F2/uMMV81xpziv71b3ddrgYt8v2CM+RL//6Ax5jvc/z83xmSvd46thDGmxxjzWWPMa3wv3rYN78G/5zX0fWPMnxlj8nfyfTDGfMIYM2mM+b5qW3PODeG/83P9sjHmka3ruWCdMfwXXkcvG2P+wlVj432/wWM4YYz5p1vT683htr3QueLRHwB4D4D7APy8Mea+23X9m0QHwK9Za+8F1VH9Ze7zRwA8ba09AuBp/v+djF8BlQ10+B0Av8f9nwXwoS3p1cbx+wD+2lp7D4A3gsaybe6BMWY3gH8H4FFr7QOgWj4fxJ19Hz4J4PEVbevN+XsAHOF/TwL42G3q4/XwSawew1cBPGCtfQOAkwB+AwD4uf4ggPv5N//DLMunuz1wOyX0xwCcttaesda2AHwawPtv4/VvGNbacWvt87y9CHqR7Ab1+1N82KcA/MzW9PD6MMbsAfCTAP6Q/28AvAvAZ/mQO73/FQDvAJc4tNa2rLVz2Eb3gBEBKBhjIgBFAOO4g++DtfZbAGZWNK835+8H8MeW8AyogPzI7enp+lhrDNbar1hJUv8MpCTz+wF82lrbtNaeBXAa27Ai2+18oe8GcFH9f4zbtgWMMQdApfi+A2DYWjsO0EsfwNDW9ey6+G8A/gMAl+W/H8CcWtR3+n04BOAqgD9is9EfGmNK2Eb3wFp7CcB/BXAB9CKfB/Acttd9ANaf8+36bP9rAP+Xt7frGJbhdr7Q16qAui1cbIwxZQCfA/Cr1tqF6x1/p8AY81MAJq21z+nmNQ69k+9DBOARAB+z1j4MSh1xx5pX1gLbmt8P4CCAUQAlkJliJe7k+3AtbLc1BWPMb4JMqn/qmtY47I4ew1q4nS/0MQB71f/3ALh8G69/UzDGZEAv8z+11n6emyecSsl/J9f7/RbjhwG8zxhzDmTiehdIYu9h1R+48+/DGIAxa+13+P+fBb3gt8s9AIAfB3DWWnvVWtsG8HkAP4TtdR+A9ed8Wz3bxpgnAPwUgF+w4re9raMrqJEAAAF9SURBVMawHm7nC/1ZAEeY2c+CCIgv3sbr3zDY3vxxAMettb+rdn0RwBO8/QSAL9zuvm0E1trfsNbusdYeAM3316y1vwDg6wA+wIfdsf0HAGvtFQAXjTF3c9O7AbyKbXIPGBcAvNUYU+Q15cawbe4DY705/yKAX2Jvl7cCmHemmTsNxpjHAfw6gPdZa2tq1xcBfNAYkzPGHAQRvN/dij5uCtba2/YPwHtBzPLrAH7zdl77Jvv7dpDa9TKAF/nfe0F26KcBnOK/fVvd1w2M5Z0AvsTbh0CL9TSA/w0gt9X9u07fHwJwjO/DXwLo3W73AMBHAbwG4PsA/gRA7k6+DwD+DGTvb4Ok1w+tN+cgc8Uf8HP9PZA3z506htMgW7l7nv+nOv43eQwnALxnq/t/M/98pKiHh4fHDoGPFPXw8PDYIfAvdA8PD48dAv9C9/Dw8Ngh8C90Dw8Pjx0C/0L38PDw2CHwL3QPDw+PHQL/Qvfw8PDYIfAvdA8PD48dgv8P8QITwTAXGKoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "dataiter = iter(testloader)\n",
    "images, labels = dataiter.next()\n",
    "\n",
    "# 显示图片\n",
    "imshow(torchvision.utils.make_grid(images))\n",
    "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们看看神经网络认为以上图片是什么。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = net(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "输出是10个标签的能量。\n",
    "一个类别的能量越大，神经网络越认为它是这个类别。所以让我们得到最高能量的标签。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted:    cat  ship  ship  ship\n"
     ]
    }
   ],
   "source": [
    "_, predicted = torch.max(outputs, 1)\n",
    "\n",
    "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n",
    "                              for j in range(4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果看来不错。\n",
    "\n",
    "接下来让看看网络在整个测试集上的结果如何。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the network on the 10000 test images: 53 %\n"
     ]
    }
   ],
   "source": [
    "correct = 0\n",
    "total = 0\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs.data, 1)\n",
    "        total += labels.size(0)\n",
    "        correct += (predicted == labels).sum().item()\n",
    "\n",
    "print('Accuracy of the network on the 10000 test images: %d %%' % (\n",
    "    100 * correct / total))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "结果看起来不错，至少比随机选择要好，随机选择的正确率为10%。\n",
    "似乎网络学习到了一些东西。\n",
    "\n",
    "\n",
    "\n",
    "在识别哪一个类的时候好，哪一个不好呢？\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of plane : 54 %\n",
      "Accuracy of   car : 65 %\n",
      "Accuracy of  bird : 33 %\n",
      "Accuracy of   cat : 15 %\n",
      "Accuracy of  deer : 35 %\n",
      "Accuracy of   dog : 59 %\n",
      "Accuracy of  frog : 74 %\n",
      "Accuracy of horse : 62 %\n",
      "Accuracy of  ship : 80 %\n",
      "Accuracy of truck : 56 %\n"
     ]
    }
   ],
   "source": [
    "class_correct = list(0. for i in range(10))\n",
    "class_total = list(0. for i in range(10))\n",
    "with torch.no_grad():\n",
    "    for data in testloader:\n",
    "        images, labels = data\n",
    "        outputs = net(images)\n",
    "        _, predicted = torch.max(outputs, 1)\n",
    "        c = (predicted == labels).squeeze()\n",
    "        for i in range(4):\n",
    "            label = labels[i]\n",
    "            class_correct[label] += c[i].item()\n",
    "            class_total[label] += 1\n",
    "\n",
    "\n",
    "for i in range(10):\n",
    "    print('Accuracy of %5s : %2d %%' % (\n",
    "        classes[i], 100 * class_correct[i] / class_total[i]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下一步?\n",
    "\n",
    "我们如何在GPU上运行神经网络呢？\n",
    "\n",
    "在GPU上训练\n",
    "----------------\n",
    "把一个神经网络移动到GPU上训练就像把一个Tensor转换GPU上一样简单。并且这个操作会递归遍历有所模块，并将其参数和缓冲区转换为CUDA张量。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cpu\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "\n",
    "# 确认我们的电脑支持CUDA，然后显示CUDA信息：\n",
    "\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "本节的其余部分假定`device`是CUDA设备。\n",
    "\n",
    "然后这些方法将递归遍历所有模块并将模块的参数和缓冲区\n",
    "转换成CUDA张量：\n",
    "\n",
    "\n",
    "```python\n",
    "\n",
    "    net.to(device)\n",
    "```\n",
    "\n",
    "记住：inputs 和 targets 也要转换。\n",
    "\n",
    "```python\n",
    "\n",
    "        inputs, labels = inputs.to(device), labels.to(device)\n",
    "```\n",
    "为什么我们没注意到GPU的速度提升很多？那是因为网络非常的小。\n",
    "\n",
    "**实践:** \n",
    "尝试增加你的网络的宽度（第一个``nn.Conv2d``的第2个参数，第二个``nn.Conv2d``的第一个参数，它们需要是相同的数字），看看你得到了什么样的加速。\n",
    "\n",
    "**实现的目标**:\n",
    "\n",
    "- 深入了解了PyTorch的张量库和神经网络\n",
    "- 训练了一个小网络来分类图片\n",
    "\n",
    "***译者注：后面我们教程会训练一个真正的网络，使识别率达到90%以上。***\n",
    "\n",
    "多GPU训练\n",
    "-------------------------\n",
    "如果你想使用所有的GPU得到更大的加速，\n",
    "请查看[数据并行处理](5_data_parallel_tutorial.ipynb)。\n",
    "\n",
    "下一步？\n",
    "-------------------\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "-  :doc:`训练神经网络玩电子游戏 </intermediate/reinforcement_q_learning>`\n",
    "-  `在ImageNet上训练最好的ResNet`\n",
    "-  `使用对抗生成网络来训练一个人脸生成器`\n",
    "-  `使用LSTM网络训练一个字符级的语言模型`\n",
    "-  `更多示例`\n",
    "-  `更多教程`\n",
    "-  `在论坛上讨论PyTorch`\n",
    "-  `Slack上与其他用户讨论`\n",
    "\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}