{ "cells": [ { "cell_type": "markdown", "id": "metric-heath", "metadata": {}, "source": [ "## Deep learning example from Accounting/Finance" ] }, { "cell_type": "markdown", "id": "operating-assignment", "metadata": {}, "source": [ "The following example demonstrates a simple example of deep learning that uses accounting/finance data. It also demonstrates, how to implement a deep learning model to traditional structured data. However, it also shows how deep learning is usually not the best option for structured data with relatively small datasets (<100k observations). Deep learning models perform better with large unstructured datasets." ] }, { "cell_type": "code", "execution_count": 1, "id": "twelve-oregon", "metadata": {}, "outputs": [], "source": [ "import tensorflow as tf\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "concerned-baking", "metadata": {}, "source": [ "The data has a little under 20k observations. The variables are different financial ratios and board characteristics of S&P1500 companies." ] }, { "cell_type": "code", "execution_count": 2, "id": "existing-ancient", "metadata": {}, "outputs": [], "source": [ "compu_df = pd.read_csv('_data.txt',delimiter='\\t')" ] }, { "cell_type": "code", "execution_count": 3, "id": "polar-active", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
GVKEYdatadatefyearcusipconmactatbkvlpscapxceq...ind_chairman_is_ex_ceoind_independent_board_membersind_strictly_independent_board_membersind_board_member_affiliationsind_non_executive_board_membersind_board_gender_diversity_percentind_board_specific_skills_percentind_executive_members_gender_diversity_percentind_average_board_tenureind_board_member_compensation
021542200812312008000360206AAON INC80.118140.7435.60889.61096.522...0.087.50057.1401.42590.9113.39550.617.7358.2701650394.0
121542200912312009000360206AAON INC96.240156.2116.85449.774117.999...0.087.50050.0001.18090.0012.91560.006.6708.7051590889.5
221542201012312010000360206AAON INC91.748160.2777.072517.470116.739...0.084.62051.9251.00090.0011.11058.339.0908.7801801674.0
321542201112312011000360206AAON INC84.387178.9814.976235.914122.504...0.086.67050.0001.09090.0011.11057.1410.0009.1801847006.5
421542201212312012000360206AAON INC91.546193.4935.634114.147138.136...0.087.50050.0001.19090.0014.29054.559.0909.1701810953.0
..................................................................
1963228191201512312015V7780T103ROYAL CARIBBEAN GROUP837.02220921.85536.98761613.3408063.039...1.084.62050.0000.90583.3316.67053.8512.5008.8801744895.0
1963328191201612312016V7780T103ROYAL CARIBBEAN GROUP748.30522310.32442.50542494.3639121.412...1.083.33050.0000.88083.3316.67053.8514.2909.3401737800.0
1963428191201712312017V7780T103ROYAL CARIBBEAN GROUP843.02822296.31750.1659564.13810702.303...1.083.33050.0000.88583.3320.00057.1413.3959.1051793588.5
1963528191201812312018V7780T103ROYAL CARIBBEAN GROUP1242.04427698.27053.13193660.02811105.461...1.085.71050.0000.74581.8222.22058.3314.2909.1801858984.0
1963628191201912312019V7780T103ROYAL CARIBBEAN GROUP1162.62830320.28458.25573024.66312163.846...1.085.16548.0750.80080.0025.00060.0015.1909.1951884002.5
\n", "

19637 rows × 102 columns

\n", "
" ], "text/plain": [ " GVKEY datadate fyear cusip conm act \\\n", "0 21542 20081231 2008 000360206 AAON INC 80.118 \n", "1 21542 20091231 2009 000360206 AAON INC 96.240 \n", "2 21542 20101231 2010 000360206 AAON INC 91.748 \n", "3 21542 20111231 2011 000360206 AAON INC 84.387 \n", "4 21542 20121231 2012 000360206 AAON INC 91.546 \n", "... ... ... ... ... ... ... \n", "19632 28191 20151231 2015 V7780T103 ROYAL CARIBBEAN GROUP 837.022 \n", "19633 28191 20161231 2016 V7780T103 ROYAL CARIBBEAN GROUP 748.305 \n", "19634 28191 20171231 2017 V7780T103 ROYAL CARIBBEAN GROUP 843.028 \n", "19635 28191 20181231 2018 V7780T103 ROYAL CARIBBEAN GROUP 1242.044 \n", "19636 28191 20191231 2019 V7780T103 ROYAL CARIBBEAN GROUP 1162.628 \n", "\n", " at bkvlps capx ceq ... ind_chairman_is_ex_ceo \\\n", "0 140.743 5.6088 9.610 96.522 ... 0.0 \n", "1 156.211 6.8544 9.774 117.999 ... 0.0 \n", "2 160.277 7.0725 17.470 116.739 ... 0.0 \n", "3 178.981 4.9762 35.914 122.504 ... 0.0 \n", "4 193.493 5.6341 14.147 138.136 ... 0.0 \n", "... ... ... ... ... ... ... \n", "19632 20921.855 36.9876 1613.340 8063.039 ... 1.0 \n", "19633 22310.324 42.5054 2494.363 9121.412 ... 1.0 \n", "19634 22296.317 50.1659 564.138 10702.303 ... 1.0 \n", "19635 27698.270 53.1319 3660.028 11105.461 ... 1.0 \n", "19636 30320.284 58.2557 3024.663 12163.846 ... 1.0 \n", "\n", " ind_independent_board_members ind_strictly_independent_board_members \\\n", "0 87.500 57.140 \n", "1 87.500 50.000 \n", "2 84.620 51.925 \n", "3 86.670 50.000 \n", "4 87.500 50.000 \n", "... ... ... \n", "19632 84.620 50.000 \n", "19633 83.330 50.000 \n", "19634 83.330 50.000 \n", "19635 85.710 50.000 \n", "19636 85.165 48.075 \n", "\n", " ind_board_member_affiliations ind_non_executive_board_members \\\n", "0 1.425 90.91 \n", "1 1.180 90.00 \n", "2 1.000 90.00 \n", "3 1.090 90.00 \n", "4 1.190 90.00 \n", "... ... ... \n", "19632 0.905 83.33 \n", "19633 0.880 83.33 \n", "19634 0.885 83.33 \n", "19635 0.745 81.82 \n", "19636 0.800 80.00 \n", "\n", " ind_board_gender_diversity_percent ind_board_specific_skills_percent \\\n", "0 13.395 50.61 \n", "1 12.915 60.00 \n", "2 11.110 58.33 \n", "3 11.110 57.14 \n", "4 14.290 54.55 \n", "... ... ... \n", "19632 16.670 53.85 \n", "19633 16.670 53.85 \n", "19634 20.000 57.14 \n", "19635 22.220 58.33 \n", "19636 25.000 60.00 \n", "\n", " ind_executive_members_gender_diversity_percent \\\n", "0 7.735 \n", "1 6.670 \n", "2 9.090 \n", "3 10.000 \n", "4 9.090 \n", "... ... \n", "19632 12.500 \n", "19633 14.290 \n", "19634 13.395 \n", "19635 14.290 \n", "19636 15.190 \n", "\n", " ind_average_board_tenure ind_board_member_compensation \n", "0 8.270 1650394.0 \n", "1 8.705 1590889.5 \n", "2 8.780 1801674.0 \n", "3 9.180 1847006.5 \n", "4 9.170 1810953.0 \n", "... ... ... \n", "19632 8.880 1744895.0 \n", "19633 9.340 1737800.0 \n", "19634 9.105 1793588.5 \n", "19635 9.180 1858984.0 \n", "19636 9.195 1884002.5 \n", "\n", "[19637 rows x 102 columns]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "compu_df" ] }, { "cell_type": "markdown", "id": "experimental-brand", "metadata": {}, "source": [ "We use only variables with few missing values, because the benefits of deep learning are seen in large datasets. The variables are different financial ratios (Compustat). There are also many heavily correlating variables included, but this should not be a serious issue with neural networks." ] }, { "cell_type": "code", "execution_count": 4, "id": "brave-maple", "metadata": {}, "outputs": [], "source": [ "variables = ['at', 'bkvlps','capx', 'ceq', 'csho', 'cstk', 'dlc', 'dltt', 'dvc', 'ebit',\n", " 'ibc', 'icapt', 'lt', 'ni', 'pstk', 'pstkl','pstkrv', 're', 'sale', 'seq', 'costat', 'prcc_c',\n", " 'prcc_f', 'sic', 'mkvalt','tobin', 'yld', 'age', 'tridx', 'mb',\n", " 'cap_int', 'lvg', 'roa', 'roe', 'roi']" ] }, { "cell_type": "code", "execution_count": 5, "id": "pediatric-poison", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "at 3\n", "bkvlps 34\n", "capx 53\n", "ceq 3\n", "csho 11\n", "cstk 13\n", "dlc 9\n", "dltt 81\n", "dvc 77\n", "ebit 15\n", "ibc 17\n", "icapt 4\n", "lt 49\n", "ni 7\n", "pstk 4\n", "pstkl 37\n", "pstkrv 19\n", "re 12\n", "sale 7\n", "seq 3\n", "costat 0\n", "prcc_c 8\n", "prcc_f 6\n", "sic 0\n", "mkvalt 13\n", "tobin 13\n", "yld 85\n", "age 0\n", "tridx 172\n", "mb 36\n", "cap_int 53\n", "lvg 86\n", "roa 7\n", "roe 16\n", "roi 8\n", "dtype: int64" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "compu_df[variables].isna().sum()" ] }, { "cell_type": "code", "execution_count": 6, "id": "extreme-screen", "metadata": {}, "outputs": [], "source": [ "compu_df[variables] = compu_df[variables].clip(lower=compu_df[variables].quantile(0.01),\n", " upper=compu_df[variables].quantile(0.99),axis=1)" ] }, { "cell_type": "code", "execution_count": 7, "id": "adopted-difficulty", "metadata": {}, "outputs": [], "source": [ "compu_df['current_roa'] = compu_df['roa']" ] }, { "cell_type": "markdown", "id": "linear-admission", "metadata": {}, "source": [ "Lag everything else" ] }, { "cell_type": "code", "execution_count": 8, "id": "cellular-center", "metadata": {}, "outputs": [], "source": [ "compu_df[variables] = compu_df.groupby(['conm']).shift()[variables]" ] }, { "cell_type": "markdown", "id": "superior-asset", "metadata": {}, "source": [ "We have to drop all missing values, because othwerise we can not optimize the network using gradient descent algorithm.\n", "\n", "I add industry, SP500 dummy and year separately as I do not want to winsorize or lag these variables." ] }, { "cell_type": "code", "execution_count": 9, "id": "essential-master", "metadata": {}, "outputs": [], "source": [ "compu_df[variables + ['fyear','ind','sp500','current_roa']] = compu_df[variables + ['fyear','ind','sp500','current_roa']].dropna()" ] }, { "cell_type": "code", "execution_count": 10, "id": "developmental-partnership", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
atbkvlpscapxceqcshocstkdlcdlttdvcebit...agetridxmbcap_intlvgroaroeroifyearcurrent_roa
0NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
1140.7435.60889.61096.52217.2090.0712.9920.0005.62143.388...204.01.0703733.7227210.0682800.0309980.2031290.0795630.2961922009.00.177459
2156.2116.85449.774117.99917.2150.0710.0760.0006.20143.754...216.01.0174572.8434290.0625690.0006440.1774590.0826210.2349262010.00.136601
3160.2777.072517.470116.73916.5060.0680.0000.0006.06732.715...228.01.4944623.9886890.1089990.0000000.1366010.0470200.1875472011.00.078142
4178.9814.976235.914122.50424.6180.0984.5750.0005.93523.971...240.01.6461414.1176000.2006580.0373460.0781420.0277270.1141682012.00.141860
5193.4935.634114.147138.13624.5180.0980.0000.0008.84044.238...252.01.7065813.7042300.0731140.0000000.1418600.0536440.1987102013.00.174277
6215.4444.47029.041164.10636.7110.1470.0000.0007.42855.803...264.03.9494857.1473310.0419650.0000000.1742770.0320120.2287972014.00.189424
7233.1173.220816.127174.05954.0420.2160.0000.0009.65671.563...276.04.1857996.9516890.0691800.0000000.1894240.0364940.2536962015.00.196381
8232.8543.375020.967178.91853.0120.2120.0000.00011.85771.695...288.04.3815916.8800000.0900440.0000000.1963810.0371490.2555812016.00.208069
9256.5303.910626.604205.89852.6510.2110.0000.00012.67679.574...300.06.2861828.4513890.1037070.0000000.2080690.0306740.2592352017.00.183631
10296.7804.525241.713237.22652.4230.2100.0000.00013.65374.148...312.07.0300238.1101390.1405520.0000000.1836310.0283260.2297302018.00.138132
11308.1974.760437.268247.49951.9910.2080.0000.00016.71757.678...324.06.7766467.3649270.1209230.0000000.1381320.0233550.1720092019.00.144608
12NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
131377.51116.893727.535656.89538.88444.20163.600392.9840.000125.529...444.00.3865371.0897550.0199890.6950640.0570960.1098700.0749142009.00.029731
141501.04218.916728.855746.90639.48444.870100.833336.1910.00095.415...456.00.5180111.2148000.0192230.5851120.0297310.0491850.0412252010.00.040984
151703.72721.0112124.879835.84539.78144.986114.075329.8022.983137.016...468.00.6959241.3073980.0732980.5310520.0409840.0638970.0599322011.00.030844
162195.65321.469791.218864.64940.27344.849122.865669.48912.081142.360...480.00.3224680.8928860.0415450.9163880.0308440.0877200.0441052012.00.025738
172136.90023.325437.600918.60039.38244.70086.400622.20011.900136.600...492.00.5467820.8008440.0175960.7713910.0257380.0747630.0356752013.00.033144
182199.50025.265426.500999.50039.56044.70069.700564.30011.800142.600...504.00.6699141.1086310.0120480.6343170.0331440.0657900.0465812014.00.006733
191515.00023.857446.300845.10035.42344.90069.00085.00011.900-8.600...516.00.8231341.1644190.0305610.1822270.0067330.0103650.0109672015.00.033077
201442.10025.084788.400865.80034.51544.90012.000136.10010.40066.100...528.00.6889191.0480490.0612990.1710560.0330770.0525680.0476102016.00.037564
211504.10026.611233.600914.20034.35445.2002.000155.30010.20077.200...540.00.9958691.2419580.0223390.1720630.0375640.0497620.0528282017.00.010232
221524.70026.970322.000936.30034.71645.3000.000177.20010.30086.000...552.01.2833691.4567880.0144290.1892560.0102320.0114370.0140102018.00.004943
231517.20026.040617.400905.90034.78845.3000.000141.70010.500110.700...564.00.8704831.4339150.0114680.1564190.0049430.0057740.0071592019.00.002116
24NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN...NaNNaNNaNNaNNaNNaNNaNNaNNaNNaN
251549.91312.637634.063644.05150.9630.5810.000230.00025.271114.309...468.00.8209941.5074060.0219770.3571150.0293140.0467980.0519812009.00.035692
261521.15313.292318.582687.05051.6880.5170.000172.50026.72782.506...480.00.9730931.5542830.0122160.2510730.0356920.0508420.0631642010.00.041404
271548.67014.040623.942739.02552.6350.5260.000140.50028.152118.882...492.01.1987131.8731390.0154600.1901150.0414040.0463200.0729042011.00.036446
281879.59814.923022.124795.88653.3330.5330.000300.00029.744124.148...504.01.1012681.3817600.0117710.3769380.0364460.0622920.0625102012.00.033480
291869.25115.634028.052850.39854.3940.5440.000215.00031.309111.711...516.01.0644611.2760650.0150070.2528230.0334800.0576710.0587402013.00.034399
\n", "

30 rows × 37 columns

\n", "
" ], "text/plain": [ " at bkvlps capx ceq csho cstk dlc dltt \\\n", "0 NaN NaN NaN NaN NaN NaN NaN NaN \n", "1 140.743 5.6088 9.610 96.522 17.209 0.071 2.992 0.000 \n", "2 156.211 6.8544 9.774 117.999 17.215 0.071 0.076 0.000 \n", "3 160.277 7.0725 17.470 116.739 16.506 0.068 0.000 0.000 \n", "4 178.981 4.9762 35.914 122.504 24.618 0.098 4.575 0.000 \n", "5 193.493 5.6341 14.147 138.136 24.518 0.098 0.000 0.000 \n", "6 215.444 4.4702 9.041 164.106 36.711 0.147 0.000 0.000 \n", "7 233.117 3.2208 16.127 174.059 54.042 0.216 0.000 0.000 \n", "8 232.854 3.3750 20.967 178.918 53.012 0.212 0.000 0.000 \n", "9 256.530 3.9106 26.604 205.898 52.651 0.211 0.000 0.000 \n", "10 296.780 4.5252 41.713 237.226 52.423 0.210 0.000 0.000 \n", "11 308.197 4.7604 37.268 247.499 51.991 0.208 0.000 0.000 \n", "12 NaN NaN NaN NaN NaN NaN NaN NaN \n", "13 1377.511 16.8937 27.535 656.895 38.884 44.201 63.600 392.984 \n", "14 1501.042 18.9167 28.855 746.906 39.484 44.870 100.833 336.191 \n", "15 1703.727 21.0112 124.879 835.845 39.781 44.986 114.075 329.802 \n", "16 2195.653 21.4697 91.218 864.649 40.273 44.849 122.865 669.489 \n", "17 2136.900 23.3254 37.600 918.600 39.382 44.700 86.400 622.200 \n", "18 2199.500 25.2654 26.500 999.500 39.560 44.700 69.700 564.300 \n", "19 1515.000 23.8574 46.300 845.100 35.423 44.900 69.000 85.000 \n", "20 1442.100 25.0847 88.400 865.800 34.515 44.900 12.000 136.100 \n", "21 1504.100 26.6112 33.600 914.200 34.354 45.200 2.000 155.300 \n", "22 1524.700 26.9703 22.000 936.300 34.716 45.300 0.000 177.200 \n", "23 1517.200 26.0406 17.400 905.900 34.788 45.300 0.000 141.700 \n", "24 NaN NaN NaN NaN NaN NaN NaN NaN \n", "25 1549.913 12.6376 34.063 644.051 50.963 0.581 0.000 230.000 \n", "26 1521.153 13.2923 18.582 687.050 51.688 0.517 0.000 172.500 \n", "27 1548.670 14.0406 23.942 739.025 52.635 0.526 0.000 140.500 \n", "28 1879.598 14.9230 22.124 795.886 53.333 0.533 0.000 300.000 \n", "29 1869.251 15.6340 28.052 850.398 54.394 0.544 0.000 215.000 \n", "\n", " dvc ebit ... age tridx mb cap_int lvg \\\n", "0 NaN NaN ... NaN NaN NaN NaN NaN \n", "1 5.621 43.388 ... 204.0 1.070373 3.722721 0.068280 0.030998 \n", "2 6.201 43.754 ... 216.0 1.017457 2.843429 0.062569 0.000644 \n", "3 6.067 32.715 ... 228.0 1.494462 3.988689 0.108999 0.000000 \n", "4 5.935 23.971 ... 240.0 1.646141 4.117600 0.200658 0.037346 \n", "5 8.840 44.238 ... 252.0 1.706581 3.704230 0.073114 0.000000 \n", "6 7.428 55.803 ... 264.0 3.949485 7.147331 0.041965 0.000000 \n", "7 9.656 71.563 ... 276.0 4.185799 6.951689 0.069180 0.000000 \n", "8 11.857 71.695 ... 288.0 4.381591 6.880000 0.090044 0.000000 \n", "9 12.676 79.574 ... 300.0 6.286182 8.451389 0.103707 0.000000 \n", "10 13.653 74.148 ... 312.0 7.030023 8.110139 0.140552 0.000000 \n", "11 16.717 57.678 ... 324.0 6.776646 7.364927 0.120923 0.000000 \n", "12 NaN NaN ... NaN NaN NaN NaN NaN \n", "13 0.000 125.529 ... 444.0 0.386537 1.089755 0.019989 0.695064 \n", "14 0.000 95.415 ... 456.0 0.518011 1.214800 0.019223 0.585112 \n", "15 2.983 137.016 ... 468.0 0.695924 1.307398 0.073298 0.531052 \n", "16 12.081 142.360 ... 480.0 0.322468 0.892886 0.041545 0.916388 \n", "17 11.900 136.600 ... 492.0 0.546782 0.800844 0.017596 0.771391 \n", "18 11.800 142.600 ... 504.0 0.669914 1.108631 0.012048 0.634317 \n", "19 11.900 -8.600 ... 516.0 0.823134 1.164419 0.030561 0.182227 \n", "20 10.400 66.100 ... 528.0 0.688919 1.048049 0.061299 0.171056 \n", "21 10.200 77.200 ... 540.0 0.995869 1.241958 0.022339 0.172063 \n", "22 10.300 86.000 ... 552.0 1.283369 1.456788 0.014429 0.189256 \n", "23 10.500 110.700 ... 564.0 0.870483 1.433915 0.011468 0.156419 \n", "24 NaN NaN ... NaN NaN NaN NaN NaN \n", "25 25.271 114.309 ... 468.0 0.820994 1.507406 0.021977 0.357115 \n", "26 26.727 82.506 ... 480.0 0.973093 1.554283 0.012216 0.251073 \n", "27 28.152 118.882 ... 492.0 1.198713 1.873139 0.015460 0.190115 \n", "28 29.744 124.148 ... 504.0 1.101268 1.381760 0.011771 0.376938 \n", "29 31.309 111.711 ... 516.0 1.064461 1.276065 0.015007 0.252823 \n", "\n", " roa roe roi fyear current_roa \n", "0 NaN NaN NaN NaN NaN \n", "1 0.203129 0.079563 0.296192 2009.0 0.177459 \n", "2 0.177459 0.082621 0.234926 2010.0 0.136601 \n", "3 0.136601 0.047020 0.187547 2011.0 0.078142 \n", "4 0.078142 0.027727 0.114168 2012.0 0.141860 \n", "5 0.141860 0.053644 0.198710 2013.0 0.174277 \n", "6 0.174277 0.032012 0.228797 2014.0 0.189424 \n", "7 0.189424 0.036494 0.253696 2015.0 0.196381 \n", "8 0.196381 0.037149 0.255581 2016.0 0.208069 \n", "9 0.208069 0.030674 0.259235 2017.0 0.183631 \n", "10 0.183631 0.028326 0.229730 2018.0 0.138132 \n", "11 0.138132 0.023355 0.172009 2019.0 0.144608 \n", "12 NaN NaN NaN NaN NaN \n", "13 0.057096 0.109870 0.074914 2009.0 0.029731 \n", "14 0.029731 0.049185 0.041225 2010.0 0.040984 \n", "15 0.040984 0.063897 0.059932 2011.0 0.030844 \n", "16 0.030844 0.087720 0.044105 2012.0 0.025738 \n", "17 0.025738 0.074763 0.035675 2013.0 0.033144 \n", "18 0.033144 0.065790 0.046581 2014.0 0.006733 \n", "19 0.006733 0.010365 0.010967 2015.0 0.033077 \n", "20 0.033077 0.052568 0.047610 2016.0 0.037564 \n", "21 0.037564 0.049762 0.052828 2017.0 0.010232 \n", "22 0.010232 0.011437 0.014010 2018.0 0.004943 \n", "23 0.004943 0.005774 0.007159 2019.0 0.002116 \n", "24 NaN NaN NaN NaN NaN \n", "25 0.029314 0.046798 0.051981 2009.0 0.035692 \n", "26 0.035692 0.050842 0.063164 2010.0 0.041404 \n", "27 0.041404 0.046320 0.072904 2011.0 0.036446 \n", "28 0.036446 0.062292 0.062510 2012.0 0.033480 \n", "29 0.033480 0.057671 0.058740 2013.0 0.034399 \n", "\n", "[30 rows x 37 columns]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "compu_df[variables + ['fyear','current_roa']].head(30)" ] }, { "cell_type": "markdown", "id": "wound-cinema", "metadata": {}, "source": [ "I remove the first year (2008), because we do not have any observations there due to the lag procedure." ] }, { "cell_type": "code", "execution_count": 11, "id": "separate-genesis", "metadata": {}, "outputs": [], "source": [ "compu_df = compu_df[compu_df['fyear'] > 2008.]" ] }, { "cell_type": "markdown", "id": "caroline-index", "metadata": {}, "source": [ "We try to predict current ROA with the last year's variable values." ] }, { "cell_type": "code", "execution_count": 12, "id": "generous-pocket", "metadata": {}, "outputs": [], "source": [ "y_df = compu_df['current_roa']" ] }, { "cell_type": "code", "execution_count": 13, "id": "unlike-consortium", "metadata": {}, "outputs": [], "source": [ "x_df = compu_df[variables + ['fyear','ind','sp500']]" ] }, { "cell_type": "markdown", "id": "planned-nightlife", "metadata": {}, "source": [ "Train/test split" ] }, { "cell_type": "code", "execution_count": 14, "id": "prepared-shade", "metadata": {}, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split" ] }, { "cell_type": "markdown", "id": "permanent-journalism", "metadata": {}, "source": [ "Tensoflow does not like Pandas dataframes, so I change them to Numpy array.s" ] }, { "cell_type": "code", "execution_count": 15, "id": "derived-thousand", "metadata": {}, "outputs": [], "source": [ "# Split data into training and test sets\n", "x_train, x_test , y_train, y_test = train_test_split(x_df.values, y_df.values, test_size=0.20, random_state=1)" ] }, { "cell_type": "code", "execution_count": 16, "id": "loving-principal", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "numpy.ndarray" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "type(x_train)" ] }, { "cell_type": "code", "execution_count": 17, "id": "ultimate-address", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(14017, 3505)" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(x_train), len(x_test)" ] }, { "cell_type": "markdown", "id": "decent-niagara", "metadata": {}, "source": [ "Let's check that there is no missing values any more." ] }, { "cell_type": "code", "execution_count": 18, "id": "expressed-series", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "at 0\n", "bkvlps 0\n", "capx 0\n", "ceq 0\n", "csho 0\n", "cstk 0\n", "dlc 0\n", "dltt 0\n", "dvc 0\n", "ebit 0\n", "ibc 0\n", "icapt 0\n", "lt 0\n", "ni 0\n", "pstk 0\n", "pstkl 0\n", "pstkrv 0\n", "re 0\n", "sale 0\n", "seq 0\n", "costat 0\n", "prcc_c 0\n", "prcc_f 0\n", "sic 0\n", "mkvalt 0\n", "tobin 0\n", "yld 0\n", "age 0\n", "tridx 0\n", "mb 0\n", "cap_int 0\n", "lvg 0\n", "roa 0\n", "roe 0\n", "roi 0\n", "current_roa 0\n", "fyear 0\n", "dtype: int64" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "compu_df[variables+['current_roa','fyear']].isna().sum()" ] }, { "cell_type": "markdown", "id": "under-looking", "metadata": {}, "source": [ "### Densely connected network\n", "Let's build a traditional densely connected neural network. We could also use recurrent or LSTM networks, but we would need to reorganize data in that case.\n", "\n", "![image.png](./images/feed_forward.png)!" ] }, { "cell_type": "markdown", "id": "continued-prime", "metadata": {}, "source": [ "One way to define a neural network with Keras is a single **Sequential**-command that has the layers in a list as a parameter. The densely connected layers have **ReLU** as an activation function. The last layer has one neuron, because we want to have a single valua as an output (current ROA). There is also no activation function, because we want to have linear output. There is also a dropout layer to counter overfitting. For the first layer, we need to define the shape of our input." ] }, { "cell_type": "code", "execution_count": 19, "id": "absent-chinese", "metadata": {}, "outputs": [], "source": [ "model = tf.keras.models.Sequential([\n", " tf.keras.layers.Dense(128, activation='relu',input_shape = (38,)),\n", " tf.keras.layers.Dense(64, activation='relu'),\n", " tf.keras.layers.Dropout(0.1),\n", " tf.keras.layers.Dense(32, activation='relu'),\n", " tf.keras.layers.Dense(16, activation='relu'),\n", " tf.keras.layers.Dense(1)])" ] }, { "cell_type": "markdown", "id": "handed-atmosphere", "metadata": {}, "source": [ "You can check your model with the **summary()** -function. The model has 203 530 parameters." ] }, { "cell_type": "code", "execution_count": 20, "id": "constant-department", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential\"\n", "_________________________________________________________________\n", "Layer (type) Output Shape Param # \n", "=================================================================\n", "dense (Dense) (None, 128) 4992 \n", "_________________________________________________________________\n", "dense_1 (Dense) (None, 64) 8256 \n", "_________________________________________________________________\n", "dropout (Dropout) (None, 64) 0 \n", "_________________________________________________________________\n", "dense_2 (Dense) (None, 32) 2080 \n", "_________________________________________________________________\n", "dense_3 (Dense) (None, 16) 528 \n", "_________________________________________________________________\n", "dense_4 (Dense) (None, 1) 17 \n", "=================================================================\n", "Total params: 15,873\n", "Trainable params: 15,873\n", "Non-trainable params: 0\n", "_________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "markdown", "id": "quiet-ebony", "metadata": {}, "source": [ "With **compile()**, we build our neural network to be ready for use. For regerssion problems, **MSE** is the correct loss function. We measure our performance with **Mean Absolute Error**, because it is easier to interpret than MSE." ] }, { "cell_type": "code", "execution_count": 21, "id": "digital-bobby", "metadata": {}, "outputs": [], "source": [ "model.compile(loss='mse',metrics=['mae'])" ] }, { "cell_type": "markdown", "id": "blank-detective", "metadata": {}, "source": [ "Next, we feed the training data to our model and train it using back-propagation. Everything is automatic, so, we do not need to worry about the details. The training data accuracy rises to 0.986 = 98.6%. However, true performance needs to be evaluated using test data. We can save to **history** information about the training process. The model is trained with batches of 64 images. So, to go through all the images, we need 938 rounds (the last batch has 32 images). One epoch is one round of going through all the data." ] }, { "cell_type": "code", "execution_count": 22, "id": "annual-clone", "metadata": {}, "outputs": [], "source": [ "history = model.fit(x_train,y_train,epochs=150,batch_size=1024,validation_split=0.1,verbose=False)" ] }, { "cell_type": "markdown", "id": "introductory-deputy", "metadata": {}, "source": [ "The following code plots the progress of training. Within the code is info for different commands." ] }, { "cell_type": "code", "execution_count": 23, "id": "sustainable-memorial", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEGCAYAAAB8Ys7jAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABNLklEQVR4nO2deXiU1dm472dmMtkTshkQUFQURQTFFZFFQQTFBdRqrfqpqNWi/bXaz7q11rq2Vvu1dW21Vau0uFBFRa0bCmpdAHFhRwwiTMg6k4Qks53fH/POMIQJCclk3qXnvq5cJO9y3nPzzjxz5rzPOUeUUmg0Go3GfrjMroBGo9FoeoYO4BqNRmNTdADXaDQam6IDuEaj0dgUHcA1Go3GpugArtFoNDZFB3CNRqOxKTqAayyFiAwUkXYR2SwiHrPro9FYGR3ANVZjFvAy0Aicam5VQESyzK6DRtMZOoBrLIOIuIgF8MeBJ4DLO+zfQ0T+JiLVItImIqtF5JKk/fuJyHMiUi8i20TkcxGZbuy7SETCHcobJCJKRCYaf080/j5FRBaLSBtwqYiUiMhTIrJRRFqN614rItKhvHNEZIlRtzoRedU49yIRaRSRvA7H/1JE1nYsR6PpLvorqsZKTAOygVeBJcBtIjJEKfWNiOQC7wKtwA+Ar4GhQCmAiPQHPgC+AE4DtgAjgGgP6nEv8L/Al0DIqNOXwH1AAzAWeBioB/5mXP9i4M/Ar4ELiL23jgfcwFzg98DZxD6Y4h9WlwAPKT2fhaaH6ACusRKXA08rpcLAZhF5G7gUuBk4D9gHGKqU2mQc/3XSubMBBZyulGoxtq3vYT3uUEq91GHb3Um/bxCRI406/c3YdivwiFLqtqTjPo//IiJ/By7DCODAicCeSedrNLuN7kLRWAIRGQicQqz7JM4TwCXGw8zDgRVJwbsjhwMfJAXv3vBxh7q5ROR6EflMRGpFpBm4Atjb2L8HMBj49y7KfAQYKyIHGX9fBsxXSm1NQ301/6XoFrjGKswi1t2wrEOXsJv0PMxM1ZXS2QPKjh8C1wI3AD8FlgFNxu+ndPfiSqmvRGQxcJmI3E2sm2d6d8/XaFKhW+Aa00l6eHkncGiHn38Q61pZAgwXkUGdFLMEOFZE8jvZvxVwi0hl0rbR3azieOA1pdRflVLLlFLrgP3jO41W9CZgShflPAJcSMznO+CNbl5fo0mJDuAaKzCNWBfEI0qpL5N/iHWpTAE+BKqA+SIyWUT2EZFJInKOUcaDxF7PL4rIWGP/dBGZZuz/mFjL+W4R2V9EpgK/7Gb9VgMTReR4ETlARG4Hju5wzK3AD0XkFyJykIgcLCJXiUh50jHPGf/+AnhUP7zU9BYdwDVW4HLgI6XUxhT73iaW7XEeMIFYNsg/gZXAA0AugFJqC3AcsSC9APgKuAMQY3898H3gGGIPF38BXNfN+t1GLAPmRWIfJCXAH5MPUEo9ClwEnAV8BrxH7IMpnHRMG/B3Yu+7v3bz2hpNp4huBGg0mUNEngGylFIzzK6Lxv7oh5gaTQYQkRLgKGAGMMnk6mgcgg7gGk1mWAaUAb9VSr1ndmU0zkB3oWg0Go1N0Q8xNRqNxqboAK7RaDQ2JaN94AsXLlTZ2dmZvKRGo9HYnm3bttVOmjSpouP2jD/EPPDAAzN9ybRSVVXF3nvvbXY1eoV2sA5O8NAOfc/SpUurUm3PaBeKE6Y9zsqy//z+2sE6OMFDO5hHRgO42+3O5OX6hOLiYrOr0Gu0g3Vwgod2MI+MBvBwONz1QRantrbW7Cr0Gu1gHZzgoR3MI6N94LoFbg20gzVQSpGbm0sgEDC7Kr0iPz9fO6QJEaGgoKDb3c0ZDeBOGDQUDAbNrkKv0Q7WoLm5mYKCAvLy8ro+2MKEQiHb9iHHsYpDMBikubmZwsLCbh2f0S6UaLQnyxNai9bWVrOr0Gu0gzVQSjniW6kT3tdWcfB6vbvV0M1oALfCJ1xv6d+/v9lV6DXawTo44T2hHcwjowE8FApl8nJ9gs/nM7sKvUY7WAcz3xOnnXYab7311g7bHnroIa699tpOzzn11FNZtmwZAN/73vfw+/07Odx999386U9/2uW1X3nlFVatWpX4+84772ThwoW7aZA+7BqbMhrAXS77j9z3er1mV6HXaAfrYObYiJkzZzJv3rwdts2bN48zzzyzW+c/88wzFBcX98hhwYIFrF69OvH3jTfeyMSJE3e7nHTRE4dIJNIHNdk9HBnAS0pLKSktxbN4cdrL7u7DBSujHayDmX3gp59+Om+88UbigfDGjRvx+XyMGTOGa6+9lhNOOIExY8Zw1113pTx/1KhR1NXV4Xa7uffeeznyyCOZNm0a69atSxzzxBNPMGnSJMaNG8eFF17Itm3b+Oijj3j11Ve55ZZbGD9+PBs2bGD27Nm8+OKLALz77rtMmDCBsWPHctVVV9He3p643l133cXEiRMZO3Ysa9as2alOGzdu5OSTT2bixIlMnDiRjz76KLHvD3/4A2PHjmXcuHHceuutAHz99dfMmDGDSZMmMXHiRDZs2MDixYs599xzE+ddd911zJkzJ1GHX/3qV0ycOJEXXnghpR/A1q1bueCCCxg3bhzjxo3jo48+4s477+Shhx5KlHv77bfz8MMP7/6NS8LReeCujalW6OoddXV1aS8z02gH65D8nog3PFL9eB9/PHGc9/HHd3lsdykpKWH06NG8+eabQKz1fcYZZyAi3Hzzzbz99tssXryYDz74gK+++qrTcpYsWcK8efN49913mTt3bqKLBWJdLm+99RaLFi1i2LBhPPXUUxx99NFMmzaNW2+9lffee4999tkncXxbWxuzZ8/mscce4/333ycSifDXv25ffa6srIyFCxdyySWXcP/99+9Ul/LycubNm8fChQt57LHHuOGGGwB44403WLBgAW+88QaLFi3ixz/+MQCXX345s2bN4s033+S1116jsrJypzI7UlpaysKFCznzzDNT+gFcf/31HHvssSxatIiFCxdy0EEHcf755zN37lwg9tB03rx5fO973+vyersiowHc48nw1Ct98PW0pKQk7WVmGu1gHTL+nujAmWeemehGSe4+eeGFF5g4cSITJkxg1apVO/RXd+STTz7hlFNOIS8vj6KiIqZOnZrYt3LlSk4++WTGjh3Ls88+u8tyANatW8fee+/N0KFDATj33HP58MMPE/unT58OxFrC33777U7nh8NhfvKTnzB27FguvvjiRDfNu+++y3nnnZdI2SwpKaGpqYktW7Ywffp0PB4POTk53UrpnDFj+2p4nfktWrSISy65BIh9yyoqKmKvvfaipKSEzz//nLfffptDDjmE0t34wE1FRl89VknV6Q2tra0UFRWZXY1eoR2sQzQaTXSjNNTXd+uc4EUXEbzoorRcf9q0adx0000sX76c1tZWDj30UKqqqrj//vt566236NevH7Nnz050Y6RiV2lvs2fP5qmnnmLEiBHMmTOH999/v1f1jc9m6na7U36jf/DBB6moqGDRokVEo1EGDBjQrXKT74PH49khVrW1te1wbHKQ312/Cy64gDlz5rB161bOP//8btVtV+g88N2k4820I9rBOpj9nigoKOC4447j6quvZubMmQA0NTUlWtNbt25NdLF0xtFHH82CBQtobW2lqamJ119/PbGvubmZyspKQqEQzz777A7XbW5u3qmsoUOHsnHjRr7++msg9qD02GOP7bZPIBCgsrISl8vF3LlzEw8aJ06cyJw5cxJ91A0NDRQWFrLnnnvyyiuvEI1GaW9vZ9u2bQwaNIjVq1fT3t6O3+/nvfc6XwGvM7/x48cnun4ikUhilOf06dN5++23WbZsGSeccEK3vTrD2XngfdCF4oT8Y+1gHayQf3zmmWfy5ZdfJrpPRowYwciRIzn66KO5/PLLOfroo3d5/uGHH86MGTMYP3483/ve9zjssMMS+2688UZOPPFEpk2bxgEHHJDYPmPGDP70pz8xYcIENmzYkNiek5PD/fffz8UXX8zYsWMRES6++OJuu8yaNYt//vOfjBs3jrVr15Kfnw/A5MmTmTZtGieccALjx49P9J8//PDD/PnPf2bSpElMnTqVrVu3MmjQIM4444xEN8zIkSM7vV5nfnfddReLFy9m7NixHH/88YmuFa/Xy3HHHccZZ5yRlgfYGV0Tc+HChWrUqFE9Pr+tDVatcjNqVGSXsTn+IKfloYcInnNOj6+XCqvPG9wdtIM1CAQCZGdnY/dFTtrb27VDN4lGo0ycOJG//e1v7LfffimPCQQCO3UPLl26dMmkSZOO6HisrdIIL7kknxNOKOL557vXaomWlfXqeqnIyclJe5mZRjtYByeMjdAO3WPVqlUcfvjhjB8/vtPgvbtk9CFmb/+TXnstNnjj+ee9nHVW5yOnuvswqCfk5ub2WdmZQjtYBx38rEEmHA488MAdUizTQZe1FpEcEflYRJaLyFcicmuKY7JFZK6IrBORj0RkSKqy0pUHXlho3qyGDQ0Npl07XWgH6+CEOfK1g3l052OnHThBKTUKOBSYKiLHdDhmFtCglBoK/B74TaqC0pXzustBeEohmzYhmzZBHwx1LeuDbplMox2sg9l54OlAO5hHlwFcxYjn+2QZPx2bwKcDTxi/PwdMkhSTC/Q2Zeqee7YxfHiY44/fxcQzkQj9Ro6k38iReJ95plfXS0VTU1Pay8w02sE6WGE+jd6iHcyjWx87IuIGlgBDgQeUUh91OGQg8C2AUiosIn6gDNhhnaLeBvBZs9qZNavzAQUAJH8V6oOb4oSFBLSDdXDCIifawTy6FcCVUhHgUBHpB/xLREYopb7c3Yv5/X7Gjh2Lx+MhEokwc+ZMZs+ejc/nIz8/H7fbTSAQoKKigvr6epRSVFRUUF1dTUFBAbA9cb6mpgYRobS0lJqaGoqKiohEIrRu3Up8kLW/ro5oSwu1tbUUFxcTDAZpbW2lf//++Hw+vF4vhYWF1NXVUVJSQmtrK21tbYn9OTk55Obm0tDQQFlZGU1NTUQiEdrb2/H5fOTm5uL1evH7/ZSXlyem1oyfny6nlpaWRJlZWVkUFxf3yik/P5+qqqqEUzAYTOy3i1PyfUh1n+zg5PV6yc7OJhKJEI1GiUajZGVlEQqFcLlcuFwuwuFw4v2ilNphv4gQiUS6vR9iXQWhUAi32019fT1nnnkmIkJ1dTVutzvRNfX666+TlZW1Q51EJDEC8osvvmDu3LncfvvteDwe2tvbd9gfH82YaSeItaazsrIS/drxOrndbpRSKZ0g1g/e2f5MOrW0tOD1end47XXGbueBi8gvgW1Kqd8lbXsd+JVS6kMR8QA+oEJ1KLw3eeAffODh3ntz2HffCD/8YTtDh6ZuzUtjI/323ReAbffcQ/usWT26Xmc4If9YO1gDK+WB33333eTn53P11VcntsWDUldYLQ+8u/VOxkoOac0DF5EKo+WNiOQCJwIdZ6SZD/yP8ftZwNsdgzf0LlVn5Uo377yTxWOP5XDTTbuYcKaPu1CckL6mHayD1VLwZs+ezTXXXMPkyZO55ZZbWLJkCVOmTGHChAmcdNJJrF27FmCHKVfvvfderrrqKk499VQOO+wwHnnkkZRldzZF7dKlSznppJMYN24ckydPTnzT/cUvfsGxxx7Lcccdx5///Gdg+xS2AMuWLePUU08FYh9AV1xxBVOnTuWKK67YrWllN2zYwJQpUxL7169fb+rc5LtDdz6mBgBPGP3gLuAZpdTLIvJr4FOl1HzgMeDvIrIOqAfOTVVQbyav9/m2n2tMZ5Ca5ADeB6lBTlhIQDtYh+T3RGlp5zMs3ndfCxddFOv3f/xxL9dck9/psfX1vUux3Lx5M6+//nqiW2nBggV4PB4WLlzIbbfdxpNPPrnTOWvXrmX+/Pk0Nzdz1FFHcckll+w0TcDNN99MSUkJkUiEM844g6+++or999+fWbNm8dhjjzF69GgCgQC5ubk88cQTbNy4kffeew+Px9OttNHVq1ezYMECcnNz2bZtG/PmzSMnJ4f169dz2WWX8fbbb+8wrWxeXh4NDQ2UlJRQVFTEF198wSGHHMKcOXM477zzevV/mCm6DOBKqc+Bw1Js/2XS723A2V2V1ZsnvT7f9pbKtm27+CDo4wDu9/vp169f2svNJNrBOsT7Pq3E6aefnugXDgQCzJ49m/Xr1yMiKfOllVJMmTIl0R1UXl7O1q1bGThw4A7HxRdACIfDVFdXs2rVKkSEyspKRo8eDZDoOli4cCEXX3xx4v+mO9MHT506NfHNLBwOc9111/HFF1/gdrtZv349kHpaWYDvf//7PP3009xxxx3861//6nICL6uQ0VdOb16oyQG8paXzAK7Kymj93//FvX49ocmTe3y9zigvL097mZlGO1iH5PdEd1vOF10UTLTG+4Lk6VLvuusujjvuOP7+97+zcePGRJdFMi6Xa4dvRG63e6fG2u5OUdsZyVO9djw/ud67O63s6aefzu9//3vGjx/PqFGjej1Pd6bIaAdcb1rg1dXd7ELJzaXthhtoefRRosOH9/h6neH3+9NeZqbRDtbB6vnHgUAgEfziy4p1pDvpwZ1NUTt06FCqq6tZunRp4rhwOMzxxx/P448/nmjxx7tQ9tprLz777DMA5s+fv8t6d3daWYjNCnnCCSfws5/9zDbdJ5DhAN6bXMvutsD7GruuXp2MdrAOVs8/vvrqq7ntttuYMGFCrz5sOpui1uv18thjj/Hzn/+ccePGMXPmTNra2rjgggsYNGgQxx13HOPGjeO5554DYutT3njjjZxwwgm7nI51d6eVVUpx1llnISJpmac7U2R0Otn3339fDe9Bq1gpmDUrnw0bXCxf7iE7W7FlS2PKY6W6mvwf/xjXpk203HcfkS7mMt5drJRu1FO0gzUIBAIUFBRYLhNld4lGo45weOCBBwgEAtx0002m1mV30ggz2gfe01aTCPz1ry0oBZ984ia/8wfwuHw+st54A4Cs119PewD3+Xy2zz/WDtYhFArZ/oPICQ7nn38+Gzdu5MUXXzS7KrtFRgN4b1egEIGjjuria1zSU3LpgyyU/F19etgE7WAd7N5yBWc4PPHEE5ZYHWl3scX/fGOj8O23Lrr10LqPB/KkYxkks9EO1qE3YyOsgnYwD1tkoTz/vJdRo4q5/vo8fvvbHK66Km+HrJRkJPkafRDA44uT2hntYB2snoXSHbSDedhiUeP4KMz+/aO8+KKXOXOyqa3tpOp9PJCnoqIi7WVmGu1gDUTE8lko3cFqA5F6glUcgsHgbn0byGite7rqRTyFsLIySl5e7AXf0tLpRRK/Sh98qtbX1+8wYMCOaAdrUFBQwHfffbdTxoHd2LZtm+3vhVUcRKTLGQiTscbHThfEA/iAAYr8/FgA73Q4fR9/kjqhxaQdrEG8BW73AO73+7WDSdhiKH28vzu5Bd5ZAA+PH9+nixo74au7drAOTvDQDuaR0T7wnuaBx1vg/ftHiX/L2eVw+j6kurranAunEe1gHZzgoR3MI6MBvCepX8Eg1Na6cLkUFRUqqQ/cnLSf3emfsirawTo4wUM7mIfl88Ddbnj99QBz5jTjdsO++0YYPTpMv36p+0Cz5s+npLSUktJScm+4IcO11Wg0msxh+TxwtxuOPDLClCmx7JKf/KSdN99s4vTTU3fHSFJ6iuvbb3tW0V3Q3Nyc9jIzjXawDk7w0A7mYYs88N2ij0diVlZWpr3MTKMdrIMTPLSDeWQ0gPc0D7wjkUisb7zTnQZ9MRdKTU1N2svMNNrBOjjBQzuYh+X7wDvy1FNeKipKuPba1En30sctcLvOmZCMdrAOTvDQDuaR0QCejuGqOTldDOTp4wBul6WWdoV2sA5O8NAO5mGLPPBkuswD7+O5UOz6VSsZ7WAdnOChHczD8nngHelyJOaRR9L+ve8R2WcfQtOn9/p6HbHjcNuOaAfr4AQP7WAeXfZpiMhg4EmgElDAn5VSf+hwzETgRWCDsWmeUurXaa2pQVcDeSJHH822NK/Cs0P5Np12MhntYB2c4KEdzKM7LfAwcK1SajhwDDBbRFItbLlIKXWo8ZMyeKfjPym+EEunfeB9TEun0yDaB+1gHZzgoR3Mo8sWuFJqC7DF+L1JRFYCA4EVu3uxdOSBb+9CSb3ftWoV7hUrkHCYyIEHEhk5stfXTKZ///5pLc8MtIN1cIKHdjCP3eoDF5EhwGHARyl2jxGR5SLyqogcnOr8dDzErKiIct99Lfz6160p92fPnUvBpZeSf8UV5F95Za+v1xGfz5f2MjONdrAOTvDQDubR7bw+ESkAngd+opTquJ7VUmBvpVSziJwMvADs37GMhoYGxo4di8fjIRKJMHPmTGbPno3P5yM/Px+3200gEKCiooL6+nqUUlRUVFBdXZ2YbKa5uZlzz62kpqaGTZuE0tJSampqKCoqIhKJUFxfT45xvXBbGy0tLdTW1lJcXEwwGKS1tZX+/fvj8/nwer0UFhZSV1dHSUkJra2ttLW1Jfbn5OSQm5tLQ0MDZWVlNDU10dTURHt7Oz6fj9zcXLxeL36/n/Lycvx+P6FQKHH+7jhVVsacRHZ2amlpSZSZlZVFcXFxr5wikQhVVVUJp2AwmNhvF6fk+5DqPtnFqampiUAg0K3XnlWdRISqqqoevZ+s4tTU1ERjY2NaYkRfOHUal7szMb6IZAEvA68rpe7rxvHfAEcopWqTty9evFgdfHDKxnnayL3hBnIeeQSAyL77Evj007SW39LSYvsV0bWDdXCCh3boe5YuXbpk0qRJR3Tc3mUXisSGKD0GrOwseItIf+M4ROQoo9y6jselayj9nDleHnwwO/Uq9X08kKe2trbrgyyOdrAOTvDQDubRnS6UscAFwBci8pmx7UZgLwCl1MPAWcCVIhIGWoFzVYqmfTrywAFuuSWXujoXZ58dpKJix8skD6Xvi7lQiouL015mptEO1sEJHtrBPLqThbIY2GXOnlLqfuD+bpTV/Zrtgrw8RV0dtLYKsdT0JPq4BR7sdBYt+6AdrIMTPLSDeWR0JGY0Gk1LOfHh9ClTN5ODdh8E8NbW1NkvdkI7WAcneGgH88joosbpmg98V8Ppt919N6033ohr82ZUH3wtsmu+aDLawTo4wUM7mIftJrOCLuZDKSpCDRpE5KijiA4blpbrJWPXfNFktIN1cIKHdjCPjAZwlys9l9s+I2Hmh9N7vd6MXzPdaAfr4AQP7WAeGe1CSV8AV2Rnq5Sr8uT85jd4Pv4Y1+bNhA8/nG33d/lsdbcoLCxMa3lmoB2sgxM8tIN52HJJtccea2HLlkZOPXXnLhnP0qVkvfMO7tWr8c6dm5brJVNXt1N6u+3QDtbBCR7awTxstyIPwC4b8kn97BKJQJpSF+OUlJSktTwz0A7WwQke2sE8bJlGuEs6pg6m+Zp2TTdKRjtYByd4aAfzsGUA//vfvYwZU8T992fvvLNjN02aR2O2tbWltTwz0A7WwQke2sE8MhrA05UH7vcLq1e72bx55+rvNHw+zYN57Jovmox2sA5O8NAO5mHLPPD8/F3kgXcM2Glugds1XzQZ7WAdnOChHczDpmmEsX9TrcoTPuYYohUVuFeuRPXBp2pOTk7XB1kc7WAdnOChHczDpgG88xZ46+23p+UanZGbm9un5WcC7WAdnOChHczDlnngXa1M35c0NDRk/JrpRjtYByd4aAfzyGgLPF154LvqA5ctWwBQJSWxHPDs7C4Sx3ePsrKytJVlFtrBOjjBQzuYhy3TCAcOVFx6aRtnnLHzWPrCk0+m38EHU7LnnpQMHIjr22/Tcs04TU1NaS3PDLSDdXCCh3Ywj4y2wNMVwAcPjvLb36ZOvI+nEaqsLCQUSnsaoV0nfk9GO1gHJ3hoB/OwZR74Lon3s2dn7/h3mrBrvmgy2sE6OMFDO5iHLfPAARobhZdfzuKttzp8iYi3wPsogNs1XzQZ7WAdnOChHczDlvOBAyxa5OHCCwv405865G/GA7aR1ylpngvFrulGyWgH6+AED+1gHhkN4CLpS/s75phYoP70U0/yBISxGQjpuxa4XSd+T0Y7WAcneGgH88hoAI+k8YFiRYVi//0jbNsmLF/u3r6jj/vA/X5/WsszA+1gHZzgoR3Mw5Z54HGOOSbM2rVuPvzQwxFHxD4cmp98MpaJEo0igQDRIUPSes3y8vK0lmcG2sE6OMFDO5hHly1wERksIu+IyAoR+UpE/l+KY0RE/igi60TkcxEZnaqsdLbAAcaMibWu//Of7R8M4cmTCU2dSujkkwmeey4qzTfGrp/UyWgH6+AED+1gHt1pEoeBa5VSS0WkEFgiIm8opVYkHTMN2N/4ORp4yPh3B1SaV8dJDuDRaFoHXHZKOjNpzEI7WAcneGgH8+gygCultgBbjN+bRGQlMBBIDuCnA0+qWIT+j4j0E5EBxrkJ0p0HvtdeUQYOjNKvX5S6OqGiX5Cc3/4WcnKIDB+Oa+NGQiefTHTw4LRd0675osloB+vgBA/tYB671SktIkOAw4CPOuwaCCSPWd9kbNshgG/dupXLLrsMj8dDJBJh5syZzJ49G5/PR35+Pm63m0AgQEVFBfX19SilqKiooLq6moKCAgCam5uprKykpqYGEeHdd7fR1FSDSBFffdZK5b0vsl/eZpoPPZSCDz6gZuBAvotGKS4uJhgM0traSv/+/fH5fHi9XgoLC6mrq6OkpITW1lba2toS+3NycsjNzaWhoYGysjKampqoqalh2LBh+Hw+cnNz8Xq9+P1+ysvL8fv9hEKhxPk9dSotLaWmpoaioiIikQgtLS2JMrOysiguLqa2trbHTn6/n+zs7IRTMBhM7LeLU3V1deI+pLpPdnH6+uuvGTJkSLdee1Z1ikQiuN3uHr2frOK0fv169tprr7TEiL5w6jQmd7dbQ0QKgHeBO5RS8zrsexm4Wym12Pj7LeDnSqlPk49btGiRGjFiRLeut7usWOHiuOOKOYgVfFk0hvAxx+D9979p/sc/CJ10UtquU1tba9sHHnG0g3Vwgod26HuWLl26ZNKkSUd03N6tXmMRyQKeB57uGLwNvgOS+ykGGdsyRllZ7IOohgrweGI/kPa5UNxud9cHWRztYB2c4KEdzKM7WSgCPAasVErd18lh84ELjWyUYwB/x/5vSH8WSjLxAF5HGRG3F+I3JM154IFAIK3lmYF2sA5O8NAO5tGdPvCxwAXAFyLymbHtRmAvAKXUw8AC4GRgHbANuDhVQX05mZXHAyXFYRr8HuqknNI+CuAVFRVpLc8MtIN1cIKHdjCP7mShLAZ2OQbeyD6Z3VVZ6VqRpzMqSmMBvFoqKYl3oaR5LpT6+nry4oty2hTtYB2c4KEdzCOjQ+n7mrJ+sQ+ImuKhkJWF8nqRNOeepzuX3Qy0g3Vwgod2MA9bD6XvSNlALyyDqut+x+EzQ2x74IG0X8OuX7WS0Q7WwQke2sE8bDsfeCp+9rM2XnmlieOP77uumurq6j4rO1NoB+vgBA/tYB4ZbYH3darOyJF9l+USp6vEejugHayDEzy0g3k4qg/c/fnnFB9wAAVnn03O739P0ZgxeJ95xuxqaTQaTZ9g2/nAU7FmrZuf1d7A/WumIlu34l69GqmtTes1mpub01qeGWgH6+AED+1gHo5a1Li6xsPvuYZ/NU7qs5GYlZWVaS3PDLSDdXCCh3Ywj4wG8D7PAy9sBWBruGT7SMw0B/Campq0lmcG2sE6OMFDO5iHo/rAy/O3AVAbKkEZLXBJ84dGOtf1NAvtYB2c4KEdzCOjAby3eeBZzz1H0WGH4VqzJuX+0txtCFHqwsWExeiuSXMLvLS0NK3lmYF2sA5O8NAO5mGrPPCCyy/HXVVF3vXXp9zviYYoow6AumBhbGOaW+B2/aqVjHawDk7w0A7mkdEAnq488MioUam3Dx1KuTEroW/vI2i7/HLCRx6ZlmvGKSoqSmt5ZqAdrIMTPLSDeWR0IE9vCU6Zgvff/yZ8zDEp90f335/hE/LJ+yZM6KgxtF5yVNrr0NepkJlAO1gHJ3hoB/OwVR64tLcDoLKzOz3m0UdbePPNJkaM6Jsb0tLS0iflZhLtYB2c4KEdzCOjLfDe5oG7164FwFVVlXK/69tvcX/+OdHBg4lWVuLasAG1xx5E9923V9dNxq6LnyajHayDEzy0g3nY6iGma/NmALzPPZdyv+fddym44AI8D/+FyLxXKTr5ZLIfeqhX1+yIz+dLa3lmoB2sgxM8tIN5ZDSA9zbXcttvfwtAZNiw1AeEwzzKLArnPsl1L0+KXTPNWSh9PZo0E2gH6+AED+1gHrbKQlE5OQBIa2vK/RKJUIyfqHJR02KsrpHmAF5cXJzW8sxAO1gHJ3hoB/Ow1VD6RAA3HmbuRChEBbF8ztp4AE/zkmq1aZ4cywy0g3Vwgod2MA9btcALLr8cANmy04L3McLh7QG8WbfAO0M7WAcneGgH88hoAE/XunPSWTpiJMIebAWgtslorac5gAeDwbSWZwbawTo4wUM7mEdGA3g0Xd0ZnXwQSDhMKfUIURq25RDCk/a5UFo76X+3E9rBOjjBQzuYR5cBXET+KiJbReTLTvZPFBG/iHxm/Pyys7J69aQ3qSXdPGdOykParrySptUrKTOG03/9wvtsu+OOnl8zBXbNF01GO1gHJ3hoB/PoTgv8cWBqF8csUkodavz8urODepUHHh+FmZeHKitLbBa/H/cnn8T+yMtDVVRw2+1t/OUvzeSN3h81cGDPr5kCu+aLJqMdrIMTPLSDeXQ5ElMp9Z6IDEnHxVyunvfYJIbRe707bC+cMgX32rU0PfMM4cmTATjnnL7rz/J2uL4d0Q7WwQke2sE80jWUfoyILAc2Az9TSn2V6qD6+nrGjh2Lx+MhEokwc+ZMZs+ejc/nIz8/H7fbTSAQoKKigvr6epRSVFRUUF1dTXFzM/0AV2Mj2VdcwbqbbkJEOMQYXh996SWCq1eT+9prRC6+mOp+/ej/5JPI8OFsOOcciouLCQaDtLa20r9/f3w+H16vl8LCQurq6igpKaG1tZW2trbE/pycHHJzc2loaKCsrIympiaam5tpb2/H5/ORm5uL1+vF7/dTXl6O3+8nFAolzu/KKb4SdnNzM5WVldTU1CAilJaWUlNTQ1FREZFIhJaWlkSZWVlZFBcXU1tb22Mnl8tFVVVVwikYDCb228Up+T6kuk92cWpubiYQCHTrtWdVp6KiIqqqqnr0frKKU3NzM42NjWmJEX3h1BnSncwQowX+slJqRIp9RUBUKdUsIicDf1BK7Z+qnIULF6pRnUwF22Ud/H5ybruNnL/+lWhZGX4jcOdfcAHeV15h2z334Fq3jpxHHuH9Kx/lvdajOPbxqzh2PDS/8EKPrpmKqqoq9t5777SVZwbawTo4wUM79D1Lly5dMmnSpCM6bu91FopSKqCUajZ+XwBkiUh5qmN7syKPKi6m9ZZbAJC2tsT2aLyPOxRKZJy8tWEoNz5+CC8zPe1ZKCUlJWktzwy0g3Vwgod2MI9eB3AR6S/GJCcicpRRZl2qY3udRpibG/s3KYCrwtjKO9LUlMj53qM4tr+ayrTngds13SgZ7WAdnOChHcyjyyaxiPwDmAiUi8gm4BYgC0Ap9TBwFnCliISBVuBc1Um/TG8CuNTV4fn009jvkUisxZ2VRe699wLg2rIlkWpYWRy7GT76p30kZlvSh4dd0Q7WwQke2sE8upOF8v0u9t8P3N+di/UmD9z91VcUfD+pKm1tkFRe+Igj8Lz/PgB7lMQyVqqpTPtcKHbNF01GO1gHJ3hoB/Owz3zgHSawkra2HYJz8NxzE63tPfrF0girqUx7C9yu+aLJaAfr4AQP7WAeGV2Rp1d54ElfcYJTpoDLBdu2AbHBPbhcREaNItjcTNlBsYE+NVQQPGinxJlekWPMiGhntIN1cIKHdjAP2wTweAs8OHMmLY8+CoDUxGYelG3bcH/5Je0/+hHtP/oRAlRWRnG5BN9dD1JCeibRAsiNP0i1MdrBOjjBQzuYh23mA0+1oLEYLXCAvB/9aIfjV6zw89VXfkpK0he8ARoaGtJanhloB+vgBA/tYB4ZDeC9yQNP9IGHQrH5wNvaIGklaWlqQrZuRb77DtraEFTsnDSnB5UlzcNiV7SDdXCCh3YwD9tMJxvvA89+7jn6HXxwLKUwN5fwUUfF9jc3k3/llfQ75BA877+Pa9MmSgYMoNjYny6amprSWp4ZaAfr4AQP7WAetgng7RddROOXXxIePTq2obWV6D770GQMk5empu2jLj0e/vT3PRjIJv7QdEkva70jdp34PRntYB2c4KEdzCOjDzF7NR94Xh4qL4/ogAFA0rqY2dkojwcJBpF4l4rHQ1i52Ux/NoXSm99p13zRZLSDdXCCh3YwD/vkgceJP8Rsa4Omplift4H4/QAot5vKPWKtfV8k5bQsPcau+aLJaAfr4AQP7WAeGQ3gvUkj9P7tb+T/4Ad4Fi0CYn3i3vnz6TdyZGK+E4k/SfZ4qKiM/VodqehVnTti13SjZLSDdXCCh3Ywj4wGcGPOqx7h+eILvK++itTWxspqa0OMDJPwEUfgX7wYVWEEa4+HiopYC3xrNL0B3K4TvyejHayDEzy0g3lkNIBHejO1azwPvLg49ndbW2IkZviYY4gOH45yu2P7PB72GBD7sPCpPXp+zRT4jW4aO6MdrIMTPLSDeWT0IWZv8sDjaYTtF11E5IgjiBx0EN65cwFQxtefbffcgwQCRPbai/JsDyKKWioIhxvpTQp6MuXl6e1TNwPtYB2c4KEdzCOjATwdLfDI6NGETj4Z2D4S0zt/Pq5Nmwiefz7hKVOAmNjPftZGYaEiEiFtAdzv95Ofn5+ewkxCO1gHJ3hoB/PIaADvzvJtnRFvgScPpY+PsnSvXo179WrCRx4JY8Ykdt9wQ/rn+E1LJo3JaAfr4AQP7WAe9skDNxLt3WvW4Fm6lMghhyRa4NGSElwNDeTecQeuzZtpu+oqKCoi+7HHoL2d9ssvT1sT3K75osloB+vgBA/tYB62yQOPHH44ocmTcW3eTO5vfkPWggW0XXUVTf/8J6Hp0wFw1dWR+7vfJQb0bLh5Di/d/Dmrv0rfog52zRdNRjtYByd4aAfzyGgL3B3PEukB8QWNs557DoiNxIwedBDRgw7CvW7djgcbre1Ho5fwf1zFtc/7yX7DSzAIN97Yu24VO/aTdUQ7WAcneGgH88hoAE8LySMxDeILGycwAnh/91YIwb33Fyd2XXhhO4MG9bwvvjcfQlZBO1gHJ3hoB/OwTR64bNkSW8DBSLiXtjayH36YnDvv3GFecCCRD76nZ+tO5TQ09E45EAj06nwroB2sgxM8tIN5ZDSA9+YhZtGkSfQbNgzi0z62tZH9xBPk/u53qOzs7bMUQqIFfnrO69zCr3j+Lxup+cXdtP7PxRwyondrZFZUpHdkpxloB+vgBA/tYB62WZEn3mUSH4kpySMxjz+epjffRMUzTYwPioKsdn7FrZxwTIDy224g54nHkerqntcBqK+v79X5VkA7WAcneGgH8+gygIvIX0Vkq4h82cl+EZE/isg6EflcREanOq63JJZUKy0lWl6OKipKdJ2ovDwAogMGxKabNbpQVG4uKjcXiURQxkRaqpcrb/Qml90qaAfr4AQP7WAe3WmBPw5M3cX+acD+xs/lwEOdHdjjofRKJVrgkcMOw79mDc3PP5+YzErl5kIkQmDxYvxffRVbsR4IfPopjd99R7RfP+ZFz2CkfMGd9xR2epnuYNevWsloB+vgBA/tYB5dBnCl1HvArr5fnA48qWL8B+gnIgNSHdjjPPBQCFEKlZWVCM5Eo4kWuITDlFRU0G/EiJSnSyBAGzl8oUbw9ae9e1hR3csuGCugHayDEzy0g3mkI41wIPBt0t+bjG1bOh7Y2NjI2LFj8Xg8RCIRZs6cyezZs/H5fOTn5+N2uwkEAlRUVFBfX49SioqKCmrWr6cEUF4vVVVVVFZWUltVFduWnc03dXUcCtDcTNU339B/wAB8Ph9ZWVkUFxfTvGIFlcRukO+LGtrby/H5fHi9XgoLC6mrq6OkpITW1lba2tro378/Pp+PnJwccnNzaWhooKysjKamJlpbW2lvb8fn85Gbm4vX68Xv91NeXo7f7ycUCiXO35VTdXU1BQUFxKrdTGVlJTU1NYgIpaWl1NTUUFRURCQSoaWlJVFm3Km2tpbi4mKCwSCtra2J/d1xcrlcVFVVJZyCwWBiv12cku9DqvtkF6fW1lYCgUC3XntWdcrOzqaqqqpH7yerOLW2ttLY2Nij91MmnDpDutP3IyJDgJeVUjs1cUXkZeBupdRi4++3gJ8rpT7teOyiRYvUiE5aybu8fk0N/YYNI1pejn/ZMoqPPhpaWlClpeD1EvjwQ0pKSwGIVlbiX7kSgPwLL8S9ejVtV17Jhmse4xC+ZFheFR9uKtrtOsSpq6uz7QrWcbSDdXCCh3boe5YuXbpk0qRJR3Tcno4W+HfA4KS/BxnbdqKneeCqsJAmY+pYvF5cmzejPB78GzZsPyYrCwmFcCV9FXJt3Ih77VpcmzcnWuBb24qBnj+waG5utvSN7g7awTo4wUM7mEc60gjnAxca2SjHAH6l1E7dJ9CLPPCcHMInnkj4xBPB40G5XLFl1JLTEnNydj7PeGgaPvJIsh+/DzdhGqL9CLb3PIBXVlb2+FyroB2sgxM8tIN5dCeN8B/Ah8AwEdkkIrNE5AoRucI4ZAHwNbAO+Avwo87K6lUe+PYKQXz9uuTh9EYq4Q7EUweLi4mcNp0KiS3HVru256tv1NTU9Phcq6AdrIMTPLSDeXTZhaKU+n4X+xUwO201SoFs2kT2E08QHTKE4A9+gMrORlpaKNlrL4LTp9Py5JOJVXl2qJvRAhej6+aysucJ1/rJrZsA9OtZXXqxrqdV0A7WwQke2sE8MjoSs6d54O5vvyX33nvxPv10bEPSog7xXPD2K68EOrTEjet5n32W3F/9il/JrdzJTezRlpw0s3uUGg9L7Yx2sA5O8NAO5mGP+cDjXSVG4E5ubcd/D02YAEB04MDt5xkjMrNefZWcP/6RtquvpmHTJkInndSzemDfr1rJaAfr4AQP7WAeGQ3gPZ2yMTGM3nhQ2X7ppUT23Te2zZjHV5WV0XrzzbT98IeJ80KnnELbD3+YOOa7skNY+HERK1f2XLuoqOcpiFZBO1gHJ3hoB/PIaADvMR1a4O1XXJHoMok/0HRVVaGys4keeGDitPZLL6X1rrtQxjDZuUuGMXNmIU89lbSu5m7Sq4WZLYJ2sA5O8NAO5mGL+cATLfDkBY07TGSV9e675P3iF2S98cbO5zc2ArCHJzYjQMP8j3pUD4AWY7k2O6MdrIMTPLSDedhjPvAOLXD3smV4jaXVEg8tjUWPs/7978RprrVr8Xz4Ia716wEo3yt2fk1Nz7XtuvhpMtrBOjjBQzuYhz0eYmZnEx0wIDZ0Hsi59148X3xB+PDDCU+cCIBr82YA3MYweoDcO++k8JRTYoN+gPLD9gSgOtgvEfB3F7sufpqMdrAOTvDQDuaR0TUxe5prGTz3XILnnrt9g9ESb7viCsLHHguQMg+cpLTFyN57UzEkdkw1lbi2bCa69967XZferCpkFbSDdXCCh3YwD1tkoXQkno0iSSMxQ6ecAkBk//23H2dcr+XBBwksW0Z5BQhRaiknWpVyupYuKS4u7vogi6MdrIMTPLSDedhnSbVkjADufeYZZEts2pXwuHEE3n6bQPJDzPgHhvHw1OOBMm8TChcNq2p7dOna2p6dZyW0g3Vwgod2MA9btMBz7riD4gMPxPv448D2bJSsRYvwLF+eOC5y6KGQnM8Z70JJ6ntfeO4fqaGc/oG1PaqLXT+pk9EO1sEJHtrBPDIawHu67pz4/bi2bkWMB4+pRmKmxPjAyL/mGvIvuQSAvU4ZRuHFpxEZNbJHdQn28OGnldAO1sEJHtrBPDL6EDMajfbovHhfdyIPPCkfPOUshPF9yXOvGA9QE9PS9pBWY+4VO6MdrIMTPLSDedgjD9wYyBPv+2674gqi8cnXdxHA2376U9pmzQJA9esHwPPPZ3H22QU884y3R1Wxa75oMtrBOjjBQzuYhy3ywHcaiVlUhDLWittlC3zAAJQxUXvUCOCbNgpvvZXFpy/0LO/TrvmiyWgH6+AED+1gHhkN4C5XDy/XoQUOJFak32UfOLH+c4gt6gAw9phYd8zLr+URDu5+l47X27OWu5XQDtbBCR7awTxsEcATfeDGf7Lngw9wGdM/7qoFnjV/PjkPPBA7zgjghx/jYn/XOnwM4L2Xd7/fq7CwcLfPsRrawTo4wUM7mIct8sCD551H689/TtSYQlaMnM3QmDGwi/9495dfJn6P94GLwHmlrwLwzNzd75Ovq6vb7XOshnawDk7w0A7mkdEslJ6uyBM8++wd/k50m+TnJ7JLOrlg7PiCAiIjt6cNfn+/D7m19mpefq+M5uYmjO70blFSUtL9gy2KdrAOTvDQDuaR0RZ4T9MIdyL+MDNpKH1KjADedtllRPfZJ7F5yL4wlsVsa/fw8std9325Vq1C6mNT0do13SgZ7WAdnOChHczDFnngnjfeQMJhQuPHQ34+EggAkLV48S7Pi8+FIh26bqKVldzInWw+9WJOmj55l2W4qqooPvZYomVl+Neupa2rDw0boB2sgxM8tIN52CIPPP+aayj4wQ+QhobYhu6mIxoB3Dt3LiQFcdW/PyfzKufv8XqX3SfuL74AwFVXB0rZNl80Ge1gHZzgoR3Mo1sBXESmishqEVknIten2H+RiNSIyGfGz6WpyunxfODxNEIjCyU0dSrhESNo/clPuqo4AK6tW3foKw+eeSb+Tz+l9dZbAfjiCzebN3fSl568ilAwaNt80WS0g3Vwgod2MI8uu1BExA08AJwIbAI+EZH5SqkVHQ6dq5S6aldl9TqNMJ4HnptL03vvdXmeSm5eJ02kpcrKUMZIztdey+Lii/M54ogwL7zQTMf5tsR4Ot3+P/8D2dnkJOWi2xXtYB2c4KEdzKM7EfUoYJ1S6mulVBD4J3B6jy7W24E82bu3GHF4wgQAIoMHd3rM6NFhiosV77+fxdy5Oz/QdBkpi9HycgByuxg4ZAe0g3Vwgod2MI/uRNSBwLdJf28ytnXkTBH5XESeE5GUEbNHeeDRKBLvetnN0VLxxYzjOeDJZeZdfjkFM2awR3mEq6+OtfCXLNn5C0m8Be7+6ivcn35KQ7wf3sZoB+vgBA/tYB7pykJ5CfiHUqpdRH4IPAGc0PGgxsZGxo4di8fjIRKJMHPmTGbPno3P5yM/Px+3200gEKCiooL6+nqUUlQY3SDR7Gzq6utpbm6msrKSmpoaRITS0lJqamooKioiEonQ0tJC//798fl89Fu7liKg3bh2MBiktbWV/v37U/j663iamti2aROlpQXAfnz5ZYSqqqrE+Tk5OeRfcQXZbW3s8fe/0xaJELz9dtrb2/H5fOTm5uL1evH7/ZSXl+P3+wmFQonzO3WqqKC6upoCw213nLKysiguLqa2tpbi4uIdnHw+H16vl8LCQurq6igpKaG1tZW2trYdnHJycqiqqqKsrIympiaCwWBiv12cgsFg4j7k5OSQm5tLQ0OD7ZyCwSCBQCDlfbKLU1FREVVVVd167VnVKRgM7hQjuvt+yoRTZ0hXc3SLyBjgV0qpk4y/bwBQSt3VyfFuoF4ptdMM6e+995465JBDdnm9ncprbKTfvvsSLS7Gv2HDbp2bf8kleF94AYAGI487TtGYMbhXryawaBFVxSMYObIfFRVRVq/271SOe+lSiiZPJnzwwax55hkGDBiwW/WwGlu2bNEOFsEJHtqh71m6dOmSSZMmHdFxe3e6UD4B9heRfUTEC5wLzE8+QESSzU8DVpKCnuSBq6IiGtevJ/Cf/+z2ua033khk8GBa/vSnnetipA2Jz8fAgYr8fEVNjYuGhp2zUaL77QeAe8MGgvH+eBtj18nrk3GCAzjDQzuYR5ddKEqpsIhcBbwOuIG/KqW+EpFfA58qpeYDPxaR04AwUA9clKqsHuWBu1yoHg5zjQ4dSiBpybUd9hkB3OXzIQIHHBBh7Vo3mza5KCnZnjqYd/XVkJWFyslBtm1jzx7VxFrYNec1GSc4gDM8tIN5dCstRCm1QCl1gFJqP6XUHca2XxrBG6XUDUqpg5VSo5RSxyulVqUqp8d54H1AfJ5wV3U1APPmNVNV1cghhyTlfUejeOfOJfvxx4kMGwZAYMmSjNc13dg15zUZJziAMzy0g3nYYjrZviDRhWIE8OJitdO8WOL3I+EwqrCQyIEHAlCwZUtG69kX2DVlKhknOIAzPLSDeWQ0osquZg7MMJGDDiI4dSqRgw7aYXvyM11JygGPT2WbvXVrxurYV9h18vpknOAAzvDQDuaR0QAeSR6WbjLhCRNomTOH4EUXAVBdLUyYUMiYMUWJY+I54Kq8nPZLL6Vx/XqqjDU27Yzfv3Omjd1wggM4w0M7mIct5gPPBKWlipUr3YTDQktLbKrx5FGY8Qep5Tb9pE6m3BhVamec4ADO8NAO5vFf2wKHWP+3+/PPIRIhKwv23TeW5rhunTENrRHA4/OmgH0/qZPRDtbBCR7awTwyGsC7GjSUaQpPOomiiRMTU8YecEDsA2bNmlgAV6WlhI49NtFPnjd7NvudeiqyaZM5FU4TVsoG6ilOcABneGgH87DFfOB9RXj8eAA8xsyGw4bFA3jsvyV02mk0v/wy7VdeCYBrwwZyNmzAvW6dCbVNH3bNeU3GCQ7gDA/tYB4ZDeBW+5QLGbMVZr37LgAHHBDrQlm92p3y+KjREk9eLNmO2DXnNRknOIAzPLSDeWQ0gLs7TrZtMokW+H/+A+3tiRZ4Y2Ms3VFqaqClZfvxxsLI7s8/z3BN00t+fr7ZVeg1TnAAZ3hoB/OwzsgaE1AVFYSHD0daW/F88gnDhkU46KAI//d/2wAoOOssSgYPxv3ZZwBERo0CwGPzAG61D9Ke4AQHcIaHdjCP/+osFNi+6IPn3XfJzoZ33gkkslFctbWE8CQWc4gcdBDK48G1du0OLXO7ETAWhbYzTnAAZ3hoB/P4r36ICdv7weMPJuNp3itWuBjje4FTeWl7GmF2NqEDDkCUsnU/eEVFhdlV6DVOcABneGgH88joyJoercjTx4THjcO/fDnRDsuu9c8P8Kk6HDcRAqEWioypEmrOOIPSGTNQFp47uCvq6+vJy8szuxq9wgkO4AwP7WAe/9V94ADk5u4UvAHKI1s5lg8I4eWdd7Z/c6g75xzarr2W6F57ZbKWacVq+fg9wQkO4AwP7WAeGQ3gVh5Kj1K4VqxI/Cm1tZzKS0Bs5XqARx/N5vvfP4hZs/L5wx+yUy7+YAfs+nUxGSc4gDM8tIN5/FfngSeIRik47TSKjzsO15o1ALjq6hIB/I03sli/3sUNN+SyYkUW//qXl1tvzeOsmfnY8YO72phC1844wQGc4aEdzOO/Og88gctFdP/9Aci5/34AwocfzqCnfsG+A1qor3dRUyP87W8tXHZZA3/Z4wYq8bFseRb//rf1Hsx2RVcLpdoBJziAMzy0g3noPnCDttmzUSJ4n3kG8flQe+xB+ORpnHRG7EPntde8TJ8e4rrr6rhw3Fp+zm8AuOeeHFu2wjUajf35r88DjxPdbz9C06cjwSB5115Lzu9+h2vFCn7wg3b+8pdmfvrTNgCam5sJjxzJD3mEgXn1HHhghNZWkyu/mzQ3N5tdhV7jBAdwhod2MI+MPlW0Yh54Mm0//jHel17C++qr8OqrRAcPZvg5wxk+PJo4prKykvBRR1FEK6vbh6BOup9Q3qkoxU5LslmVSmM9UDvjBAdwhod2MI+MtsCtmAeeTOTww2m5/37arrqK1muu2Wm5NYCamhoiRx1F21VXkR9pIv+SS8h65RVuvDGXGTMK+N3vcnjjDQ+bN4tlu1ZqamrMrkKvcYIDOMNDO5iHhfP6zCF43nm73C8iIELrrbeiPB5y/+//oK6eV17JYtMmN+++u/1bRoF7G0OLtnLhfu/xo/1fQxUU8M6M3/GLX+Th9SryqtaQHWkhS8JkSQSXR7jnyDlUFrQQmjKFP1bNYOlSD66An/ZVGwmEcgkrF/28rYzZYw3XjngVgK3X3MLP794TAPfatUhDww51vuLAtzh8j41EDjyQ+Xv9iLlzB5GXlYVn2TJAxRYCVVCQ1c59Rz8NQPusWdz07FHU1wvy3WZcm75NHIdLOHnw58wYsgRVUMCSc2/nwQdjzwLcyz9DgiGIfxtRcNfh/6Q8u4nQlCk8+N0Mli93I42NuNeu2V5JpRhe9C0/OeBlcLnYet1tXP/bgTGnNWuQxgZAjHKFC/d+iWGDa4gcdBDzB1/JSy9lQSiEZ8mn8TsFIjGnY+aACO0XX8xNzxxJfb3g2vwdru++S7o+TNtzGTMGf4wqLGTJ+XfzwAM5sesvX460t4OKJsq968hnKM/yEzrpJB78bgaffeZG/I2416w1XiexT+/h/TbzkxGvx+7TT3/J9XcP2MEpHA7j8fgBxRUHvMno0g1Ehg9n/j5XM3++4fTZMoiqxPULvEF+f+w/Yvfp4ou5cW7caXPCKX79aQOXM2PAB6jcXJZccl/CyfPZZxAKJY4D4Y4jn6U8p5nwiSfy4MbTWL7cjauxIZGZFf//H95vMz8Z/ipEIlRffwe//OUg8vLycK9dgzQ0gkji2+gVB77F6LJvYvdprx/x0ktZSCiIe6nx2ovfJ08b9x3xJChF+2WXcdNzRyfdp807vJ6nGq89CgtZcvatPPhg3GkZBIMoJYnq3mU4haZMSThJYyPuNWt2KHNo/tdcd9jvY/fpmlu236e1a5HGxu0vEpV0nw4+mPlDrtr+2ou/nwwKPNtfe22XXpqYzTStKKW6/AGmAquBdcD1KfZnA3ON/R8BQ1KVs3jxYlVfX2/rn02bNm3/u65Otdx6q/K/8YZav75B/eUvTerHR76nJvCOKmdrPDKq67lTKVCR8nL14ouBxPZUP18zRClQ2372M3Xqqe2dHjeF1xJ/rHtvxS7LfJ4ZSoEKTp6sbrppW6fHlVGT+KPpqafUoEHhTo+9gTtiThUV6tlnd+30DXslnKZP76bTopXdczrxxPQ77bFH952uvTbtTu1Tpqibb+6m09NP79LpRm6POVVWdt/pf//X0k49uk9dOJ3Eq33qFHjmmV7FnTfffPPTVDFVuhqBJCJuYA1wIrAJ+AT4vlJqRdIxPwJGKqWuEJFzgRlKqXM6lrVw4UI1ypjRz65UVVWx9957d7pf6upw+XxIIED9N018szZKeWEbQ/ZoQeXlUTdpBqtWuQmFhNDiJQS3hQlH3YQiLlQwzMkHrKLAGyQyahTvtx/Bpk0uorUNFK75jKKcIC6JEmj1UpAd4vhh3wIQOPVsnn8ttmane8UKpKF+hzqN3+9bhvRrJDJ4MJ/tOY233mqkrLAQz8cfxzrujVZttifMOaNXARA+4QSe+3Q/tm0TXBs34t60EVxGGmg0ysGVWxm911bIzWXDuO8nRqtmffIRtAcT11YKZh62loKcMJFRo3g7cCTffutC6utxf/XVDg8O9ixtZfKI7yAaJXDyTJ5b0C/mtHIlUl+//b0BjCr/nFH7uYkOGsSyAdNYvtwNwSCeJUu2i0ejZLvDnDN6JaIUocmTeW7J0ISTa+PGHf6fRgysY/TeNai8PL4ZczbvvJOFCHg++gjCIZS4ERSoKDNGrtnBadOm7U6KJKfiJiYPq4rdp9O+x3OvlSbdpwZampsTU5lOGPYde5c1ER04kM8qp2x3+ugjcLtRrtj1s2nnnENXoEQIT5rE80uTnKqqdrj+iD1rGb1vPaqoiKojTk/cJ8/HH6HaQ9tvEjBz5GoKc0KEDz2Ut5uPjt2nhgbcK1Ykjkk4HbgR3G4CJ8/kz3MUZWVluFeuQOrrUZHtx04cupG9ypqIDh7MZwOm8tlnhtOnnxjXjl0/OyvKOUesBhFCJ5zA8x/vE3OqqtrpPh0yoIbRg6tROTlsmHh+ktPHYIw1EVEQVcwYtYaC7FDsPsWd6utjTkmv0SL3ZmYc2bD9Pr2a9H4yFjcHwOViwgGb2Ls0QHTQID6rnBJzam/H84nhZLymk197wZNOQg0aRE9ZunTpkkmTJh3RcXt3AvgY4FdKqZOMv2+ISau7ko553TjmQxHxAD6gQnUofNGiRWrEiBE9lrACDQ0NlBgLHNsV7WAdnOChHfqezgJ4dx5iDgS+Tfp7k7Et5TFKqTDgB8pwIFZOhewu2sE6OMFDO5hHRh9i1tbWMnbsWDweD5FIhJkzZzJ79mx8Ph/5+fm43W4CgQAVFRXU19ejlKKiooLq6urESKnm5mYqKyupqalBRCgtLaWmpoaioiIikQgtLS30798fn89HVlYWxcXF1NbWUlxcTDAYpLW1NbHf6/VSWFhIXV0dJSUltLa20tbWltifk5NDbm4uDQ0NlJWV0dTURE1NDYWFhfh8PnJzc/F6vfj9fsrLy/H7/YRCocT5VnXy+/20tLQknILBYGK/XZyqq6sT9yHVfbKL0+bNm/F6vd167VnVKX5cT95PVnHavHkzHo8nLTGiL5w6I6NdKO+//74aPnx415HewrS3t5OdnW12NXqFdrAOTvDQDn1Pb7pQPgH2F5F9RMQLnAvM73DMfOB/jN/PAt7uGLzBwpNZ7QZ2Xfw0Ge1gHZzgoR3Mo8sAbvRpXwW8DqwEnlFKfSUivxaR04zDHgPKRGQdcA1wfaqyGhP5lPblhRdeMLsKvUY7WAcneGgH8+jWSEyl1AKl1AFKqf2UUncY236plJpv/N6mlDpbKTVUKXWUUurrVOU4IYDPmzfP7Cr0Gu1gHZzgoR3MI6ND6e266kUyVp8OoDtoB+vgBA/tYB5dPsRMJwsWLGjKzs5enbEL9gH19fXlpaWltWbXozdoB+vgBA/tkBH2njRp0k7LBmU0gGs0Go0mfegFHTQajcam6ACu0Wg0NqXPAriIDBaRd0RkhYh8JSL/z9heKiJviMha41/rTkBAbDIvEVkmIi8bf+8jIh+JyDoRmWvkxlsaEeknIs+JyCoRWSkiY2x4H35qvI6+FJF/iEiO1e+FiPxVRLaKyJdJ21L+v0uMPxoun4vIaPNqvp1OHO4xXkufi8i/RKRf0r4bDIfVInKSKZXuQCqHpH3XiogSkXLjb0veh87oyxZ4GLhWKTUcOAaYLSLDieWIv6WU2h94i05yxi3E/yOW/x7nN8DvlVJDgQZglim12j3+ALymlDoQGEXMxzb3QUQGAj8GjlBKjQDcxAaUWf1ePE5sKuZkOvt/nwbsb/xcDjyUoTp2xePs7PAGMEIpNZLYTKU3ABjv73OBg41zHjRmMzWbx9nZAREZDEwBkqc7tOp9SE135gNPxw/wIrEpaVcDA4xtA4DVmapDD+o8iNib7ATgZWITr9YCHmP/GOB1s+vZhUMxsAHjgXXSdjvdh/hkaaXE5u95GTjJDvcCGAJ82dX/O/AIsWmadzrO7J+ODh32zQCeNn6/Abghad/rwBiz69+ZA/AcsQbNN0C51e9Dqp+M9IGLyBDgMGKLPVQqpbYYu3yAlRej+z/gOiC+KGYZ0Khio1Mh9cyMVmMfoAb4m9EV9KiI5GOj+6CU+g74HbGW0hZis10uwX73Ajr/f+/OrJ9W5BLgVeN32ziIyOnAd0qp5R122cYBMvAQU0QKgOeBnyilAsn7VOwjzpJ5jCIyHdiqlFrS5cHWxgOMBh5SSh0GtNChu8TK9wHA6Cc+ndiH0Z5APim+EtsNq/+/d4WI3ESsq/Rps+uyO4hIHnAj8Euz69Jb+jSAi0gWseD9tFIqPla1WkQGGPsHAFv7sg69YCxwmoh8A/yTWDfKH4B+xoyLEOti+S716ZZhE7BJKfWR8fdzxAK6Xe4DwGRgg1KqRikVAuYRuz92uxfQ+f/7d8DgpOMs7SMiFwHTgR8YH0RgH4f9iDUGlhvv70HAUhHpj30cgL7NQhFik1ytVErdl7QreebC/yHWN245lFI3KKUGKaWGEHsw87ZS6gfAO8RmXAQL1z+OUsoHfCsiw4xNk4AV2OQ+GGwEjhGRPON1FXew1b0w6Oz/fT5woZEFcQzgT+pqsRQiMpVY1+JpSqltSbvmA+eKSLaI7EPsQeDHZtRxVyilvlBK7aGUGmK8vzcBo433im3uA9B3DzGB44h9Pfwc+Mz4OZlYP/JbwFrgTaDU7AcB3XCZCLxs/L4vsRflOuBZINvs+nWj/ocCnxr34gWgxG73AbgVWAV8Cfyd2ELalr4XwD+I9dmHiAWJWZ39vxN7QP4AsB74gljGjVUd1hHrJ46/rx9OOv4mw2E1MM3s+nfm0GH/N2x/iGnJ+9DZjx5Kr9FoNDZFj8TUaDQam6IDuEaj0dgUHcA1Go3GpugArtFoNDZFB3CNRqOxKTqAazQajU3RAVyj0Whsig7gGo1GY1P+P1w+q5t5LNaIAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXMAAAEGCAYAAACXVXXgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA5jklEQVR4nO2deXxcVd3wv79MMslkkkyzTJNCoQWB2gUKWFmsAg+RAlIE+7rA41IVXPsojzwuLCroIz76uCEP6qMvIKC+LMpWKgiIVhYFpSmtXegq6TrpZJtJ0klmMjnvHzOTTpMJCTN35tx7Pd/PJ5/MnHvvOfebm/zm5tzfOUeUUhgMBoPB2ZTpPgGDwWAwFI4J5gaDweACTDA3GAwGF2CCucFgMLgAE8wNBoPBBZhgbjAYDC7ABHODwWBwASaYG1yLiNwpIr/XfR4GQykwwdxgMBhcgAnmhn9KRGSOiPxWRPrTX4+KyHFZ2+tE5OciEhKRIRHZLSLfz9r+VhF5XkT60l/rROR8PTYGA5TrPgGDodSIiA94EtgOnJ0u/i7wOxGZp5SKA98ATgUuAfYDM4H56ePLgZXAncCH08cvAA6WxsBgGI8J5oZ/Rv4VCAJvUkp1AojIZcCrwGXA3cAsYK1S6sX0MbuAP6df1wL1wEql1LZ0Wea7waAF081i+GdkPrApE8gBlFIdwJb0NoAfA+8WkQ0i8kMRuVBEytL79gC3AU+IyOMico2IzCmxg8FwGCaYGww5UEo9ARwN3ARUAb8E/iAinvT2jwFvAp4i1VWzQUQ+oel0DQYTzA3/lGwE5olIU6ZARJqBOcCGTJlSqlspdY9S6hPARaSC9rys7RuUUt9XSl0I3A58vFQCBsNYTJ+5we3UiMjJY8r+DISB+0TkC4CQegC6F7gPQERuAtaQCvwjwPuBfmBXOuvlY8CjwG7gCOBtQFuxZQyGiTDB3OB2TgfWjinbAiwBfgA8ky5bDVyQzmQBGAS+DswGksDLwIVKqYiIVAPHA/eSepDaBfwW+HyxJAyGyRCz0pDBYDA4H9NnbjAYDC7ABHODwWBwASaYGwwGgwswwdxgMBhcgAnmBoPB4AImTU0UkTuApcABpdSCrPLPACtIpW39Vin1xXT5tcAV6fLPpkfSjWP16tWqsrKycAODwWD4J+LgwYOdra2twbHlU8kzvxO4ldTkQwCIyL+Qmk1uoVJqSESmp8vnkZqoaD6pgRS/F5ETlFLJXBW/8Y1vfL0etqK9vZ1Zs2bpPo2CMA72wQ0exqH4tLW1tecqn7SbRSn1DNA9pvhTwLeUUkPpfQ6kyy8B7lVKDSml/kFqitHTctUrIlM8dftSUVGh+xQKxjjYBzd4GAd95NtnfgLwNhF5UUT+JCJvTpcfSWp4c4Y96bJxeDyePJu2D4FAQPcpFIxxsA9u8DAO+sh3OH850ACcAbwZuF9Ejn09FYTDYRYvXkx5eTnJZJJly5axYsUKQqEQfr8fj8dDNBolGAzS3d2NUopgMEhHRwc1NTUA9Pf309zcTDgcRkRoaGggHA5TV1dHMplkYGCAlpYWQqEQFRUVBAIBOjs7CQQCxONxYrHY6Hav10ttbS1dXV3U19cTi8UYHBwc3V5VVYXP56Onp4fGxkb6+voIh8PMmTOHUCiEz+fD6/USiURoamoiEomQSCRGj7erUyQSobKyctQpHo+PbneKU0dHx+h1yHWdnOK0c+dOZs+ePaXfPbs6JZNJPB5PXn9PdnHasWMHRx99tCUxohhOEzGl4fwiMhtYlXkAKiK/A76tlPpj+v2OdGC/EkAp9V/p8ieAG5VSfxlb57PPPqsWLFgwtthR9Pb2Mm3aNN2nURDGwT64wcM4FJ+2trY1ra2ti8aW59vN8jDwLwAicgLgBTpJLaV1mYhUisgxpCYj+muuCtwwJ0w8Hp98J5tjHOyDGzyMgz6mkpp4D3AO0CQie4AbgDuAO0RkAxAHlqtUdN4oIvcDm4BhYMVEmSwjIyPWGGgkFovpPoWCMQ72wQ0exkEfkwZzpdTlE2z6wAT730RqdZbXpNAnxsPD8OqrZRx3nL4PhZaWFm1tW4VxsA9u8DAO+tA2AjSRSBR0/JVX+jnttAAPPKAvjSgUCmlr2yqMg31wg4dx0Ie2YF5WVljTK1d6AfjFL/SNIvV6vdratgrjYB/c4GEc9OHYYJ5B59ij2tpafY1bhHGwD27wMA760BbMh4eHLanHos+EvOjq6tLXuEUYB/vgBg/joA9tobC8vLDlR6+6apD6+hEuvlhfGlF9fb22tq3CONgHN3gYB31oC+aFpibecEOMHTsifPjD+oK5U1OYsjEO9sENHsZBH44N5nZgcHBQ9ykUjHGwD27wMA760BbMC80zX77cz+mn17Fxo74Ju5yaj5qNcbAPbvAwDvpwbJ75o4962bbNw803V1l0Rq8fp+ajZmMc7IMbPIyDPhyfmtjfb0k1eVFVpe+DxCqMg31wg4dx0Ifjg7nOPHOfz6evcYswDvbBDR7GQR+OzzPXGcx7enr0NW4RxsE+uMHDOOjDsXnmGXQOGmpsbNTXuEUYB/vgBg/joA/HpybqvDPv6+vT17hFGAf74AYP46APxwbzGTNSx+tce9Wpk9hnYxzsgxs8jIM+rOnryINC88x/+ct+Hn20gpNPzrn2RUlwaj5qNsbBPrjBwzjow7F55qeckuSrXx3kne8srJ5CcGo+ajbGwT64wcM46GPSYC4id4jIgfQScWO3/YeIKBFpSr8XEblFRLaLyHoROXXChgt4cqkUPPNMOS+8oG/0Jzg3hSkb42Af3OBhHPQxlYh6J3DB2EIROQpYAuzKKr6Q1CLOxwMfB34yUaVSwJPLeBwuvbSWd7yjjpdf1hfQnTqJfTbGwT64wcM46GPSYK6UegbozrHpB8AXAZVVdglwt0rxAjBNRGbkqjeZzL+vO/v5xLe/rW+0ViQS0da2VRgH++AGD+Ogj7wegIrIJcBepdS6MXfYRwK7s97vSZftH1tHb28vixcvpry8nGQyybJly1ixYgWhUAi/34/H4yEajRIMBunu7kYpRTAYpKOjg0SiDkjNOTw0NMLu3bsRERoaGgiHw9TV1ZFMJhkYGKClpYVQKERFRQWBQIDOzk4CgQDxeJxYLDa63ev1UltbS1dXF/X19cRiMQYHB0e3V1VV4fP56OnpobGxkb6+PhKJBENDQ4RCIXw+H16vl0gkQlNTE5FIhEQiMXr8ZE41NTUA9Pf309zcTDgcLolTVVUV7e3to07xeHx0u1Ocsq9DruvkFKdEIkE0Gp3S755dnQKBAO3t7Xn9PdnFKZFI0Nvba0mMKIbThHFZKfWaO6SD92xglVJqgYhUA38EliilIiLyKrBIKdUpIquAbymlnksf9zTwJaXUS2Pr/NOf/qROOumkSdvORUeHMHfuNADOOivBww/rmaBl3759HHHEEVratgrjYB/c4GEcik9bW9ua1tbWRWPL87kzfwNwDJC5K58JtInIacBe4KisfWemy8YxlQ+RichOhLFoVoA8z0NfJo1VGAf74AYP46CP151SopT6u1JqulJqtlJqNqmulFOVUiFgJfChdFbLGUBEKTWuiwUKyzOPxw917SQS+oaAOjUfNRvjYB/c4GEc9DGV1MR7gL8Ac0Rkj4hc8Rq7PwbsBLYD/xf49EQ7FvLpl/0AtIDnqAXj1HzUbIyDfXCDh3HQx6TdLEqpyyfZPjvrtQJWTKVhjyf/lMJjjx3httv6ufLKGnT+R+T3+/U1bhHGwT64wcM46EPbcP5C8HrhoosSrF0boaoq/773QinkA8kuGAf74AYP46APbcP5C8kzB6ishFmzRmhu1hfMo9GotratwjjYBzd4GAd9OHJB502byrjiCj8/+IHe5Z2CwaDW9q3AONgHN3gYB304cqWhUKiMhx7y8p//6eOzn6228KxeH93duQbGOgvjYB/c4GEc9KFxnZ78yU5HfPppfROaF5IrbxeMg31wg4dx0Icjl43LTk3Umc3i1H/HsjEO9sENHsZBH46czzw7mOscAdrR0aGvcYswDvbBDR7GQR/agnkh6T/Z3SzDw/pGgE428Y0TMA72wQ0exkEfDu0zP/Ra5525wWAw2AVH5plPn644/fRUFNcZzPv79czWaCXGwT64wcM46MORCzqff36CJUsSLFtWQ0VFahm5AhYuypvm5ubSN2oxxsE+uMHDOOjDkXnmkAreDz3Uz/3392sJ5ADhcFhPwxZiHOyDGzyMgz4c2Wcei0E0qjctEQpbx9QuGAf74AYP46APR+aZ//CHVcyeXc9Xv+ojHBZGRiw8sddBQ0ODnoYtxDjYBzd4GAd9ODLPPHPoT39axZw504hE9HySOvXfsWyMg31wg4dx0Icj88yzVxoCfd0tdXV1ehq2EONgH9zgYRz0MZWVhu4QkQMisiGr7Dsi8oqIrBeRh0RkWta2a0Vku4hsEZHzi3HSY4O3rvTEQqfxtQPGwT64wcM46GMqd+Z3AheMKXsKWKCUOgnYClwLICLzgMuA+eljfiwiOW/BC/mBjV33U9co0IGBAS3tWolxsA9u8DAO+pg0mCulngG6x5Q9qZTK3A+/AMxMv74EuFcpNaSU+geptUBPy1VvYQs6H/5e1525Uxd+zcY42Ac3eBgHfVjRZ/5R4PH06yOB3Vnb9qTLxmHFA9CJ3pcKpy78mo1xsA9u8DAO+ihoBKiIXA8MA796vcf29PSwePFiysvLSSaTLFu2jBUrVhAKhfD7/Xg8HqLRKMFgkO7ubpRSBINBOjo6eM97GjnjjAquvz5ILOZh794O/P4hGhoaCIfD1NXVkUwmGRgYoKWlhVAoREVFBYFAgM7OTgKBAPF4nFgsNrrd6/VSW1tLV1cX9fX1xGIxBgcHR7dXVVXh8/no6emhsbGRvr4++vr6GBoaIhQK4fP58Hq9RCIRmpqaiEQiJBKJ0eMnc8pM7tPf309zczPhcBgRKbpTMpmkvb191Ckej49ud4pT9nXIdZ2c4tTX10c0Gp3S755dnUSE9vb2vP6e7OLU19dHb2+vJTGiGE4TxuOpTMQuIrOBVUqpBVllHwY+AbQqpQ6my64FUEr9V/r9E8CNSqm/jK3zueeeU/Pnz5+07dfiD38o5+BB4ayzEuh4AD0wMODYlbwzGAf74AYP41B82tra1rS2ti4aW55XN4uIXAB8EXhnJpCnWQlcJiKVInIMcDzw11x1FDqcH+Dcc4dZulRPIAfo7OzU07CFGAf74AYP46CPSbtZROQe4BygSUT2ADeQyl6pBJ5KD319QSn1SaXURhG5H9hEqvtlhVIqZ9pKIXnmDzxQwf79ZVx6aZyZM/Ut8RQIBLS1bRXGwT64wcM46GPSYK6UujxH8e2vsf9NwE1TqHeyXSbkrrsqee65CrZv9zBz5gjveU+cWbNKP6Y/PjatxoEYB/vgBg/joA9tU+COFDChSmYE6P33exkcFBYtGtYSzGOxWMnbtBrjYB/c4GEc9KFtOH8heeaZVESfTx32vtQ4NR81G+NgH9zgYRz04ciJtjL/Bfl8qe/JpJ4RoE7NR83GONgHN3gYB31oC+ZlZfk3nelmqa7We2fu9Xr1NGwhxsE+uMHDOOjDkcE8k9WY6WbRNZy/trZWT8MWYhzsgxs8jIM+HLlsXFUV+P2K6urUe12TnHV1delp2EKMg31wg4dx0IcjVxp6/vkou3f3Mm9eksbGEQpIWS+I+vp6PQ1biHGwD27wMA760BbMC0lNzPC97x1k27YI73qXnk5zp6YwZWMc7IMbPIyDPhwdzHUzODio+xQKxjjYBzd4GAd9ODLP/LTT6jjttDqGhiw8oTxwaj5qNsbBPrjBwzjow3F55krB9u0etm/3cPPNVSxYEODOO/WkEjk1HzUb42Af3OBhHPThuNTETOaKx6Po7xf27SsjGtUzaKiqqkpLu1ZiHOyDGzyMgz4cF8wzoz+9XqioyOSZ6wnmvswQVAdjHOyDGzyMgz4cl2eeWcy5okKRyW7UNWiop6dHT8MWYhzsgxs8jIM+HJdnnrkzr6hgNJjrGs7f2Niop2ELMQ72wQ0exkEfjktNPLybJfVaVzdLX1+flnatxDjYBzd4GAd9OG4+c78frrpqkOpqhcejd24Wp05in41xsA9u8DAO+pjKsnF3AEuBA5kFnUWkAbgPmA28CrxXKdUjqTXkfgi8AzgIfFgp1Zar3nzzzKdNU9xwQ2qE1gsveLj66hinn64nmjs1HzUb42Af3OBhHPQxlW6WO4ELxpRdAzytlDoeeDr9HuBCUos4Hw98HPjJRJUWMp95hjPOSPLlLw9y3nl6grlT81GzMQ72wQ0exkEfkwZzpdQzQPeY4kuAu9Kv7wIuzSq/W6V4AZgmIjNyNpxnamI0CqtXl7NunabZtbJwagpTNsbBPrjBwzjoI98+82al1P706xDQnH59JLA7a7896bL9jKG7u5vFixdTXl5OMplk2bJlrFixglAohN/vx+PxEI1GCQaDdHd3o5QiGAzypz/1sXz5CSxcOMh///cOwuEZVFR0MnfuEA0NDYTDYerq6kgmkwwMDNDS0kIoFKKiooJAIEBnZyeBQIB4PE4sFhvd7vV6qa2tpauri/r6emKxGIODg6Pbq6qq8Pl89PT00NjYSF9fH9FolEAgQCgUwufz4fV6iUQiNDU1EYlESCQSo8e/llNHRwc1NTUA9Pf309zcTDgcRkSK7pRMJmlvbx91isfjo9ud4tTb2zt6HXJdJ6c49fT0UFVVNaXfPbs6VVdX097entffk12cenp68Hq9lsSIYjhNhCilJo3cIjIbWJXVZ96rlJqWtb1HKVUvIquAbymlnkuXPw18SSn10tg6V69erRYuXDhp22P585/LWbq0ljPOSPChD8X59Kf9vO99Q/zkJwdfd12F0t7ezqxZs0rerpUYB/vgBg/jUHza2trWtLa2Lhpbnm9qYkem+yT9/UC6fC9wVNZ+M9Nl4yg0z9zrhfLyzLJxelITm5qatLRrJcbBPrjBwzjoI99gvhJYnn69HHgkq/xDkuIMIJLVHXMYyTyXB8o8N80eNKQrNTESiehp2EKMg31wg4dx0MdUUhPvAc4BmkRkD3AD8C3gfhG5AmgH3pve/TFSaYnbSaUmfmSieqfSvZOLzF2416t/OL8VGTm6MQ72wQ0exkEfkwZzpdTlE2xqzbGvAlZMpeF888yzh/MfGgGaV1UF49R81GyMg31wg4dx0Ifj5jPPvjPPjADV1Wfu1HzUbIyDfXCDh3HQh7bh/J48V2G+8MI4a9YMU1Wl2LIlVUee3e8F4/f79TRsIcbBPrjBwzjoQ1swz5eaGqipSc3rMm3aMOvX96Irxz/fDyQ7YRzsgxs8jIM+tHWz5JvNko3PBzNnKhob83uYWijRaFRLu1ZiHOyDGzyMgz4ct6Dz449X8JGP+Ln/fj3rfmYTDAZ1n0LBGAf74AYP46APx600tHVrGY884mXTJg/t7WV88IN+rrlGTz9Ld/fYKWuch3GwD27wMA76cFyfeTx+aNm4gQH47W+9vPGNSSBW8nPJN1feThgH++AGD+OgD8cuG5cazp96rSvP3Kn/jmVjHOyDGzyMgz4cm2duhwWdOzo69DRsIcbBPrjBwzjoQ1swzzf9x04jQCebktIJGAf74AYP46APbcE8XzI39F4vWWuA6hkBajAYDHbBcXnmb3jDCGedlWDmzBHtd+b9/f16GrYQ42Af3OBhHPShLZsl3zzzT35yiE9+cghILSF37rkJAgE9T5+bm5sn38nmGAf74AYP46APx+WZZ1NXB7/5TT+33z5gwRm9fsLhsJZ2rcQ42Ac3eBgHfTguzzwaBaUEv/9QNosuRJzfV28c7IMbPIyDPhyXZ/6pT/k55phpPPlkBUpBZ6cQCun54Tc0NGhp10qMg31wg4dx0EdBwVxEPiciG0Vkg4jcIyJVInKMiLwoIttF5D4RyTmJSr555tkjQAFOOGEa8+ZNQ8egLaf+O5aNcbAPbvAwDvrIO5iLyJHAZ4FFSqkFgAe4DPg28AOl1HFAD3BFruPzzTPPXgNU5NCizjoyWurq6krfqMUYB/vgBg/joI9Cu1nKAZ+IlAPVwH7gXOA36e13AZcW2MZhZOeZw6GBQzqW7bNiGl/dGAf74AYP46CPvIO5Umov8F1gF6kgHgHWAL1Kqcx98h7gyFzH5/sDG9vNkrnB13FnPjCgJ4vGSoyDfXCDh3HQR975ICJSD1wCHAP0Ar8GLpjq8ZFIhMWLF1NeXk4ymWTZsmWsWLGCUCiE3+/H4/EQjUYJBoN0d3ejlCIYDDIwUAmUE4tFaG/voqIiAAi7du3n2GMDhMNh6urqSCaTDAwM0NLSQigUoqKigkAgQGdnJ4FAgHg8TiwWG93u9Xqpra2lq6uL+vp6YrEYg4ODo9urqqrw+Xz09PTQ2NhIX18fyWSSoaEhQqEQPp8Pr9dLJBKhqamJSCRCIpEYPf61nDo6OkaHEPf399Pc3Ew4HEZEaGhoKKqT3++nvb191Ckej49ud4pT9nXIdZ2c4pRMJolGo1P63bOrU0NDA+3t7Xn9PdnFKZlM0tvba0mMKIbThDE53+keReQ9wAVKqSvS7z8EnAm8B2hRSg2LyJnAjUqp88cev3r1arVw4cLX3e5b3lLHK694eO65CPPmjTBnToBwuIzNm3tpbi7tU9D29nZmzZpV0jatxjjYBzd4GIfi09bWtqa1tXXR2PJCMrV3AWeISDWpycRbgZeAPwLvBu4FlgOP5Do431zOb3zjIL29wsyZqXVAdc6cmO8oVjthHOyDGzyMgz7yDuZKqRdF5DdAGzAMrAV+BvwWuFdEvpEuuz3X8flms5x77uFR++abB0gkhPr60ucmBgKBkrdpNcbBPrjBwzjoo6AxlEqpG4AbxhTvBE6b7FgrhvMDnHeeplm2gM7OTvx+v7b2rcA42Ac3eBgHfWgbEJ/vnfmPf1zJ8DBceeUQ1dUWn9TrxKmf4NkYB/vgBg/joA9twTzfB6///d9VRKNlLF8eBxR33+1l//4yPvzhoZI/AI1nVspwMMbBPrjBwzjoQ1swHxkZyeu47GXjAO64o5L168u54IIEzc2lTfaPxUq/iLTVGAf74AYP46APbRNt5fvEOHtBZziUzaJjBGhLS0vpG7UY42Af3OBhHPThqAWdk0lIJlN35pkud52piaFQqPSNWoxxsA9u8DAO+tAWzMvKXn/Th+ZlUWTS1A9NtFX6aXC93pwTQjoK42Af3OBhHPTh0GB+qEznOqC1tbWlb9RijIN9cIOHcdCHo5aNSyaFhoYR6usPPTzV2Wfe1dVV+kYtxjjYBzd4GAd9OGqloWnTFNu3R1i3LjpaVl8/wvTpI3g8sG1bGT//uZdSzWBZX19fmoaKiHGwD27wMA76cFxq4lh+9rODo68bGlIXweeDyy4rfq5oLBZz7ET2GYyDfXCDh3HQh7Y7c6uCeS42b85vdOnrZXBwsCTtFBPjYB/c4GEc9OG4PPOpUFtbmpGgTs1HzcY42Ac3eBgHfTgqzzwXX/6yj/nzAzz4YAUrVqQ+Ub3e0gRzp+ajZmMc7IMbPIyDPhyVmpiLSETYv7+M/n6hri4VxPv6SpNzXlVVVZJ2iolxsA9u8DAO+nB8MM/01nR3C9u3l1Fbq2hpKc2duc/nK0k7xcQ42Ac3eBgHfTgqzzwXmRGg69aV8+tfVzJ3bpIrrhiypO7J6OnpKUk7xcQ42Ac3eBgHfRQUzEVkmoj8RkReEZHNInKmiDSIyFMisi39PWfSZj555rnrSX0PhVIqwWDxsmTG0tjYWLK2ioVxsA9u8DAO+ij0zvyHwO+UUm8EFgKbgWuAp5VSxwNPp9+Pw6rUxEPBPNVPHggootHXOMBC+vr6StNQETEO9sENHsZBH3kHcxEJAGeRXuNTKRVXSvUClwB3pXe7C7g01/FWBfPMvOaZO/P/9/8queSS0syt4NRJ7LMxDvbBDR7GQR+F9HUcA4SBn4vIQmANcBXQrJTan94nBDTnOtiqPPO3vnUYjyfGb3/rHR0sFI2WJpvFqfmo2RgH++AGD+Ogj0KCeTlwKvAZpdSLIvJDxnSpKKWUiORMLTlw4AAf+9jHKC8vJ5lMsmzZMlasWEEoFMLv9+PxeIhGowSDQbq7u1FKEQwG6ejooKamBoD+/n7OPLOZ444Ls3790WzePA2ASCS1KOvAwAAtLS2EQiEqKioIBAJ0dnYSCASIx+PEYrHR7V6vl9raWrq6uqivrycWizE4ODi6vaqqCp/PR09PD42NjfT19REOh5kzZw6hUAifz4fX6yUSidDU1EQkEiGRSIwe/3qcmpubCYfDiAgNDQ2Ew2Hq6upIJpOWO0UiESorK0ed4vH46HanOHV0dIxeh1zXySlOO3fuZPbs2VP63bOrUzKZxOPx5PX3ZBenHTt2cPTRR1sSI4rhNBGS71qcItICvKCUmp1+/zZSwfw44Byl1H4RmQGsVkrNGXv8M888o0488cS82s7FxRfX8Pzzqbt9r1cRCvVaVvdEHDhwgOnTpxe9nWJiHOyDGzyMQ/Fpa2tb09raumhsed595kqpELBbRDKBuhXYBKwElqfLlgOP5DpexJqukF27ynjyyXKuvz7GX/8aASAeF0oxvYJTJ7HPxjjYBzd4GAd9FJrN8hngVyKyHjgZ+CbwLeA8EdkGvD39fhxJi+apfeyxCi67rJaHH/Zy3HEjNDamHqyWot88EokUvY1iYxzsgxs8jIM+Ckr2Vkq9DIy73Sd1l/7aDVucZ54Zg1RXp+jqSg3pnz69uCNBm5qailp/KTAO9sENHsZBH9rmM7fqzjwzAvSOO1LzKdx4Y4zh4dIMHopEIvj9/qK3U0yMg31wg4dx0Ie2YJ7vg9exZN/gP/98Bd/9bsySeqeCVTM/6sQ42Ac3eBgHfTh+PvPsapqaSjeUH5ybj5qNcbAPbvAwDvpw/HzmmW4WgKYmxbPPlnPLLZWsW1f81YacOu9xNsbBPrjBwzjoQ1sw93isCbbZ1QSDI6xaVcGNN1bzl78UvwfJif1qYzEO9sENHsZBH9qCuVUsWZLg8stTU942NanRBSpKkZpo1QeSToyDfXCDh3HQh7ZgblU2S3X1oX7zYHBkdP3PUqw2FC3V9IxFxDjYBzd4GAd9OP4BKMD8+UnOPz/OcceNlPTOPBgMFr2NYmMc7IMbPIyDPhy/0tCmTWWsXl3O3LlJ3va24dE781IE8+7u7qK3UWyMg31wg4dx0Ie2PHOr6O0t4/HHvfT2prJjSnlnblWuvE6Mg31wg4dx0Ie2O3OrhvN7PKkf/L59ZSiVWmmoulqNLlpRTJz671g2xsE+uMHDOOjD8XnmIyOpO/D29tQT6NNOS7JnTy/33jsAwIsveti/XyjGh21HR4f1lZYY42Af3OBhHPShrZvFqvSfSORQd8rYWXWHhuCii1JLyO3d20tlpSVNjjLZZPFOwDjYBzd4GAd9OD7PfP78VIpj5sFnNv/4RxkjI0J5eSqwGwwGg1txfJ75UUeN8Le/RVi/PjUHsVJwxhl1zJsX4JVXUnf/8biwapX1E8739/dbXmepMQ72wQ0exkEf2rpZrMwzf8MbDk2wJQL795fR1yesXXtIrxjZLc3NOdeqdhTGwT64wcM46MPxeea5yKQntrUd6peP9lg/o2I4HLa8zlJjHOyDGzyMgz4KDuYi4hGRtSKyKv3+GBF5UUS2i8h9IlLyBfUOBfNDd+b9O62/QFatY6oT42Af3OBhHPRhxZ35VcDmrPffBn6glDoO6AGuyHWQVXnmucgE81js0EXpi1r/T0hDQ4PldZYa42Af3OBhHPRRUIQTkZnARcBt6fcCnAv8Jr3LXcCluY4t5moemcyWH/1ogE8f/TBQnD5zp/47lo1xsA9u8DAO+ij0dvVm4ItApkO6EehVSmU6xPcAR+Y6sJjTTGbuzMvLYemunwLFmUWxrq7O8jpLjXGwD27wMA76yLuvQ0SWAgeUUmtE5JzXe3xXVxeLFy+mvLycZDLJsmXLWLFiBaFQCL/fj8fjIRqNEgwG6e7uRilFMBiko6NjNKm/v7+f5uZmwuEwIkJDQwPhcJizz55Bc3MFger9vIk1rORiKt+2lIGBi+js7CQQCBCPx4nFYrS0tBAKhfB6vdTW1tLV1UV9fT2xWIzBwcHR7VVVVfh8Pnp6emhsbKSvr4/e3l6qq6sJhUL4fD68Xi+RSISmpiYikQiJRGL0+EKd6urqSCaTDAwMjNZZUVFBIBAoyGloaIhoNDrqFI/HR7c7xam7u3v0OuS6Tk5xCoVCeDyeKf3u2dXJ6/XS3t6e19+TXZxCoRAiYkmMKIbThDE530llROS/gA8Cw0AVUAc8BJwPtCilhkXkTOBGpdT5Y49fvXq1WrhwYV5tT4Vbbqnkb8+P8PmnlnIOf+LgN77B0Kc/bWkb7e3tzJo1y9I6S41xsA9u8DAOxaetrW1Na2vrorHleXezKKWuVUrNVErNBi4D/qCUej/wR+Dd6d2WA4/kOt7KPPNcPP98Bb99qoYe6gGQgQHL23Dqwq/ZGAf74AYP46CPYuSZfwm4WkS2k+pDvz3XTsV8ABoKCU89lfqwOJadfI2vcv32nEk1BbbjzIVfszEO9sENHsZBH5bkByqlVgOr0693AqdNdkwxczmfeOLQXf8JbOVGvga/hut+3IOVz12L/d9FKTAO9sENHsZBH9pGgBYzm6Wr65CWj0FqPakuFqt7WgKBgLUVasA42Ac3eBgHfbhyOP/FF8cBuOj0/YxMm0bdSC9gfa55Z2enpfXpwDjYBzd4GAd9OH4+81wcf/wIW7f2Ul9fxcAffkrgfb3s5ch0MLdulQqnfoJnYxzsgxs8jIM+tAXzYq+z19SUrr+mhgCp6XGtvjOPx+OW1qcD42Af3OBhHPShrZtlZMT6WQzHoRTK76eOKGD9KNBYLGZpfTowDvbBDR7GQR/agnkpnhhXffOb1J1zDkezi2PL28ctK1coTs1HzcY42Ac3eBgHfTh+QefXQtIrhvyMT7Ct4TTe/nZrH7o6NR81G+NgH9zgYRz0oS2Yl5UVv+nsUZ/FGAHq9ZZ8qnbLMQ72wQ0exkEf7g7m2Wv5HTwIFvfT19bWWlqfDoyDfXCDh3HQhyvzzDNkgvn/vv/31NcNc+31fkvr7+rqsrQ+HRgH++AGD+OgD23BvJgrDY2S6VppaiQSKaO319onoPX19ZbWpwPjYB/c4GEc9OHq1MTMnXltY+qDw6Qmjsc42Ac3eBgHfWgbNFSKYD541VWU7dtH0xP3AwuIhgatrX/Q2vp0YBzsgxs8jIM+tAXzUuSZJ5YtAyBw778D0Be1dtSpU/NRszEO9sENHsZBH67OM89QV5v6LyDaZ+18ME7NR83GONgHN3gYB31ouzMvemriyAjeu+9G1dZSm16fNXqwHLDuQ6SqqsqyunRhHOyDGzyMgz4KWdD5KOBuoJnUVIQ/U0r9UEQagPuA2cCrwHuVUj1jjy96MB8YwH/11aiaGtQF7+QbXI//HUuAkyxrwufzWVaXLoyDfXCDh3HQRyERdRj4D6XUPOAMYIWIzAOuAZ5WSh0PPJ1+P/7gIueZZzJZlN9PRV0V1/NNPv6mv1raRk/PuM8ox2Ec7IMbPIyDPgpZ0Hm/Uqot/boP2AwcCVwC3JXe7S7g0lzHFzvPPDN8X/n94E8PFjp40NI2GhsbLa1PB8bBPrjBwzjow5K+DhGZDZwCvAg0K6X2pzeFSHXDjKPYqYmjd+Y1NQyfdBKPvfVr/GLfeZYOHOrr67OsLl0YB/vgBg/joI+Cb49FpAZ4APh3pVQ0e6FmpZQSkZz5gF1dXSxevJjy8nKSySTLli1jxYoVhEIh/H4/Ho+HaDRKMBiku7sbpRTBYJCOjg5qamoA6O/vp7m5mXA4jIjQ0NBAOBymrq4O77591AEjPh/bFy3i89+fw+af+fjlWVt4y1u8xONxYrEYLS0thEIhvF4vtbW1dHV1UV9fTywWY3BwcHR7VVUVPp+Pnp4eGhsb6evrIxwO09DQQCgUwufz4fV6iUQiNDU1EYlESCQSo8db4ZRMJhkYGBits6KigkAgQGdnJ4FAIC+nvr4+4vH4qFM8Hh/d7hSn7OuQ6zo5xSkcDuP3+6f0u2dXp2QySXt7e15/T3ZxCofD+Hw+S2JEMZwmjMWFrPgjIhXAKuAJpdT302VbgHOUUvtFZAawWik1Z+yxzz//vJo3b17ebU9G+ZNPUnvZZSTOO4/+++7j4otreP75Ch56qI+zz7amv35oaIjKykpL6tKFcbAPbvAwDsWnra1tTWtr66Kx5Xl3s0jqFvx2YHMmkKdZCSxPv14OPJLr+GLnmR/WZ37wIHVlqW4XK4f0OzUfNRvjYB/c4GEc9FFIN8ti4IPA30Xk5XTZdcC3gPtF5AqgHXhvroOLnZqYeNe76Fm6FIaHqVi9mqZnh4EPWboOqFNTmLIxDvbBDR7GQR95B3Ol1HPARJGxdbLjxeo13HJRUQEVFel1QHcD1i7q7NRJ7LMxDvbBDR7GQR/ahvMnk8mStaX8fgJEAGu7WSKRiGV16cI42Ac3eBgHfbh2PvPKH/2I2re/nYoHHkBVVxclmDc1NVlWly6Mg31wg4dx0Idr78w9O3dS3tZGWW8v1NTwGf6HvhnH8bWvWTdXsVM/wbMxDvbBDR7GQR/agnkhKZET4f3lL6k7/XQqb711dJUh5fej/H58DFI92I2VXfWlnPmxWBgH++AGD+OgD23BvCjzmQ8P49m2Dc+GDYeNAFXp4fwyMEA0Cle/9RUG3v1pKHB+GKfOe5yNcbAPbvAwDvpw7Hzm3rvuInDssZRt2gSAZ8MG/FdfnXq9adPheeZeL30PP0zfqlV88fM+7tx0Jv/6h08hL7UVdA5OzUfNxjjYBzd4GAd9aAvmHk9hC0X4P/c5ynp7qb7uOgDKtmw5VPeWLUh65jPl94MIw2edRXLRIr7+gfVMp4NnOYuXnyxsdjR/ZgIvB2Mc7IMbPIyDPrQF84LI6m+X9MMKz7Zth8oSCcrXr0/tOmY+gyPD6/k/PADA06sLm4S+0A8kO2Ac7IMbPIyDPpyZzXLwIMOnngpAWfpfIs/WrYfXf+yxDF1+OSqdZlR56634Pv95Rt74RlrfkdJ+atsb8j8HIBqNFnS8HTAO9sENHsZBH858AOr30/fUU4zU1VHW0YHs309Z+s48ce65AMTf+U4O/uhHqOnTAfA+9BBVd9wBQ0Oc9v2L8VaM8NeDC+juzj+9JRgM5u9gE4yDfXCDh3HQh7ZgXvBKQyIMfuELDNx8M1RV4dmxA4DBT32K2Je+ROL88w/bPTujpWZ6NWe+JYlSwh//mP/gpe7u7ryPtQvGwT64wcM46EPbgs6F4Fm/npHmZoZWrACgrL0dGRxkZMYMhltbGT7nHDxr1lC2eTMjc+cCoKqrAaj5wAeIXXcd73vfZ5gzJ8mcOfkvklGMXPlSYxzsgxs8jIM+HDmc3798OdPmzqXslVeAVKA++PWvM/jJTwIgvb3UXXABtRddlHVQ+s68rw/v/ffzgZYnuXXzEhatuinv83Dqv2PZGAf74AYP46AP5+WZR6N42ttRlZWMzJ5NxQMPUHnbbQz9278x9JnPAFD+3HMAqaH8aVRWulFy3jxIJKh49lnK//znvB06OjryPtYuGAf74AYP46APx+WZl2/cCEBy7lzwevFffTW+73wHyUr0r7rllnHHjQ3myZNOooPp/PxvJ/PT/81vysvJlnFyAsbBPrjBwzjow3F55p6//x2A5IIFUFbG8MKFAPi++U1ID+EfOfrocceNzJw5+jo5bx6quZkd9W/i00M/4Nrr/Nx556GAHo+nvisFTzxRwZ//7MhHCwaD4Z8Ix+WZHxbMgeTJJwNQ+ctfIululfh7U4sbJeccWnp06FOfGh1AlJw/H4A3Lxrmp3wCgC98oZqbbqriyiv9vPnNdezZIzzwQAWXX17DF7/oI9fp9qc/PJyMcbAPbvAwDvooWjAXkQtEZIuIbBeRa8ZuzzfP3JPpZjnxRABGZs0a3aaOOAKAxIUX0vfgg/StXDm6rWz3bqS/n5Hp00cHEg2fdBJXcAfXHXUnyaTwve/5ePBBL7t3e/j1rytZujTBUUcl2bSpnPvvH98V09zcnJeDnTAO9sENHsZBH0UJ5iLiAX4EXAjMAy4XkXnZ++SVZz48jGfz5tTL9N11YsmS1Pe3vAWy1hUdPuccVPZT6aEh4uefPzqoCCD+/vcz0tTEf+7+KNe9dyOXXz7E977Tz3N/7ORznxukqgquv34QgJtuqmLnC13ZMwkQe/RRBvZG2LKljEikBMvgFYFwOKz7FArGDQ7gDg/joI9idQafBmxXSu0EEJF7gUuATQXVWl5OZPNmyrZuhbo6AEaOOoreTZsOe8CZi5ETTmDgnnsOL5s9m77HH6f8pZf4/PuOAA5S8bvfUfMv/8rIEUeQnDOHDzUG+cm0b7Fu3zEsesdxHH10kltuOcjZJ4Y59hOf4EZu4GvcCMAJtXuZWx+ioiwJc46j7shqvvvdGBUPP0z5c8/xlZeW0TvkRyT1iSCA8vlInnIyS5YkWLJkGN811/D38Axu23z2OIfkMbNRM2bw1a/GqN/6Et777uNnm89hY/eRo3VmGD79DOYuUHz0o3Eqb72V6NYD3LjmXePqjNXWUXGij498ZIj59Xup+v73eXzXiTyx58Tx7S84kdqZNXzlK4OHOfXFfYetBqt81SRPOZnzzks7XXstGw40c9srOZxmH4M6YgZf+UqW06az2dRz5Lh9h8/I7ZRIJKio6Dx0XYNBRo4/Pj+n55/nK39bRl/i8Hl7xjldcw0bwi1Td9p8Dpt6jhjvlHWdgnffzVDHUM7rNM7pe9/j8d0n8cTutFPWz39Cp/gYp+oCr1MOp0Qigbz1uCn97pXkOuXhtO5A8LDfJ5j639NUnD7y5QZOuHDWuGMLRill+RfwbuC2rPcfBG7N3ufhhx9Wc+fOVSeeeKKaN2+e+vKXv6z279+v1q5dq7Zu3ap27Nih1q5dq/bs2aPWr1+v1q1bp/bt26fWrl2rtm3bprZt26bWrl2r9u3bp9atW6fWr1+v9uzZo9auXat27Nihtm7dqtauXTta54YNG9Tu3bvV2rVr1c6dO9Urr7xy2PaNGzeqXbt2qe3/8z9qxONRKvX8UylQ2zlWvZd7VYN0KVDqW9/qUtsee0xFTz5Z/crzAfUGtqlKYtmHKFCqqSmu9u/frw5cfrlSoJrZP26fzNdnPtOltm3bppJer1rJ0gn3A6VefrlDvfr1rysFaikrJ9zv7LOjateuXarv5JPVXma8Zp23335AbX/kEaVA3cANE+431qmFfZM7VVa6z6kI1ym6cKG5Tv8ETrd/eqXavXu32rBhw2ExaKpx7/e///1LueKuFGO0k4i8G7hAKXVl+v0HgdOVUv+W2Wf16tVqYToTxXYMD1PW3o5n61YkGoWREVR1NYOtS9i4s4amphGOOELR3t7OrGCQ8pdfJhHqZuOWStpDVSRVGfGTTqGqJcDSpQnK//IXPJs28Yu/zWMwkXpWoNK3Uqk781M45ZRhTj01iffnP2dX2M8TrxyLQg5bGSk5ezYjM47g8suHqN27lYpnnuF3m4+lvScwXuH00zn6GFiyZJiKRx8ltrubX760YNx+kcpK/G96E+edl2B2XRfeBx7gpV0trNkzY3yd8+fja6nj/e+Pjzrd/bcFxOKH/4OX7fSmN6Wc2sM1PPHKsePqHJk9m5EjCnMaGBg4bNrSkWATI8efkJ/Txo0pp8Thz3RK4dR7xx3UDHpyXqdxTg8+mHLaPX4hhSk7VftInmyt08DAAJXnnjul3z27XqdNe8rHTYM71b+nqTi9/RNHMvP0/BfAaGtrW9Pa2rpobHmxgvmZwI1KqfPT768FUEr9V2afZ599Vi1YMP6H4SR6enqor6/XfRoFYRzsgxs8jEPxmSiYFyub5W/A8SJyjIh4gcuAlZMc4ziKvSh1KTAO9sENHsZBH0UJ5kqpYeDfgCeAzcD9SqmN2fs49QeWzUB6aTonYxzsgxs8jIM+ija0USn1GPDYRNuLsqBziXHqwq/ZGAf74AYP46AP5020ZSOcuvBrNsbBPrjBwzjoQ1sw782a0dCpPPzww7pPoWCMg31wg4dx0IcJ5gXw4IMP6j6FgjEO9sENHsZBH9qCuVNX88im4KXvbIBxsA9u8DAO+ihKnvlUeOyxx/oqKyu3aGncIrq7u5saGho6J9/TvhgH++AGD+NQEma1traOWw5JWzA3GAwGg3U4bnEKg8FgMIzHBHODwWBwASUJ5iJylIj8UUQ2ichGEbkqXd4gIk+JyLb0d/tOiEBqnnYRWSsiq9LvjxGRF9MLcNyXnrrA1ojINBH5jYi8IiKbReRMB16Hz6V/jzaIyD0iUmX3ayEid4jIARHZkFWW8+cuKW5Ju6wXkVP1nfkhJnD4Tvp3ab2IPCQi07K2XZt22CIi52s56THkcsja9h8iokSkKf3eltdhIkp1Zz4M/IdSah5wBrAivVjFNcDTSqnjgafT7+3MVaSmJ8jwbeAHSqnjgB7gCi1n9fr4IfA7pdQbgYWkfBxzHUTkSOCzwCKl1ALAQ2ruH7tfizuBC8aUTfRzvxA4Pv31ceAnJTrHybiT8Q5PAQuUUicBW4FrAdJ/35cB89PH/Di9aI1u7mS8AyJyFLAE2JVVbNfrkJtizGc+hfnOHwHOA7YAM9JlM4AtOs5niuc8k9Qf3LnAKlLLAXQC5entZwJP6D7PSRwCwD9IP/jOKnfSdTgS2A00kJqOYhVwvhOuBTAb2DDZzx34KXB5rv10f411GLPtXcCv0q+vBa7N2vYEcKbu85/IAfgNqZubV4Emu1+HXF8l7zMXkdnAKcCLQLNSan96Uwiw8+J7NwNfBEbS7xuBXpWaVAxgD6lAY2eOAcLAz9PdRbeJiB8HXQel1F7gu6TuoPYDEWANzrsWMPHPPfOBlcEpPh8FHk+/doyDiFwC7FVKrRuzyTEOUOIHoCJSAzwA/LtSKpq9TaU++myZJykiS4EDSqk1us+lQMqBU4GfKKVOAQYY06Vi5+sAkO5XvoTUB9MRgJ8c/zY7Dbv/3CdDRK4n1Z36K93n8noQkWrgOuCrus+lUEoWzEWkglQg/5VSKjNetkNEZqS3zwAOlOp8XieLgXeKyKvAvaS6Wn4ITBORzMyTM4G9ek5vyuwB9iilXky//w2p4O6U6wDwduAfSqmwUioBPEjq+jjtWsDEP/e9wFFZ+9naR0Q+DCwF3p/+UALnOLyB1I3BuvTf90ygTURacI4DULpsFgFuBzYrpb6ftWklsDz9ejmpvnTboZS6Vik1Uyk1m9RDnT8opd4P/JHUeqdg4/PPoJQKAbtFZE66qJXUItuOuA5pdgFniEh1+vcq4+Coa5Fmop/7SuBD6WyKM4BIVneMrRCRC0h1P75TKXUwa9NK4DIRqRSRY0g9RPyrjnN8LZRSf1dKTVdKzU7/fe8BTk3/rTjmOgCleQAKvJXUv5DrgZfTX+8g1e/8NLAN+D3QoPshwhRczgFWpV8fS+oXdDvwa6BS9/lN4fxPBl5KX4uHgXqnXQfga8ArwAbgF0Cl3a8FcA+pPv4EqYBxxUQ/d1IP138E7AD+Tipzx64O20n1K2f+rv83a//r0w5bgAt1n/9EDmO2v8qhB6C2vA4TfZnh/AaDweACzAhQg8FgcAEmmBsMBoMLMMHcYDAYXIAJ5gaDweACTDA3GAwGF2CCucFgMLgAE8wNBoPBBZhgbjAYDC7g/wMUksyETa0H0QAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "plt.style.use('bmh') # We select as a plot-style 'bmh' that is in my opinion usually the prettiest.\n", "burnout = 25\n", "epochs = range(1, len(history.history['val_mae']) + 1) # Correct x-axis values (epochs)\n", "plt.plot(epochs[burnout:], history.history['val_mae'][burnout:], 'r--',label='Validation accuracy') # Plot epochs vs. accuracy\n", "plt.plot(epochs[burnout:], history.history['mae'][burnout:], 'b--',label='Train accuracy') # Plot epochs vs. accuracy\n", "plt.legend()\n", "plt.title('Accuracy') # Add title\n", "plt.figure() # Show the first figure. Without this command, accuracy and loss would be drawn to the same plot.\n", "plt.plot(epochs[burnout:], history.history['val_loss'][burnout:], 'r--',label='Validation loss') # Plot epochs vs. loss\n", "plt.plot(epochs[burnout:], history.history['loss'][burnout:], 'b--',label='Train loss')\n", "plt.title('Loss') # Add title\n", "plt.show() # Show everyhting" ] }, { "cell_type": "markdown", "id": "polyphonic-browse", "metadata": {}, "source": [ "**Evaluate()** can be used to evaluate the model with the test data. Acccuracy with the test data is 0.052." ] }, { "cell_type": "code", "execution_count": 26, "id": "central-player", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "110/110 [==============================] - 0s 248us/step - loss: 0.0066 - mae: 0.0521\n" ] } ], "source": [ "test_loss,test_acc = model.evaluate(x_test,y_test)" ] }, { "cell_type": "code", "execution_count": 27, "id": "generic-extraction", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.052059270441532135" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "test_acc" ] }, { "cell_type": "markdown", "id": "sophisticated-lincoln", "metadata": {}, "source": [ "Let's compare the performance to a linear model." ] }, { "cell_type": "code", "execution_count": 28, "id": "several-japan", "metadata": {}, "outputs": [], "source": [ "import sklearn.linear_model as sk_lm" ] }, { "cell_type": "markdown", "id": "visible-performer", "metadata": {}, "source": [ "We define our LinearRegression object." ] }, { "cell_type": "code", "execution_count": 29, "id": "endless-secretariat", "metadata": {}, "outputs": [], "source": [ "model = sk_lm.LinearRegression()" ] }, { "cell_type": "markdown", "id": "consistent-afternoon", "metadata": {}, "source": [ "**fit()** can be used to fit the data." ] }, { "cell_type": "code", "execution_count": 30, "id": "dress-sheriff", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "LinearRegression()" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.fit(x_train,y_train)" ] }, { "cell_type": "markdown", "id": "european-error", "metadata": {}, "source": [ "**coef_** -attribute has the coefficients of each variable and **intercept_** has the intercept of the linear regression model." ] }, { "cell_type": "code", "execution_count": 31, "id": "armed-latvia", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 5.31468126e-08, -1.05923685e-04, -3.53267974e-06, -5.74187084e-07,\n", " 4.89268554e-06, 3.28549174e-08, -1.02011246e-08, -4.32534321e-07,\n", " 3.03267883e-07, 7.33851624e-06, 5.42509020e-07, 5.14662474e-08,\n", " -1.07960714e-07, -8.41952205e-06, -1.13453398e-06, -1.67169301e-05,\n", " 1.25340601e-05, -1.00876074e-07, 8.42772444e-08, 3.58063483e-07,\n", " 7.40836707e-03, 6.03105032e-04, -4.75728895e-04, 1.11622895e-06,\n", " -2.38159257e-08, 1.54004100e-02, 9.43484048e-02, 1.94168470e-05,\n", " 2.44164138e-03, 3.21901086e-04, -3.24353490e-02, -1.33602510e-03,\n", " 4.25739547e-01, -3.32303633e-02, 3.77840060e-02, -2.76142065e-03,\n", " -2.97726456e-04, 2.84737219e-03])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.coef_" ] }, { "cell_type": "code", "execution_count": 32, "id": "presidential-narrative", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5.530386213093688" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.intercept_" ] }, { "cell_type": "markdown", "id": "bizarre-documentary", "metadata": {}, "source": [ "**score()** can be used to measure the coefficient of determination of the trained model. How much our variables are explaining of the variation of the predicted variable.*" ] }, { "cell_type": "code", "execution_count": 33, "id": "quiet-juvenile", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.42032935471032107" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model.score(x_test,y_test)" ] }, { "cell_type": "markdown", "id": "missing-asian", "metadata": {}, "source": [ "Mean absolute error." ] }, { "cell_type": "code", "execution_count": 34, "id": "confused-honolulu", "metadata": {}, "outputs": [], "source": [ "from sklearn.metrics import mean_absolute_error" ] }, { "cell_type": "code", "execution_count": 35, "id": "express-distinction", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.03547756137451534" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mean_absolute_error(y_test,model.predict(x_test))" ] }, { "cell_type": "markdown", "id": "foreign-doctor", "metadata": {}, "source": [ "As expected, the linear model performs better for this data." ] }, { "cell_type": "code", "execution_count": null, "id": "serious-jurisdiction", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.7" } }, "nbformat": 4, "nbformat_minor": 5 }