{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Deep Policy Gradient (PG)\n", "\n", "\n", "In this notebook, our goal is to implement the REINFORCE algorithm for policy gradient using [JAX](https://jax.readthedocs.io/en/latest/). We will apply this RL algorithm to control a single quantum bit of information (qubit). \n", "\n", "## The REINFROCE Algorithm\n", "\n", "The reinforcement learning objective $J$ is the expected total return, following the policy $\\pi$. If the transition probability is denoted by $p(s'|s,a)$, and the initial state distribution is $p(s_0)$, the probability for a trajectory $\\tau = (s_0,a_0,r_1,s_1,a_1,\\dots,s_{T-1},a_{T-1},r_T,s_T)$ to occur can be written as\n", "\n", "$$ \n", "P_\\pi(\\tau) = p(s_0)\\prod_{t=1}^T \\pi(a_t|s_t)p(s_{t+1}|s_t,a_t). \n", "$$\n", "\n", "The RL ojbective then takes the form\n", "\n", "$$\n", "J = \\mathrm{E}_{\\tau\\sim P_\\pi} \\left[ G(\\tau) | S_{t=0}=s_0 \\right],\\quad G(\\tau)=\\sum_{t=1}^T r(s_t,a_t).\n", "$$\n", "\n", "Policy gradient methods in RL approximate directly the policy $\\pi\\approx\\pi_\\theta$ using a variational ansatz, parametrized by the unknown parameters $\\theta$. The goal is then to find those optimal parameters $\\theta$, which optimize the RL objective $J(\\theta)$. To define an update rule for $\\theta$, we may use gradient ascent. This requires us to evaluate the gradient of the RL objective w.r.t. the parameters $\\theta$:\n", "\n", "$$\n", "\\nabla_\\theta J(\\theta) = \\nabla_\\theta \\mathrm{E}_{\\tau\\sim P_\\pi} \\left[ \\sum_{t=1}^T r(s_t,a_t) | S_{t=0}=s_0 \\right] \n", "= \\int\\mathrm{d}\\tau \\nabla_\\theta P_{\\pi_\\theta}(\\tau) G(\\tau).\n", "$$\n", "In a model-free setting, we don't have access to the transition probabilities $p(s'|s,a)$ and this requires us to be able to estimate the gradients from samples. This can be accomplished by noticing that $\\nabla_\\theta P_{\\pi_\\theta} = P_{\\pi_\\theta} \\nabla_\\theta \\log P_{\\pi_\\theta}$ (almost everywhere, i.e. up to a set of measure zero):\n", "\n", "$$\n", "\\nabla_\\theta J(\\theta) = \\int\\mathrm{d}\\tau \\nabla_\\theta P_{\\pi_\\theta}(\\tau) G(\\tau) = \\int\\mathrm{d}\\tau P_{\\pi_\\theta}(\\tau) \\nabla_\\theta \\log P_{\\pi_\\theta}(\\tau) G(\\tau) = \\mathrm{E}_{\\tau\\sim P_\\pi} \\left[\\nabla_\\theta \\log P_{\\pi_\\theta}(\\tau) G(\\tau)\\right].\n", "$$\n", "Since the initial state distribution and the transition proabilities are independent of $\\theta$, using the definition of $P_{\\pi_\\theta}$, we see that $\\nabla_\\theta P_{\\pi_\\theta}(\\tau) = \\nabla_\\theta \\pi_\\theta(\\tau)$ where $\\pi_\\theta(\\tau) = \\prod_{t=1}^T \\pi(a_t|s_t)$. \n", "\n", "We can now use MC to estimate the gradients directly from a sample of trajectories $\\{\\tau_j\\}_{j=1}^N$:\n", "$$\n", "\\nabla_\\theta J(\\theta) = \\mathrm{E}_{\\tau\\sim P_\\pi} \\left[\\nabla_\\theta \\log P_{\\pi_\\theta}(\\tau) G(\\tau)\\right]\n", "\\approx \\frac{1}{N}\\sum_{j=1}^N \\nabla_\\theta \\log \\pi_\\theta(\\tau_j) G(\\tau_j).\n", "$$\n", "\n", "In class, we discussed problems that arise due to large the variance of the gradient estimate. In particular, we showed that one can use causality and a baseline to reduce variance. The PG update then rakes the form\n", "\n", "$$\n", "\\nabla_\\theta J(\\theta)\n", "\\approx \\frac{1}{N}\\sum_{j=1}^N \\sum_{t=1}^T \\nabla_\\theta \\log \\pi_\\theta(a^j_t|s^j_t) \\left[\\sum_{t'=t}^T r(a^j_{t'}|s^j_{t'})) - b\\right].\n", "$$\n", "The corresponding gradient ascent update rule reads as\n", "\n", "$$\n", "\\theta \\leftarrow \\theta + \\alpha \\nabla_\\theta J(\\theta),\n", "$$\n", "for some step size (or learning rate) $\\alpha$. \n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Qubit Environment\n", "\n", "Let us recall the qubit environment we defined in Notebook 2.\n", "\n", "### Basic Definitions\n", "\n", "The state of a qubit $|\\psi\\rangle\\in\\mathbb{C}^2$ is modeled by a two-dimensional complex-valued vector with unit norm: $\\langle\\psi|\\psi\\rangle:=\\sqrt{|\\psi_1|^2+|\\psi_2|^2}=1$. Every qubit state is uniquely described by two angles $\\theta\\in[0,\\pi]$ and $\\varphi\\in[0,2\\pi)$:\n", "\n", "\\begin{eqnarray}\n", "|\\psi\\rangle=\n", "\\begin{pmatrix}\n", "\\psi_1 \\\\ \\psi_2\n", "\\end{pmatrix}=\n", "\\mathrm{e}^{i\\alpha}\n", "\\begin{pmatrix}\n", "\\cos\\frac{\\theta}{2} \\\\\n", "\\mathrm{e}^{i\\varphi}\\sin\\frac{\\theta}{2}\n", "\\end{pmatrix}\n", "\\end{eqnarray}\n", "\n", "The overall phase $\\alpha$ of a single quantum state has no physical meaning.\n", "Thus, any qubit state can be pictured as an arrow on the unit sphere (called the Bloch sphere) with coordinates $(\\theta,\\varphi)$. \n", "\n", "To operate on qubits, we use quantum gates. Quantum gates are represented as unitary transformations $U\\in \\mathrm{U(2)}$, where $\\mathrm{U(2)}$ is the unitary group. Gates act on qubit states by matrix multiplication to transform an input state $|\\psi\\rangle$ to the output state $|\\psi'\\rangle$: $|\\psi'\\rangle=U|\\psi\\rangle$. For this problem, we consider four gates\n", "\n", "\\begin{equation}\n", "U_0=\\boldsymbol{1},\\qquad \n", "U_x=\\mathrm{exp}(-i\\delta t \\sigma^x/2),\\qquad\n", "U_y=\\mathrm{exp}(-i\\delta t \\sigma^y/2),\\qquad \n", "U_z=\\mathrm{exp}(-i\\delta t \\sigma^z/2),\n", "\\end{equation}\n", "\n", "where $\\delta t$ is a fixed time step, $\\mathrm{exp}(\\cdot)$ is the matrix exponential, $\\boldsymbol{1}$ is the identity, and the Pauli matrices are defined as\n", "\n", "\\begin{equation}\n", "\\boldsymbol{1}=\\begin{pmatrix}\n", "1 & 0 \\\\ 0 & 1\n", "\\end{pmatrix}\n", ",\\qquad\n", "\\sigma^x=\\begin{pmatrix}\n", "0 & 1 \\\\ 1 & 0\n", "\\end{pmatrix}\n", ",\\qquad\n", "\\sigma^y=\\begin{pmatrix}\n", "0 & -i \\\\ i & 0\n", "\\end{pmatrix}\n", ",\\ \\qquad\n", "\\sigma^z=\\begin{pmatrix}\n", "1 & 0 \\\\ 0 & -1\n", "\\end{pmatrix}\n", "\\end{equation}\n", "\n", "To determine if a qubit, described by the state $|\\psi\\rangle$, is in a desired target state $|\\psi_\\mathrm{target}\\rangle$, we compute the fidelity\n", "\n", "\\begin{eqnarray}\n", "F=|\\langle\\psi_\\mathrm{target}|\\psi\\rangle|^2 = |(\\psi_\\mathrm{target})^\\ast_1 \\psi_1 + (\\psi_\\mathrm{target})^\\ast_2 \\psi_2|^2,\\qquad F\\in[0,1]\n", "\\end{eqnarray}\n", "\n", "where $\\ast$ stands for complex conjugation. Physically, the fidelity corresponds to the angle between the arrows representing the qubit state on the Bloch sphere (we want to maximize the fidelity but minimize the angle between the states).\n", "\n", "### Constructing the Qubit Environment\n", "\n", "Now, let us define an episodic RL environment, which contains the laws of physics that govern the dynamics of the qubit (i.e. the application of the gate operations to the qubit state). Our RL agent will later interact with this environment to learn how to control the qubit to bring it from an initial state to a prescribed target state. \n", "\n", "We define the RL states $s=(\\theta,\\varphi)$ as an array containing the Bloch sphere angles of the quantum state. Each step within an episode, the agent can choose to apply one out of the actions, corresponding to the four gates $(\\boldsymbol{1},U_x,U_y,U_z)$. We use the instantaneous fidelity w.r.t. the target state as a reward: $r_t=F=|\\langle\\psi_\\ast|\\psi(t)\\rangle|^2$: \n", "\n", "**state space:** $\\mathcal{S} = \\{(\\theta,\\varphi)|\\theta\\in[0,\\pi],\\varphi\\in[0,2\\pi)\\}$. Unlike in Notebook 2, there are no terminal states here. Instead, we consider a fixed number of time steps, after which the episode terminates deterministically. The target state (i.e. the qubit state we want to prepare) is $|\\psi_\\mathrm{target}\\rangle=(1,0)^t$: it has the Bloch sphere coordinates $s_\\mathrm{target}=(0,0)$.\n", "\n", "**action space:** $\\mathcal{A} = \\{\\boldsymbol{1},U_x,U_y,U_z\\}$. Actions act on RL states as follows: \n", "1. if the current state is $s=(\\theta,\\varphi)$, we first create the quantums state $|\\psi(s)\\rangle$; \n", "2. we apply the gate $U_a$ corresponding to action $a$ to the quantum state, and obtain the new quantum state $|\\psi(s')\\rangle = U_a|\\psi(s)\\rangle$. \n", "3. last, we compute the Bloch sphere coordinates which define the next state $s'=(\\theta',\\varphi')$, using the Bloch sphere parametrization for qubits given above.\n", "Note that all actions are allowed from every state. \n", "\n", "\n", "**reward space:** $\\mathcal{R}=[0,1]$. We use the fidelity between the next state $s'$ and the terminal state $s_\\mathrm{target}$ as a reward at every episode step: \n", "\n", "$$r(s,s',a)= F = |\\langle\\psi_\\mathrm{target}|U_a|\\psi(s)\\rangle|^2=|\\langle\\psi_\\mathrm{target}|\\psi(s')\\rangle|^2$$\n", "\n", "for all states $s,s'\\in\\mathcal{S}$ and actions $a\\in\\mathcal{A}$. " ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import import_ipynb\n", "from Notebook_2_RL_environments import QubitEnv2 # import environment, notebooks must be in same directory" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# set seed of rng (for reproducibility of the results)\n", "n_time_steps = 60\n", "seed=0 \n", "np.random.seed(seed)\n", "\n", "# create environment class\n", "env=QubitEnv2(n_time_steps, seed=seed)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Policy Gradient Implementation\n", "\n", "The implementation of PG follows similar steps as the MNIST problem from Notebook 7:\n", "\n", "1. Define the a SoftMax model for the discrete policy $\\pi_\\theta$.\n", "2. Define the pseudo loss function to easily compute $\\nabla_\\theta J(\\theta)$.\n", "3. Define generalized gradient descent optimizer.\n", "4. Define the PG training loop and train the policy.\n", "\n", "\n", "### Define the a SoftMax model for the discrete policy $\\pi_\\theta$\n", "\n", "Use JAX to construct a feed-forward fully-connected deep neural network with neuron acrchitecture $(M_s, 512, 256, |\\mathcal{A}|)$, where there are $512$ ($256$) neurons in the first (second) hidden layer, respectively, and $M_s$ and $|\\mathcal{A}|$ define the input and output sizes.\n", "\n", "The input data into the neural network should have the shape `input_shape = (-1, n_time_steps, M_s)`, where `M_s` is the number of features/components in the RL state $s=(\\theta,\\varphi)$. The output data should have the shape `output_shape = (-1, n_time_steps, abs_A)`, where `abs_A`$=|\\mathcal{A}|$. In this way, we can use the neural network to process simultaneously all time steps and MC samples, generated in a single training iteration. \n", "\n", "Check explicitly the output shape and test that the network runs on some fake data (e.g. a small batch of vectors of ones with the appropriate shape). " ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "output shape of the policy network is (-1, 60, 4).\n", "\n", "(3, 60, 4)\n" ] } ], "source": [ "import jax.numpy as jnp # jax's numpy version with GPU support\n", "from jax import random # used to define a RNG key to control the random input in JAX\n", "from jax.experimental import stax # neural network library\n", "from jax.experimental.stax import Dense, Relu, LogSoftmax # neural network layers\n", "\n", "# set key for the RNG (see JAX docs)\n", "rng = random.PRNGKey(seed)\n", "\n", "# define functions which initialize the parameters and evaluate the model\n", "initialize_params, predict = stax.serial(\n", " ### fully connected DNN\n", " Dense(512), # 512 hidden neurons\n", " Relu,\n", " Dense(256), # 256 hidden neurons\n", " Relu,\n", " Dense(env.n_actions), # 4 output neurons\n", " LogSoftmax # NB: computes the log-probability\n", " )\n", "\n", "# initialize the model parameters\n", "input_shape = (-1,env.n_time_steps,2) # -1: number of MC points, number of time steps, size of state vector\n", "output_shape, inital_params = initialize_params(rng, input_shape) # fcc layer 28x28 pixes in each image\n", "\n", "print('\\noutput shape of the policy network is {}.\\n'.format(output_shape))\n", "\n", "\n", "# test network\n", "states=np.ones((3,env.n_time_steps,2), dtype=np.float32)\n", "\n", "predictions = predict(inital_params, states)\n", "# check the output shape\n", "print(predictions.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define the pseudo loss function to easily compute $\\nabla_\\theta J(\\theta)$\n", "\n", "In class we can defined a scalar pseudoloss function, whose gradients give $\\nabla_\\theta J(\\theta)$. Note that this pseudoloss does ***NOT*** correspond to the RL objective $J(\\theta)$: the difference stems from the fact that the two operations of taking the derivative and performing the MC approximation are not interchangeable (do you see why?). \n", "\n", "$$\n", "J_\\mathrm{pseudo}(\\theta) = \n", "\\frac{1}{N}\\sum_{j=1}^N \\sum_{t=1}^T \\log \\pi_\\theta(a^j_t|s^j_t) \\left[\\sum_{t'=t}^T r(a^j_{t'}|s^j_{t'})) - b_t\\right],\\qquad \n", "b_t = \\frac{1}{N}\\sum_{j=1}^N G_t(\\tau_j).\n", "$$\n", "The baseline is a sample average of the reward-to-go (return) from time step $t$ onwards: $G_t(\\tau_j) = \\sum_{t'=t}^N r(s^j_{t'},s^j_{t'})$ .\n", "\n", "Because we will be doing gradient **a**scent, do **NOT** forget to add an extra minus sign to the output ot the pseudoloss (or else your agent will end up minimizing the return). \n", "\n", "Below, we also add an L2 regularizer to the pseudoloss function to prevent overfitting. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "### define loss and accuracy functions\n", "\n", "from jax import grad\n", "from jax.tree_util import tree_flatten # jax params are stored as nested tuples; use this to manipulate tuples\n", "\n", "\n", "def l2_regularizer(params, lmbda):\n", " \"\"\"\n", " Define l2 regularizer: $\\lambda \\ sum_j ||theta_j||^2 $ for every parameter in the model $\\theta_j$\n", " \n", " \"\"\"\n", " return lmbda*jnp.sum(jnp.array([jnp.sum(jnp.abs(theta)**2) for theta in tree_flatten(params)[0] ]))\n", "\n", "\n", "def pseudo_loss(params, trajectory_batch):\n", " \"\"\"\n", " Define the pseudo loss function for policy gradient. \n", " \n", " params: object(jax pytree):\n", " parameters of the deep policy network.\n", " trajectory_batch: tuple (states, actions, returns) containing the RL states, actions and returns (not the rewards!): \n", " states: np.array of size (N_MC, env.n_time_steps,2)\n", " actions: np.array of size (N_MC, env.n_time_steps)\n", " returns: np.array of size (N_MC, env.n_time_steps)\n", " \n", " Returns:\n", " -J_{pseudo}(\\theta)\n", "\n", " \"\"\"\n", " # extract data from the batch\n", " states, actions, returns = trajectory_batch\n", " # compute policy predictions\n", " preds = predict(params, states)\n", " # combute the baseline\n", " baseline = jnp.mean(rewards, axis=0)\n", " # select those values of the policy along the action trajectory\n", " preds_select = jnp.take_along_axis(preds, jnp.expand_dims(actions, axis=2), axis=2).squeeze()\n", " # return negative pseudo loss function (want to maximize reward with gradient DEscent)\n", " return -jnp.mean(jnp.sum(preds_select * (returns - baseline) )) + l2_regularizer(params, 0.001)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define generalized gradient descent optimizer\n", "\n", "Define the optimizer and the `update` function which computes the gradient o the pseudo-loss function and performs the update. \n", "\n", "We use the Adam optimizer here with `step_size = 0.001` and the rest of the parameters have default values. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "### define generalized gradient descent optimizer and a function to update model parameters\n", "\n", "from jax.experimental import optimizers # gradient descent optimizers\n", "from jax import jit\n", "\n", "step_size = 0.001 # step size or learning rate \n", "\n", "# compute optimizer functions\n", "opt_init, opt_update, get_params = optimizers.adam(step_size)\n", "\n", "\n", "# define function which updates the parameters using the change computed by the optimizer\n", "@jit # Just In Time compilation speeds up the code; requires to use jnp everywhere; remove when debugging\n", "def update(i, opt_state, batch):\n", " \"\"\"\n", " i: int,\n", " counter to count how many update steps we have performed\n", " opt_state: object,\n", " the state of the optimizer\n", " batch: np.array\n", " batch containing the data used to update the model\n", " \n", " Returns: \n", " opt_state: object,\n", " the new state of the optimizer\n", " \n", " \"\"\"\n", " # get current parameters of the model\n", " current_params = get_params(opt_state)\n", " # compute gradients\n", " grad_params = grad(pseudo_loss)(current_params, batch)\n", " # use the optimizer to perform the update using opt_update\n", " return opt_update(i, grad_params, opt_state)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define the PG training loop and train the policy\n", "\n", "Finally, we implement the REINFORCE algorithm for policy gradient. Follow the steps below\n", "\n", "1. Preallocate variables\n", " * Define the number of episodes `N_episodes`, and the batch size `N_MC`.\n", " * Preallocate arrays for the current `state`, and the `states`, `actions`, `returns` triple which defines the trajectory batch. \n", " * Preallocate arrays to compute the `mean_final_reward`, `std_final_reward`, `min_final_reward`, and , `max_final_reward`.\n", "2. Initialize the optimizer using the `opt_init` function. \n", "3. Loop over the episodes; for every episode:\n", " \n", " 3.1 get the current Network parameters\n", " \n", " 3.2 loop to collect MC samples\n", " \n", " 3.2.1 reset the `env` and roll out the policy until the episode is over; collect the trajectory data\n", " 3.2.2 compute the returns (rewards to go)\n", " \n", " 3.3 compile the PG data into a trajctory batch\n", " \n", " 3.4 use the `update` function to update the network parameters\n", " \n", " 3.5 print instantaneous performance" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Starting training...\n", "\n", "episode 0 in 8.90 sec\n", "mean reward: 0.4562\n", "return standard deviation: 0.2593\n", "min return: 0.0051; max return: 0.9465\n", "\n", "episode 1 in 9.14 sec\n", "mean reward: 0.4051\n", "return standard deviation: 0.2847\n", "min return: 0.0071; max return: 0.9868\n", "\n", "episode 2 in 8.67 sec\n", "mean reward: 0.4554\n", "return standard deviation: 0.2654\n", "min return: 0.0070; max return: 0.9843\n", "\n", "episode 3 in 8.77 sec\n", "mean reward: 0.5347\n", "return standard deviation: 0.2965\n", "min return: 0.0125; max return: 0.9942\n", "\n", "episode 4 in 9.10 sec\n", "mean reward: 0.4993\n", "return standard deviation: 0.2915\n", "min return: 0.0036; max return: 0.9948\n", "\n", "episode 5 in 9.29 sec\n", "mean reward: 0.5093\n", "return standard deviation: 0.2557\n", "min return: 0.0265; max return: 0.9842\n", "\n", "episode 6 in 8.86 sec\n", "mean reward: 0.5573\n", "return standard deviation: 0.2985\n", "min return: 0.0192; max return: 0.9903\n", "\n", "episode 7 in 9.31 sec\n", "mean reward: 0.4865\n", "return standard deviation: 0.2842\n", "min return: 0.0125; max return: 0.9935\n", "\n", "episode 8 in 9.03 sec\n", "mean reward: 0.5378\n", "return standard deviation: 0.2851\n", "min return: 0.0060; max return: 0.9813\n", "\n", "episode 9 in 9.46 sec\n", "mean reward: 0.5306\n", "return standard deviation: 0.3091\n", "min return: 0.0541; max return: 0.9978\n", "\n", "episode 10 in 9.03 sec\n", "mean reward: 0.5294\n", "return standard deviation: 0.2730\n", "min return: 0.0009; max return: 0.9724\n", "\n", "episode 11 in 9.41 sec\n", "mean reward: 0.5583\n", "return standard deviation: 0.2846\n", "min return: 0.0234; max return: 0.9857\n", "\n", "episode 12 in 8.96 sec\n", "mean reward: 0.6407\n", "return standard deviation: 0.2760\n", "min return: 0.0545; max return: 0.9864\n", "\n", "episode 13 in 9.20 sec\n", "mean reward: 0.6151\n", "return standard deviation: 0.2683\n", "min return: 0.0156; max return: 0.9967\n", "\n", "episode 14 in 9.36 sec\n", "mean reward: 0.5790\n", "return standard deviation: 0.2913\n", "min return: 0.0216; max return: 0.9975\n", "\n", "episode 15 in 8.87 sec\n", "mean reward: 0.5868\n", "return standard deviation: 0.2821\n", "min return: 0.0052; max return: 0.9910\n", "\n", "episode 16 in 8.99 sec\n", "mean reward: 0.7151\n", "return standard deviation: 0.2415\n", "min return: 0.1630; max return: 0.9982\n", "\n", "episode 17 in 9.41 sec\n", "mean reward: 0.6150\n", "return standard deviation: 0.2886\n", "min return: 0.0168; max return: 0.9989\n", "\n", "episode 18 in 9.68 sec\n", "mean reward: 0.6341\n", "return standard deviation: 0.2463\n", "min return: 0.0259; max return: 0.9963\n", "\n", "episode 19 in 9.30 sec\n", "mean reward: 0.6562\n", "return standard deviation: 0.2485\n", "min return: 0.0988; max return: 0.9996\n", "\n", "episode 20 in 9.23 sec\n", "mean reward: 0.6304\n", "return standard deviation: 0.2602\n", "min return: 0.0964; max return: 0.9991\n", "\n", "episode 21 in 9.55 sec\n", "mean reward: 0.7116\n", "return standard deviation: 0.2353\n", "min return: 0.0394; max return: 0.9997\n", "\n", "episode 22 in 9.52 sec\n", "mean reward: 0.6820\n", "return standard deviation: 0.2720\n", "min return: 0.0979; max return: 0.9998\n", "\n", "episode 23 in 9.68 sec\n", "mean reward: 0.6908\n", "return standard deviation: 0.2357\n", "min return: 0.1425; max return: 0.9970\n", "\n", "episode 24 in 11.33 sec\n", "mean reward: 0.7206\n", "return standard deviation: 0.2125\n", "min return: 0.0277; max return: 0.9913\n", "\n", "episode 25 in 9.63 sec\n", "mean reward: 0.7112\n", "return standard deviation: 0.2254\n", "min return: 0.0384; max return: 0.9940\n", "\n", "episode 26 in 9.24 sec\n", "mean reward: 0.7421\n", "return standard deviation: 0.2210\n", "min return: 0.0871; max return: 0.9838\n", "\n", "episode 27 in 9.04 sec\n", "mean reward: 0.7187\n", "return standard deviation: 0.2057\n", "min return: 0.2195; max return: 0.9878\n", "\n", "episode 28 in 9.19 sec\n", "mean reward: 0.7141\n", "return standard deviation: 0.2373\n", "min return: 0.2032; max return: 0.9948\n", "\n", "episode 29 in 8.96 sec\n", "mean reward: 0.7788\n", "return standard deviation: 0.1877\n", "min return: 0.2063; max return: 0.9973\n", "\n", "episode 30 in 9.16 sec\n", "mean reward: 0.7236\n", "return standard deviation: 0.2281\n", "min return: 0.0355; max return: 0.9875\n", "\n", "episode 31 in 9.15 sec\n", "mean reward: 0.7176\n", "return standard deviation: 0.1989\n", "min return: 0.2246; max return: 0.9769\n", "\n", "episode 32 in 8.86 sec\n", "mean reward: 0.7703\n", "return standard deviation: 0.1802\n", "min return: 0.3438; max return: 0.9994\n", "\n", "episode 33 in 9.12 sec\n", "mean reward: 0.7312\n", "return standard deviation: 0.2225\n", "min return: 0.1938; max return: 0.9923\n", "\n", "episode 34 in 8.97 sec\n", "mean reward: 0.7701\n", "return standard deviation: 0.1705\n", "min return: 0.3199; max return: 0.9985\n", "\n", "episode 35 in 9.50 sec\n", "mean reward: 0.7752\n", "return standard deviation: 0.1667\n", "min return: 0.3696; max return: 0.9938\n", "\n", "episode 36 in 9.07 sec\n", "mean reward: 0.7569\n", "return standard deviation: 0.1618\n", "min return: 0.2729; max return: 0.9893\n", "\n", "episode 37 in 9.03 sec\n", "mean reward: 0.7762\n", "return standard deviation: 0.1813\n", "min return: 0.2023; max return: 0.9927\n", "\n", "episode 38 in 9.56 sec\n", "mean reward: 0.7589\n", "return standard deviation: 0.1825\n", "min return: 0.1899; max return: 0.9992\n", "\n", "episode 39 in 8.75 sec\n", "mean reward: 0.8207\n", "return standard deviation: 0.1517\n", "min return: 0.1449; max return: 0.9992\n", "\n", "episode 40 in 8.80 sec\n", "mean reward: 0.7550\n", "return standard deviation: 0.1653\n", "min return: 0.2631; max return: 0.9950\n", "\n", "episode 41 in 8.78 sec\n", "mean reward: 0.7910\n", "return standard deviation: 0.1739\n", "min return: 0.1389; max return: 0.9999\n", "\n", "episode 42 in 8.79 sec\n", "mean reward: 0.7976\n", "return standard deviation: 0.1599\n", "min return: 0.2042; max return: 0.9865\n", "\n", "episode 43 in 8.83 sec\n", "mean reward: 0.8185\n", "return standard deviation: 0.1551\n", "min return: 0.2409; max return: 0.9972\n", "\n", "episode 44 in 8.75 sec\n", "mean reward: 0.7915\n", "return standard deviation: 0.2075\n", "min return: 0.1613; max return: 0.9962\n", "\n", "episode 45 in 8.79 sec\n", "mean reward: 0.7909\n", "return standard deviation: 0.2015\n", "min return: 0.1553; max return: 0.9999\n", "\n", "episode 46 in 9.26 sec\n", "mean reward: 0.8140\n", "return standard deviation: 0.1590\n", "min return: 0.2167; max return: 0.9990\n", "\n", "episode 47 in 9.72 sec\n", "mean reward: 0.8372\n", "return standard deviation: 0.1374\n", "min return: 0.2924; max return: 0.9997\n", "\n", "episode 48 in 9.65 sec\n", "mean reward: 0.8210\n", "return standard deviation: 0.1420\n", "min return: 0.2590; max return: 0.9910\n", "\n", "episode 49 in 9.23 sec\n", "mean reward: 0.8402\n", "return standard deviation: 0.1232\n", "min return: 0.5231; max return: 0.9986\n", "\n", "episode 50 in 9.14 sec\n", "mean reward: 0.8365\n", "return standard deviation: 0.1377\n", "min return: 0.4923; max return: 0.9972\n", "\n", "episode 51 in 8.36 sec\n", "mean reward: 0.8628\n", "return standard deviation: 0.1150\n", "min return: 0.5252; max return: 0.9988\n", "\n", "episode 52 in 8.26 sec\n", "mean reward: 0.8809\n", "return standard deviation: 0.0942\n", "min return: 0.5694; max return: 0.9916\n", "\n", "episode 53 in 8.20 sec\n", "mean reward: 0.8907\n", "return standard deviation: 0.0919\n", "min return: 0.5806; max return: 0.9984\n", "\n", "episode 54 in 8.24 sec\n", "mean reward: 0.8687\n", "return standard deviation: 0.1057\n", "min return: 0.5968; max return: 0.9998\n", "\n", "episode 55 in 8.21 sec\n", "mean reward: 0.8456\n", "return standard deviation: 0.1166\n", "min return: 0.5156; max return: 0.9999\n", "\n", "episode 56 in 8.20 sec\n", "mean reward: 0.8457\n", "return standard deviation: 0.1084\n", "min return: 0.5797; max return: 0.9954\n", "\n", "episode 57 in 8.18 sec\n", "mean reward: 0.8819\n", "return standard deviation: 0.0976\n", "min return: 0.5934; max return: 0.9997\n", "\n", "episode 58 in 8.20 sec\n", "mean reward: 0.8878\n", "return standard deviation: 0.0924\n", "min return: 0.6578; max return: 0.9996\n", "\n", "episode 59 in 8.20 sec\n", "mean reward: 0.8851\n", "return standard deviation: 0.1024\n", "min return: 0.6078; max return: 0.9981\n", "\n", "episode 60 in 8.32 sec\n", "mean reward: 0.8834\n", "return standard deviation: 0.1038\n", "min return: 0.4849; max return: 0.9997\n", "\n", "episode 61 in 8.21 sec\n", "mean reward: 0.8864\n", "return standard deviation: 0.0946\n", "min return: 0.5364; max return: 0.9995\n", "\n", "episode 62 in 8.19 sec\n", "mean reward: 0.8756\n", "return standard deviation: 0.0889\n", "min return: 0.6705; max return: 0.9987\n", "\n", "episode 63 in 8.20 sec\n", "mean reward: 0.8888\n", "return standard deviation: 0.1026\n", "min return: 0.6283; max return: 0.9999\n", "\n", "episode 64 in 8.20 sec\n", "mean reward: 0.9136\n", "return standard deviation: 0.0786\n", "min return: 0.5912; max return: 0.9992\n", "\n", "episode 65 in 8.36 sec\n", "mean reward: 0.8897\n", "return standard deviation: 0.0774\n", "min return: 0.6997; max return: 0.9962\n", "\n", "episode 66 in 8.22 sec\n", "mean reward: 0.9129\n", "return standard deviation: 0.0653\n", "min return: 0.7374; max return: 0.9991\n", "\n", "episode 67 in 8.24 sec\n", "mean reward: 0.9091\n", "return standard deviation: 0.0766\n", "min return: 0.5948; max return: 0.9991\n", "\n", "episode 68 in 8.21 sec\n", "mean reward: 0.9218\n", "return standard deviation: 0.0653\n", "min return: 0.6746; max return: 0.9991\n", "\n", "episode 69 in 8.22 sec\n", "mean reward: 0.9284\n", "return standard deviation: 0.0715\n", "min return: 0.6303; max return: 0.9999\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "episode 70 in 8.22 sec\n", "mean reward: 0.9421\n", "return standard deviation: 0.0653\n", "min return: 0.6942; max return: 0.9998\n", "\n", "episode 71 in 8.25 sec\n", "mean reward: 0.9335\n", "return standard deviation: 0.0581\n", "min return: 0.7803; max return: 0.9989\n", "\n", "episode 72 in 8.29 sec\n", "mean reward: 0.9331\n", "return standard deviation: 0.0591\n", "min return: 0.7460; max return: 0.9996\n", "\n", "episode 73 in 8.22 sec\n", "mean reward: 0.9406\n", "return standard deviation: 0.0512\n", "min return: 0.7864; max return: 0.9997\n", "\n", "episode 74 in 8.28 sec\n", "mean reward: 0.9360\n", "return standard deviation: 0.0658\n", "min return: 0.6442; max return: 0.9969\n", "\n", "episode 75 in 8.21 sec\n", "mean reward: 0.9306\n", "return standard deviation: 0.1241\n", "min return: 0.0649; max return: 0.9997\n", "\n", "episode 76 in 8.20 sec\n", "mean reward: 0.9414\n", "return standard deviation: 0.0589\n", "min return: 0.7616; max return: 0.9992\n", "\n", "episode 77 in 8.17 sec\n", "mean reward: 0.9313\n", "return standard deviation: 0.0571\n", "min return: 0.7284; max return: 0.9997\n", "\n", "episode 78 in 8.16 sec\n", "mean reward: 0.9351\n", "return standard deviation: 0.0800\n", "min return: 0.4576; max return: 0.9999\n", "\n", "episode 79 in 8.20 sec\n", "mean reward: 0.9483\n", "return standard deviation: 0.0439\n", "min return: 0.8036; max return: 0.9996\n", "\n", "episode 80 in 8.20 sec\n", "mean reward: 0.9413\n", "return standard deviation: 0.0550\n", "min return: 0.6899; max return: 0.9982\n", "\n", "episode 81 in 8.19 sec\n", "mean reward: 0.9439\n", "return standard deviation: 0.0487\n", "min return: 0.7783; max return: 0.9989\n", "\n", "episode 82 in 8.17 sec\n", "mean reward: 0.9374\n", "return standard deviation: 0.0513\n", "min return: 0.8131; max return: 0.9997\n", "\n", "episode 83 in 8.16 sec\n", "mean reward: 0.9390\n", "return standard deviation: 0.0580\n", "min return: 0.7053; max return: 0.9979\n", "\n", "episode 84 in 8.15 sec\n", "mean reward: 0.9378\n", "return standard deviation: 0.0516\n", "min return: 0.7748; max return: 0.9993\n", "\n", "episode 85 in 8.20 sec\n", "mean reward: 0.9363\n", "return standard deviation: 0.0452\n", "min return: 0.7753; max return: 0.9994\n", "\n", "episode 86 in 8.13 sec\n", "mean reward: 0.9408\n", "return standard deviation: 0.0506\n", "min return: 0.7392; max return: 0.9998\n", "\n", "episode 87 in 8.17 sec\n", "mean reward: 0.9413\n", "return standard deviation: 0.0510\n", "min return: 0.8213; max return: 1.0000\n", "\n", "episode 88 in 8.14 sec\n", "mean reward: 0.9276\n", "return standard deviation: 0.0665\n", "min return: 0.7419; max return: 0.9997\n", "\n", "episode 89 in 8.17 sec\n", "mean reward: 0.9267\n", "return standard deviation: 0.0626\n", "min return: 0.7629; max return: 0.9995\n", "\n", "episode 90 in 8.16 sec\n", "mean reward: 0.9204\n", "return standard deviation: 0.1075\n", "min return: 0.2549; max return: 0.9995\n", "\n", "episode 91 in 8.29 sec\n", "mean reward: 0.9205\n", "return standard deviation: 0.0975\n", "min return: 0.3478; max return: 0.9961\n", "\n", "episode 92 in 8.14 sec\n", "mean reward: 0.9312\n", "return standard deviation: 0.0591\n", "min return: 0.7153; max return: 0.9989\n", "\n", "episode 93 in 8.14 sec\n", "mean reward: 0.9343\n", "return standard deviation: 0.0664\n", "min return: 0.7087; max return: 0.9991\n", "\n", "episode 94 in 8.22 sec\n", "mean reward: 0.9189\n", "return standard deviation: 0.1261\n", "min return: 0.2715; max return: 0.9995\n", "\n", "episode 95 in 8.18 sec\n", "mean reward: 0.8925\n", "return standard deviation: 0.1021\n", "min return: 0.5659; max return: 0.9997\n", "\n", "episode 96 in 8.13 sec\n", "mean reward: 0.8824\n", "return standard deviation: 0.1471\n", "min return: 0.3693; max return: 0.9996\n", "\n", "episode 97 in 8.18 sec\n", "mean reward: 0.8992\n", "return standard deviation: 0.1243\n", "min return: 0.2676; max return: 0.9989\n", "\n", "episode 98 in 8.14 sec\n", "mean reward: 0.8647\n", "return standard deviation: 0.1670\n", "min return: 0.2451; max return: 0.9995\n", "\n", "episode 99 in 8.14 sec\n", "mean reward: 0.9135\n", "return standard deviation: 0.1198\n", "min return: 0.4102; max return: 0.9997\n", "\n", "episode 100 in 8.15 sec\n", "mean reward: 0.8907\n", "return standard deviation: 0.1430\n", "min return: 0.3313; max return: 0.9998\n", "\n", "episode 101 in 8.16 sec\n", "mean reward: 0.9076\n", "return standard deviation: 0.1043\n", "min return: 0.4901; max return: 0.9993\n", "\n", "episode 102 in 8.15 sec\n", "mean reward: 0.9323\n", "return standard deviation: 0.0698\n", "min return: 0.6613; max return: 0.9984\n", "\n", "episode 103 in 8.17 sec\n", "mean reward: 0.9249\n", "return standard deviation: 0.0901\n", "min return: 0.3862; max return: 0.9999\n", "\n", "episode 104 in 8.13 sec\n", "mean reward: 0.9185\n", "return standard deviation: 0.0726\n", "min return: 0.7143; max return: 0.9975\n", "\n", "episode 105 in 8.14 sec\n", "mean reward: 0.9379\n", "return standard deviation: 0.1014\n", "min return: 0.2571; max return: 0.9998\n", "\n", "episode 106 in 8.16 sec\n", "mean reward: 0.9465\n", "return standard deviation: 0.0448\n", "min return: 0.8116; max return: 0.9991\n", "\n", "episode 107 in 8.14 sec\n", "mean reward: 0.9408\n", "return standard deviation: 0.0590\n", "min return: 0.7569; max return: 0.9990\n", "\n", "episode 108 in 8.17 sec\n", "mean reward: 0.9483\n", "return standard deviation: 0.0427\n", "min return: 0.8300; max return: 1.0000\n", "\n", "episode 109 in 8.21 sec\n", "mean reward: 0.9419\n", "return standard deviation: 0.0681\n", "min return: 0.4777; max return: 0.9979\n", "\n", "episode 110 in 8.19 sec\n", "mean reward: 0.9614\n", "return standard deviation: 0.0410\n", "min return: 0.8126; max return: 0.9999\n", "\n", "episode 111 in 8.16 sec\n", "mean reward: 0.9611\n", "return standard deviation: 0.0412\n", "min return: 0.8131; max return: 0.9991\n", "\n", "episode 112 in 8.18 sec\n", "mean reward: 0.9590\n", "return standard deviation: 0.0425\n", "min return: 0.8133; max return: 0.9999\n", "\n", "episode 113 in 8.19 sec\n", "mean reward: 0.9672\n", "return standard deviation: 0.0345\n", "min return: 0.8631; max return: 0.9998\n", "\n", "episode 114 in 8.17 sec\n", "mean reward: 0.9613\n", "return standard deviation: 0.0362\n", "min return: 0.8465; max return: 0.9993\n", "\n", "episode 115 in 8.16 sec\n", "mean reward: 0.9525\n", "return standard deviation: 0.0498\n", "min return: 0.8003; max return: 0.9998\n", "\n", "episode 116 in 8.17 sec\n", "mean reward: 0.9564\n", "return standard deviation: 0.0440\n", "min return: 0.7952; max return: 0.9996\n", "\n", "episode 117 in 8.17 sec\n", "mean reward: 0.9609\n", "return standard deviation: 0.0384\n", "min return: 0.8364; max return: 0.9998\n", "\n", "episode 118 in 8.15 sec\n", "mean reward: 0.9631\n", "return standard deviation: 0.0474\n", "min return: 0.7872; max return: 0.9998\n", "\n", "episode 119 in 8.15 sec\n", "mean reward: 0.9661\n", "return standard deviation: 0.0302\n", "min return: 0.8693; max return: 0.9993\n", "\n", "episode 120 in 8.23 sec\n", "mean reward: 0.9635\n", "return standard deviation: 0.0347\n", "min return: 0.8605; max return: 0.9999\n", "\n", "episode 121 in 8.16 sec\n", "mean reward: 0.9666\n", "return standard deviation: 0.0334\n", "min return: 0.8362; max return: 0.9990\n", "\n", "episode 122 in 8.16 sec\n", "mean reward: 0.9650\n", "return standard deviation: 0.0318\n", "min return: 0.8783; max return: 0.9997\n", "\n", "episode 123 in 8.17 sec\n", "mean reward: 0.9689\n", "return standard deviation: 0.0277\n", "min return: 0.8868; max return: 0.9998\n", "\n", "episode 124 in 8.21 sec\n", "mean reward: 0.9685\n", "return standard deviation: 0.0282\n", "min return: 0.8813; max return: 0.9999\n", "\n", "episode 125 in 8.19 sec\n", "mean reward: 0.9623\n", "return standard deviation: 0.0416\n", "min return: 0.8181; max return: 0.9999\n", "\n", "episode 126 in 8.16 sec\n", "mean reward: 0.9767\n", "return standard deviation: 0.0244\n", "min return: 0.8786; max return: 0.9985\n", "\n", "episode 127 in 8.18 sec\n", "mean reward: 0.9755\n", "return standard deviation: 0.0247\n", "min return: 0.8597; max return: 0.9997\n", "\n", "episode 128 in 8.15 sec\n", "mean reward: 0.9712\n", "return standard deviation: 0.0274\n", "min return: 0.8736; max return: 0.9999\n", "\n", "episode 129 in 8.16 sec\n", "mean reward: 0.9723\n", "return standard deviation: 0.0294\n", "min return: 0.8423; max return: 0.9996\n", "\n", "episode 130 in 8.13 sec\n", "mean reward: 0.9757\n", "return standard deviation: 0.0273\n", "min return: 0.8886; max return: 0.9996\n", "\n", "episode 131 in 8.17 sec\n", "mean reward: 0.9706\n", "return standard deviation: 0.0350\n", "min return: 0.8081; max return: 1.0000\n", "\n", "episode 132 in 8.14 sec\n", "mean reward: 0.9736\n", "return standard deviation: 0.0254\n", "min return: 0.8619; max return: 0.9998\n", "\n", "episode 133 in 8.17 sec\n", "mean reward: 0.9690\n", "return standard deviation: 0.0399\n", "min return: 0.8256; max return: 1.0000\n", "\n", "episode 134 in 8.17 sec\n", "mean reward: 0.9744\n", "return standard deviation: 0.0267\n", "min return: 0.8564; max return: 0.9996\n", "\n", "episode 135 in 8.13 sec\n", "mean reward: 0.9696\n", "return standard deviation: 0.0338\n", "min return: 0.8445; max return: 0.9999\n", "\n", "episode 136 in 8.16 sec\n", "mean reward: 0.9718\n", "return standard deviation: 0.0385\n", "min return: 0.7722; max return: 0.9997\n", "\n", "episode 137 in 8.18 sec\n", "mean reward: 0.9750\n", "return standard deviation: 0.0268\n", "min return: 0.8717; max return: 0.9993\n", "\n", "episode 138 in 8.28 sec\n", "mean reward: 0.9754\n", "return standard deviation: 0.0258\n", "min return: 0.8755; max return: 0.9997\n", "\n", "episode 139 in 8.17 sec\n", "mean reward: 0.9771\n", "return standard deviation: 0.0261\n", "min return: 0.8484; max return: 0.9997\n", "\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "episode 140 in 8.16 sec\n", "mean reward: 0.9729\n", "return standard deviation: 0.0268\n", "min return: 0.8683; max return: 1.0000\n", "\n", "episode 141 in 8.16 sec\n", "mean reward: 0.9796\n", "return standard deviation: 0.0192\n", "min return: 0.9099; max return: 0.9999\n", "\n", "episode 142 in 8.15 sec\n", "mean reward: 0.9816\n", "return standard deviation: 0.0174\n", "min return: 0.9369; max return: 1.0000\n", "\n", "episode 143 in 8.19 sec\n", "mean reward: 0.9739\n", "return standard deviation: 0.0353\n", "min return: 0.7853; max return: 1.0000\n", "\n", "episode 144 in 8.15 sec\n", "mean reward: 0.9788\n", "return standard deviation: 0.0255\n", "min return: 0.8590; max return: 0.9996\n", "\n", "episode 145 in 8.21 sec\n", "mean reward: 0.9688\n", "return standard deviation: 0.0327\n", "min return: 0.8534; max return: 0.9994\n", "\n", "episode 146 in 8.15 sec\n", "mean reward: 0.9747\n", "return standard deviation: 0.0378\n", "min return: 0.8046; max return: 0.9998\n", "\n", "episode 147 in 8.17 sec\n", "mean reward: 0.9778\n", "return standard deviation: 0.0311\n", "min return: 0.8614; max return: 0.9995\n", "\n", "episode 148 in 8.26 sec\n", "mean reward: 0.9813\n", "return standard deviation: 0.0194\n", "min return: 0.9101; max return: 0.9996\n", "\n", "episode 149 in 8.15 sec\n", "mean reward: 0.9755\n", "return standard deviation: 0.0321\n", "min return: 0.8205; max return: 0.9999\n", "\n", "episode 150 in 8.13 sec\n", "mean reward: 0.9775\n", "return standard deviation: 0.0290\n", "min return: 0.8489; max return: 0.9994\n", "\n", "episode 151 in 8.17 sec\n", "mean reward: 0.9790\n", "return standard deviation: 0.0251\n", "min return: 0.8608; max return: 0.9998\n", "\n", "episode 152 in 8.12 sec\n", "mean reward: 0.9820\n", "return standard deviation: 0.0201\n", "min return: 0.9171; max return: 1.0000\n", "\n", "episode 153 in 8.25 sec\n", "mean reward: 0.9851\n", "return standard deviation: 0.0158\n", "min return: 0.9317; max return: 0.9999\n", "\n", "episode 154 in 8.18 sec\n", "mean reward: 0.9770\n", "return standard deviation: 0.0280\n", "min return: 0.8563; max return: 0.9996\n", "\n", "episode 155 in 8.16 sec\n", "mean reward: 0.9808\n", "return standard deviation: 0.0204\n", "min return: 0.8870; max return: 0.9998\n", "\n", "episode 156 in 8.15 sec\n", "mean reward: 0.9844\n", "return standard deviation: 0.0164\n", "min return: 0.9389; max return: 1.0000\n", "\n", "episode 157 in 8.13 sec\n", "mean reward: 0.9798\n", "return standard deviation: 0.0264\n", "min return: 0.8693; max return: 0.9999\n", "\n", "episode 158 in 8.16 sec\n", "mean reward: 0.9783\n", "return standard deviation: 0.0234\n", "min return: 0.8940; max return: 0.9998\n", "\n", "episode 159 in 8.14 sec\n", "mean reward: 0.9814\n", "return standard deviation: 0.0238\n", "min return: 0.8897; max return: 0.9999\n", "\n", "episode 160 in 8.17 sec\n", "mean reward: 0.9800\n", "return standard deviation: 0.0224\n", "min return: 0.8786; max return: 0.9991\n", "\n", "episode 161 in 8.14 sec\n", "mean reward: 0.9790\n", "return standard deviation: 0.0211\n", "min return: 0.9156; max return: 0.9999\n", "\n", "episode 162 in 8.15 sec\n", "mean reward: 0.9778\n", "return standard deviation: 0.0257\n", "min return: 0.8528; max return: 0.9996\n", "\n", "episode 163 in 8.17 sec\n", "mean reward: 0.9633\n", "return standard deviation: 0.0779\n", "min return: 0.4841; max return: 0.9996\n", "\n", "episode 164 in 8.14 sec\n", "mean reward: 0.9678\n", "return standard deviation: 0.0782\n", "min return: 0.3886; max return: 1.0000\n", "\n", "episode 165 in 8.14 sec\n", "mean reward: 0.9769\n", "return standard deviation: 0.0449\n", "min return: 0.6764; max return: 0.9998\n", "\n", "episode 166 in 8.12 sec\n", "mean reward: 0.9745\n", "return standard deviation: 0.0553\n", "min return: 0.5742; max return: 1.0000\n", "\n", "episode 167 in 8.22 sec\n", "mean reward: 0.9651\n", "return standard deviation: 0.0811\n", "min return: 0.3754; max return: 0.9999\n", "\n", "episode 168 in 8.15 sec\n", "mean reward: 0.9653\n", "return standard deviation: 0.0790\n", "min return: 0.4658; max return: 0.9999\n", "\n", "episode 169 in 8.17 sec\n", "mean reward: 0.9793\n", "return standard deviation: 0.0238\n", "min return: 0.9017; max return: 0.9998\n", "\n", "episode 170 in 8.15 sec\n", "mean reward: 0.9607\n", "return standard deviation: 0.0882\n", "min return: 0.4643; max return: 0.9996\n", "\n", "episode 171 in 8.16 sec\n", "mean reward: 0.9719\n", "return standard deviation: 0.0503\n", "min return: 0.6354; max return: 0.9998\n", "\n", "episode 172 in 8.15 sec\n", "mean reward: 0.9753\n", "return standard deviation: 0.0244\n", "min return: 0.8730; max return: 0.9992\n", "\n", "episode 173 in 8.14 sec\n", "mean reward: 0.9751\n", "return standard deviation: 0.0289\n", "min return: 0.8815; max return: 0.9995\n", "\n", "episode 174 in 8.14 sec\n", "mean reward: 0.9746\n", "return standard deviation: 0.0725\n", "min return: 0.4192; max return: 0.9997\n", "\n", "episode 175 in 8.15 sec\n", "mean reward: 0.9752\n", "return standard deviation: 0.0336\n", "min return: 0.8464; max return: 0.9996\n", "\n", "episode 176 in 8.12 sec\n", "mean reward: 0.9769\n", "return standard deviation: 0.0291\n", "min return: 0.8393; max return: 0.9997\n", "\n", "episode 177 in 8.14 sec\n", "mean reward: 0.9810\n", "return standard deviation: 0.0224\n", "min return: 0.8959; max return: 1.0000\n", "\n", "episode 178 in 8.15 sec\n", "mean reward: 0.9842\n", "return standard deviation: 0.0177\n", "min return: 0.9100; max return: 0.9998\n", "\n", "episode 179 in 8.15 sec\n", "mean reward: 0.9833\n", "return standard deviation: 0.0210\n", "min return: 0.9133; max return: 0.9999\n", "\n", "episode 180 in 8.16 sec\n", "mean reward: 0.9871\n", "return standard deviation: 0.0184\n", "min return: 0.8865; max return: 0.9997\n", "\n", "episode 181 in 8.16 sec\n", "mean reward: 0.9891\n", "return standard deviation: 0.0114\n", "min return: 0.9421; max return: 1.0000\n", "\n", "episode 182 in 8.25 sec\n", "mean reward: 0.9845\n", "return standard deviation: 0.0163\n", "min return: 0.9198; max return: 0.9998\n", "\n", "episode 183 in 8.20 sec\n", "mean reward: 0.9901\n", "return standard deviation: 0.0115\n", "min return: 0.9499; max return: 1.0000\n", "\n", "episode 184 in 8.15 sec\n", "mean reward: 0.9832\n", "return standard deviation: 0.0188\n", "min return: 0.9049; max return: 0.9997\n", "\n", "episode 185 in 8.17 sec\n", "mean reward: 0.9858\n", "return standard deviation: 0.0168\n", "min return: 0.9231; max return: 0.9999\n", "\n", "episode 186 in 8.16 sec\n", "mean reward: 0.9849\n", "return standard deviation: 0.0179\n", "min return: 0.9244; max return: 0.9995\n", "\n", "episode 187 in 8.16 sec\n", "mean reward: 0.9906\n", "return standard deviation: 0.0108\n", "min return: 0.9386; max return: 0.9996\n", "\n", "episode 188 in 8.15 sec\n", "mean reward: 0.9878\n", "return standard deviation: 0.0175\n", "min return: 0.8778; max return: 0.9998\n", "\n", "episode 189 in 8.16 sec\n", "mean reward: 0.9866\n", "return standard deviation: 0.0163\n", "min return: 0.9165; max return: 0.9997\n", "\n", "episode 190 in 8.15 sec\n", "mean reward: 0.9884\n", "return standard deviation: 0.0148\n", "min return: 0.9264; max return: 0.9999\n", "\n", "episode 191 in 8.16 sec\n", "mean reward: 0.9861\n", "return standard deviation: 0.0172\n", "min return: 0.9188; max return: 0.9999\n", "\n", "episode 192 in 8.15 sec\n", "mean reward: 0.9863\n", "return standard deviation: 0.0138\n", "min return: 0.9350; max return: 0.9998\n", "\n", "episode 193 in 8.14 sec\n", "mean reward: 0.9887\n", "return standard deviation: 0.0153\n", "min return: 0.9008; max return: 1.0000\n", "\n", "episode 194 in 8.15 sec\n", "mean reward: 0.9891\n", "return standard deviation: 0.0157\n", "min return: 0.9090; max return: 0.9998\n", "\n", "episode 195 in 8.16 sec\n", "mean reward: 0.9887\n", "return standard deviation: 0.0140\n", "min return: 0.9323; max return: 1.0000\n", "\n", "episode 196 in 8.17 sec\n", "mean reward: 0.9880\n", "return standard deviation: 0.0172\n", "min return: 0.9048; max return: 0.9999\n", "\n", "episode 197 in 8.24 sec\n", "mean reward: 0.9873\n", "return standard deviation: 0.0189\n", "min return: 0.8821; max return: 0.9997\n", "\n", "episode 198 in 8.17 sec\n", "mean reward: 0.9903\n", "return standard deviation: 0.0118\n", "min return: 0.9272; max return: 0.9997\n", "\n", "episode 199 in 8.17 sec\n", "mean reward: 0.9881\n", "return standard deviation: 0.0186\n", "min return: 0.8778; max return: 0.9999\n", "\n", "episode 200 in 8.15 sec\n", "mean reward: 0.9904\n", "return standard deviation: 0.0134\n", "min return: 0.9127; max return: 0.9998\n", "\n" ] } ], "source": [ "### Train model\n", "\n", "import time\n", "\n", "# define number of training episodes\n", "N_episodes = 201\n", "N_MC = 64 #128\n", "\n", "\n", "# preallocate data using arrays initialized with zeros\n", "\n", "state=np.zeros((2,), dtype=np.float32)\n", " \n", "states = np.zeros((N_MC, env.n_time_steps,2), dtype=np.float32)\n", "actions = np.zeros((N_MC, env.n_time_steps), dtype=np.int)\n", "returns = np.zeros((N_MC, env.n_time_steps), dtype=np.float32)\n", " \n", "# mean reward at the end of the episode\n", "mean_final_reward = np.zeros(N_episodes, dtype=np.float32)\n", "# standard deviation of the reward at the end of the episode\n", "std_final_reward = np.zeros_like(mean_final_reward)\n", "# batch minimum at the end of the episode\n", "min_final_reward = np.zeros_like(mean_final_reward)\n", "# batch maximum at the end of the episode\n", "max_final_reward = np.zeros_like(mean_final_reward)\n", "\n", "\n", "print(\"\\nStarting training...\\n\")\n", "\n", "# set the initial model parameters in the optimizer\n", "opt_state = opt_init(inital_params)\n", "\n", "# loop over the number of training episodes\n", "for episode in range(N_episodes): \n", " \n", " ### record time\n", " start_time = time.time()\n", " \n", " # get current policy network params\n", " current_params = get_params(opt_state)\n", " \n", " # MC sample\n", " for j in range(N_MC):\n", " \n", " # reset environment to a random initial state\n", " #env.reset(random=False) # fixed initial state\n", " env.reset(random=True) # Haar-random initial state (i.e. uniformly sampled on the sphere)\n", " \n", " # zero rewards array (auxiliary array to store the rewards, and help compute the returns)\n", " rewards = np.zeros((env.n_time_steps, ), dtype=np.float32)\n", " \n", " # loop over steps in an episode\n", " for time_step in range(env.n_time_steps):\n", "\n", " # select state\n", " state[:] = env.state[:]\n", " states[j,time_step,:] = state\n", "\n", " # select an action according to current policy\n", " pi_s = np.exp( predict(current_params, state) )\n", " action = np.random.choice(env.actions, p = pi_s)\n", " actions[j,time_step] = action\n", "\n", " # take an environment step\n", " state[:], reward, _ = env.step(action)\n", "\n", " # store reward\n", " rewards[time_step] = reward\n", " \n", " \n", " # compute reward-to-go \n", " returns[j,:] = jnp.cumsum(rewards[::-1])[::-1]\n", " \n", " \n", " \n", " # define batch of data\n", " trajectory_batch = (states, actions, returns)\n", " \n", " # update model\n", " opt_state = update(episode, opt_state, trajectory_batch)\n", " \n", " ### record time needed for a single epoch\n", " episode_time = time.time() - start_time\n", " \n", " # check performance\n", " mean_final_reward[episode]=jnp.mean(returns[:,-1])\n", " std_final_reward[episode] =jnp.std(returns[:,-1])\n", " min_final_reward[episode], max_final_reward[episode] = np.min(returns[:,-1]), np.max(returns[:,-1])\n", "\n", " \n", " # print results every 10 epochs\n", " #if episode % 5 == 0:\n", " print(\"episode {} in {:0.2f} sec\".format(episode, episode_time))\n", " print(\"mean reward: {:0.4f}\".format(mean_final_reward[episode]) )\n", " print(\"return standard deviation: {:0.4f}\".format(std_final_reward[episode]) )\n", " print(\"min return: {:0.4f}; max return: {:0.4f}\\n\".format(min_final_reward[episode], max_final_reward[episode]) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plot the training curves\n", "\n", "Plot the mean final reward at each episode, and its variance. What do you observe?" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAACq+klEQVR4nOydd3yT1f7HP0+SJmnTvRelFCgUKGVUZC8F4aqoqNeN258Txev2egG3oFdFr+KeuDcKyJCyoexVKJTS0r1HZrPO749vz5MnadqmC1qb9+vVV5rxPDlZ53vOd3y+AmMMXrx48eKl9yI71wPw4sWLFy/nFq8h8OLFi5dejtcQePHixUsvx2sIvHjx4qWX4zUEXrx48dLLUZzrAbSV8PBwlpiY2K5j9Xo9NBpN5w6oE+iu4wK679i842ob3nG1jb/juPbu3VvJGItweydjrEf9jR49mrWXjRs3tvvYrqS7joux7js277jahndcbePvOC4Ae1gz86rXNeTFixcvvRyvIfDixYuXXo7XEHjx4sVLL8drCLx48eKll+M1BF68ePHSy/EaAi9evHjp5XgNgRcvXrz0crrMEAiC8LEgCOWCIBxp5n5BEIRlgiDkCIJwSBCEUV01Fi9evHjx0jxduSP4FMCsFu6fDWBg499dAN7twrF48eLFS6vk5ubCaDSe62GcdbpMYoIxtlkQhMQWHnIZgM8bK952CoIQLAhCDGOspKvG1OVYrUB5ORAbS9draoCdOwF/f6ChAaitBcLCgGnT2nd+rRbQ64HoaIAxwGwGVKqmjzt9GigpAUaMAPz8gG3bgEGDgPBwx2MYo/MFBtL11auBkSPp3HY7YLEAKhXkBgP97+NDt588SecC6LpMRs+1axf9P2wYkJREr3fNGkCjAaKigMpKoKwMOO88oH9/Or6wECgtpf8Fgd6viRMBtZrOWVhIt+n19PyTJ4vj9cvLAzZvpteg09F7HxgIXHopne/ECTqeMWD0aKC6GjAYaHwAsGkTjZExx19sLJCW5nhtJ07QMRYL/SUnA5GRQH09cOwYvff19fRnNNLYARpvVRWQk0PPMXUqvZ6kJLq/qAg4c4b+kpLo/cjMBGY1rpvWrqX3Si53/IWEABdcQPfv3EmvW9a4jqupodc+cyZdX78eqKgAbDZ6XQD8jUYaBwD88QfdJ5M5Xnvfvo7X/vPPzu8LYzTGUaPofVizhl6X2Ux/gkDHjhhB78PPP9O5BYH+ZDJg+HB6/3Q6YMMG8f7Qw4fpmGHDgD596L3MzHQczy8HD3a890eP0m21tfQ6FQpgyhTYo6NRvG0ban75Bf3794efRgOL1Yqq6moUDR0K//79MVAuh2znTtTU1CAgIAAKHx96zbNno0GjwdFffoHabkdkbS1QXEzPAwCXXAIEBNDnfvCg43fE77/sMvreHjkCZGU1vf+KK2ic+/YBx4/T+2+30x8A3HorXWZm0u9XJqM/u52Ou+IKdCXnUmsoDkCB5Hph421NDIEgCHeBdg2IiopCRkZGu55Qp9M5Hauoq0Pg8ePQJyWhIcIhwSFYrYDdDqZUou9nn8E/NxdHFy92e84+33wDMIaaUaOQ+vTTUFVVYfOff8KuVGL0nXciICfH6fG1w4fjwJtvis9v8/eH/fRpSF+R35kziPn9dwQdPoyT8+dDm5ICABi6cCEiNm9G5qefIiIjA32+/x6nb7sNxZddBiaXi8f3f/dd9PnuOzSEhUHXvz/CMjNReMUVOHXffRj4+uvwKyiArKEBVn9/HHr1VQDAhGuvhd3HB0cXL8agV1/FqXvuQfWYMYj45RcYb78dx55+Gv2XL0fQ0aPY9ssvsAQFIWn5cgScOIHAY8cgN5kAAIVz5yLngQegrKzE+KuvbvJ+nZw/H0VXXIHoVasweOnSJvdnfvopDH37os8336D/e+853WdTKrHrq69gDgtDyv/+B+zZ43S/vm9f7A4IAACMvusuBJw86XR/5YQJOPL88wCA8VdcAWVtrdP9pTNm4PhTT0Ewm5F+113Q5Oc73Z/19NMov/BCBO/bhxH/+leTsR966SXohg3Dkf/+F8P+8x/xdiaTQbDbse3HH2EJDUXS8uVI+PZbp2NNUVHY+c03AIDhTz+NUJfXpktKwp6PPgIAjLrnHgQeP+50f+3w4TigVAIAxtx2G/wKCpzujx8zBhnJyQCAif/8JxQGg9P9xRdfjBOPPAIAmDp3bpPXVnD11Th1772QGwyYNGdOk/vz5s1D3q230ud+ww1N7j91990ouOYa+J45g/Nvvlm8fXjjZfaCBSiZMwcB2dkYfffdTY4X3/sDBzBiwYIm9x966SUUDh+OkC1bMO6//xVv9wEQDeDkCy8g32SCedcupC5ZghCX4/e8/z6q+/ZF/PHjiP3gA8S63L/ryy9hjItDn6+/Rv/332/y/PyzTfz4YyR+8UWT+zevXg27Wo0Bb7+N+B9/dLqPyWTY1K8fAGDQkiWIWb3a6X6rRoMtK1cCIK2h9s5/LSGwLmxV2bgj+J0xNszNfX8AeIkxtrXx+gYAjzHG9rZ0zvT0dLbH5UfiKRkZGZjKV0UVFbRSLCgAPv6YLLJeD7zwAvDhh7T6WL2aVjEmE5CfDyQk0LHZ2bQSSk0F3nwTeOghWrXFxQFPPAHccgvg60srH0Eg669SAcHBtKro1w/YuxeYMoVWzOXlwCef0HGnT9PqyWym1XxAALB/v2NVCNAqoa4OGD+eVijDh9NKeORI4MoraSV79CiwdCkde8MNwDvvAC++CCxeDKSn05juvBO4+24a44YNdGxdHY1pzx5g8GDsef99pD/5JK2o1WpgyRIaZ0AA/f/aa7TS5O9BWBitHs1mWh1ptbTqDwuj3UZUFBARAeTlAStW0EqSrzqDgmhsGg2Qm0vHh4TQc9XVAdu3A08+CcYY9n74IdL79aP7/P1p1eTj41h1b9rk2DXt3Uu7ofPPp/cKoB2M1epYtQoCPWbAAODAAeCpp4CrrqJx+/gASiWtWqOjabW/YwetjgMD6c/XF+jbFxl792JqQgKtyvv1ozGsX0/nvfZaeuzRo/R9iouj1ePp07RLHDeOxlZURKtkm83xp1Q6dmKHD9PKmK/4Q0Lovevbl+7PyqLXo1DQqtJqxa5du3D+vHl0/6FD9NptNufXzo8/cMD5fQHo9xAVRd+9ffvo+6xU0h9Azx8aSuc9dYrGZbcDjMFiNqPaxweyqCgEqVRQ5uQAjMFutWL/vn0YPWoU7AkJsISGoqGyEr7Z2fBRKAC7HTarFWaTCeaBA+GXlASf+noYN2+GwBjU/Ltkt6NCqcTaHTsQFxICdX09jAYDzA0NCPD3h1wmQ0N4OEyCAGN5OUItFjQ0NKDBZEJERARGjx4NeWIi1m3ZAltVFSxFRQgNDcWE8eNht9thNpthiomBGYD+zBkUHTiAsrIy9E9KQkJCArZs2YLaiAjMmD0bqro6NBQV0VcwIwP5+flISUmBMi0N4ZGRqDtxAoGMISgkBIHBwajT6bBz507kMQY/Pz/09/NDgM0Gu8UCXX09jh47Bv+gIFgHDsThw4cxdOhQPPnkk+2a/wRB2MsYS3d73zk0BO8ByGCMfd14PRvA1NZcQx02BHv20CRXUEBf6O++AyZMoB/1+PH0o5w+nSbGOXPIGFgswNtv0w/1rbeAjRvpi56TQz+2JUuAdeuATz8F4uM9G0x1NfDYY0BdHeqOHUNQTg6Na+hQ4L//pUmjpITGdtttNIndeitNYGPG0DkYA376CViwgF7Pgw8Cb7zheA7+2fJtdP/+wMUXA59/7n5MmzcDjz9Ok/v48Y73LCYGePpp4F//ckxW55CTJ08iJycHs2fPPtdDaYLTYsMFrVYLjUYDGXfpdBDGGAQ+UXswrpTGnaW/v7+oYKnT6VBVVQW5XA6NRoPAwEDo9XrU19fD19cXfn5+sNlsOHr0KACgT58+kMvlMBgM0Gq1SEhIgEKhQFlZGaqrq1FbW4u6ujrYbDb07dsXw4cPx/bt21FeXi6+bj8/P5jNZlitVqhUKpjNZthsNuTk5KC6uhpjxoxBYGAgGGPIyspCcHAwgoODIQgCQkNDUVFRAUEQ4Ovri+joaMhkMuTn58PX17dVZc6ysjJYrVbExcUBAIqLi5GSkoKwsDBs3boVMTEx0Ov1MJvNsFgsYIxBr9cjPz8ffn5+2L59OzIyMqDVauHv74/Bgwdjz549UCqVuOWWW1BTUwO9Xo+qqirs2rULACCTyRAdHQ2TyYTq6mpERkaib9++0Ov1OHnyJCwWS5NxqtVqWCwW2Gw2p9vnzp2LH112FJ7SkiE4l66h3wDcLwjCNwDOB1B3VuIDd99Nq689e2gVzn3Ky5bRKuvPP4EZMxwTblUVPT4nhyb82logJYX+5/7Fxx+nv7YQGko7DwBHf/oJ4+fPd6zQuNshLg74/Xdg7FhanSYlkY+dIwi0ir/4YvITx8Q4P4d0kggOppVxWFjzY5o8mVa6rgwaBPzwQ9teXxdy8uRJNDQ0wGazQS5xiXUlfMHk6cTrju3btyM1NRWxsa6OB8JqtQIAFAoF7HY7iouL4efnh9DQ0CaPLSkpQWZmJkaMGIGExp2qVquF3W5HUFAQGhoaYDKZIAgCbDYbDAYD1q9fD7lcDsYYIiMjodfrodfrnV6XIAiwN/qt+W12ux1KpRIymQynT58W75PJZDhy5Ajy8/OxdetWVFRUYN68eRg8eDCKioqQl5eHM2fOgDGGqKgoZGZmYsuWLZg0aRLGjBkDm82GzMxMHDp0CLt27cKZM2cAAPPmzUNSUhK++eYbHD9+HL6+vrjmmmswYsQIrF27FkePHkVUVBTi4uIQGRmJuLg4xMXFwdfXt9XPoLy8HIMHDxavR0dH43ijmy0wMBCCIMDf3x86nQ4NDQ1YsWIF1q9fj4aGBgA0qU+aNAmjRo3CunXrsHfvXlx99dXYt28f3n//fQiCAKVSCYvFgnnz5iE5ORnHjh1DUVERZDIZ+vXrh2PHjqGkpARqtRqXXXYZhg0bBo1GA6PRiJqaGtTX16Ourg5KpRITJkxAWVkZGhoakJycjAiJC7sz6TJDIAjC1wCmAggXBKEQwEKQyw6MseUAVgH4B4AcAAYAt3bVWJzw96dgW0EBIO1rcOutwLx5NGECtAIHaMu8fz+5jM6cATIyyKXTiZhDQ2lM7ia1GTNoix0YCNx8s/PkzlGrmxoBd7Szj0N3QqfToa6uDiqVChUVFYiOjj4rz3vo0CEEBwejL3eftBGTyYTy8nJkZ2c3MQR2ux3Hjh3D4cOHAQAqlQpyuRxarRYAMGTIEMTHx4sr8bKyMhw/fhwBAQHYunUr1Go1BEGA2WwGYwwKhQJWq9Vp57Ft2zZs2LABffr0waxZsxAQEABBEHDo0CGUl5fjiiuugFKpRF1dHaKioqDVapGVlYX09HTI5XKYTCZs3rwZ+fn5qK6uxsmTJxEdHY20tDR88MEH4riffvppDBo0CLt378Ytt9yCuXPnYvXq1fj5559RWloKuVyONWvWIDU1FSdOnIDJZIKPjw9SU1Nx9dVX48iRI/i8cccaHR2N++67D/v378enn34qvpb4+HgcPnxYNGIAcNFFF+GJJ54AAOTk5OC1115DcnIyLr74YiQ3xka2bduGf//737j77rtxzTXXwGazYevWrcjKyoIgCOKf3W7HyZMnxc9j5syZmDhxIgwGA5KTkxESEoKamhpMmzYNJSUl8PPzw/XXX48dO3bgvPPOg0ajgSAIqKysxJAhQzBgwADY7XbRsF5wwQVQqVTijkO6s+PPLwiCaLTT0tJw+vRpKJVKqNwlh3QCXZk1dF0r9zMA93XV87sjeP9+4MsvgZdeajopNgYZ3aJQANwN0clGQKQll5JMBnz/fdc8bzfEYrEgPz8f/fv3b7ICLysrg0wmg0wmQ25uLqKjo0UXQnR0NMJa2vF4QFVVFWQyGUJ4PAa00j548CAGDRrUrCGoqalBMF9EuKGurg4KhQKlpaXQarUIaPy+NTQ0YNeuXSgsLERUVBTkcrnoEoiLixNdJtnZ2QBoZ6JSqRAVFQWFQoHAwEDYbDbRAAD0/h0/fhw2mw0JCQnw8fHBxx9/DF9fXxQWFuLPP/9E3759UVFRAUNj0PiXX36BxWKBVqvF1KlTceTIEVRWVmLw4MGIjY3Frl27oNfrIZPJEBwcjKSkJBw8eBDbtm1DWloaFi9ejKqqKjz55JOiL/uzzz7D5s2bkZubi+HDh+Puu+9Geno63nvvPRw+fBizZ89Gamoqxo4dC7VajcrKSowYMQKhoaHo168fpk2bhurqapx//vm4//77kZubi5iYGCQlJYExhurqapw5cwbr1q3D6tWrcemllyIyMhJPPvkkGhoakJeXh7Vr1+Ltt99GYmIi3m8M8v7666+YPn06FixYgKKiInG3Y7fbxYk5NjYWl156Ka655hqEhISgsrJS/CytVitmzpyJkJAQZGZmIiQkRByTwWCAr68v9Ho9hgwZgpEjR2LgwIGorKyExWJBWFgY7HY7cnNzERwcjNDQUMpeUijE3a1Wq0V1dTVqamqQlJSEwMBADBw4EBqNBrt3727nN7tlujRG0BV0JEaQf9NN6Pv11xTA9GAbebZoya98rjkXY9u7dy8OHDiA6dOno19jNgUA1NfXY9u2bWCMwWazQafTYc6cOTAYDFizZg1kMhni4+MRGRkJjUYDf39/hIaGwmKxwGAwICgoSDwXYwxWqxWCIIgTqM1mwx9//IH6+nqkpqZi+PDhEAQBO3fuxOnTp6HRaDDHTcaM2WzGb7/9htGjRyM/Px9Tp06F3W5HeXk5SkpKkJKSglOnTiErKwt2ux1xcXHiqq+6uhpms7nJln/NmjX44osvcM8992AiT0uVUFdXh7/++gv79u3DkCFDYLFYsGPHDkRFReHMmTOiC0etVmPYsGHYu3cvPvzwQ8TFxeGHH37A4cOHER0djQkTJiAgIAAffPCBODH9+uuviImJwSWXXIJvvvkGjDGcf/754sTNjbNWq8WePXswduxYVFdXIyAgAGVlZQgPD4evry/uu+8+FBcX49FHH8X06dMBAKWlpaKPXhAE6HQ6KJVKlJeXo1+/fkhOTsb69esRHh6OsrIyDBgwAEFBQdi3bx8iIiJgs9lQV1fn9F7I5XL83//9H4KDg6HX66HVarFs2TIEBwfjnnvugUwmw6BBg7BlyxZceOGFWL9+PWJjY1FZWYmnn34aEyZMaOJi1Ol08Pf3F+Ml6enpUKlU0Ov16N+/P5Q8SO7ymdhsNoSGhrYpftMWOvJ7PGfB4q6gI4agauxYhOl0lI3SjfAaAgeFhYXYtGkTwsLCoNVqMXnyZDFYWVlZCYVCgYiICOh0OpjNZgQHB8Nut8NgMIg/XKPRCLvdDpvNhsGDB6OwsBA6nQ79+/dHamoqFAoFtm3bhvLycvj4+GDixImIiopCQUEBtmzZgqioKJSUlCA1NRVyuRwHDhxAXFwcSktLMXfuXHF7zn/sJSUlWL9+PXx8fBAREYGpU6ciMzMTubm5sNvtSEtLQ1lZGcxmM1QqFWpra6FWqwHQJKZWq/HSSy8hLy8PQUFB6NOnD3755ReoVCqYTCbceuutmNeY8bNr1y58+eWXolEJDw8XV6tDhgxBTU0NfH198c9//hNhYWH49NNPcfToUUyfPh3/93//B7vdDo1GI+5IANpBKBQKceKqra2FRqOBj4+Po4NVo5vJbDaL8Qc+6VVUVGD06NFITk7G6tWrIZPJUF9fD4PBgODgYHF3VVVVhaCgIKjVapSWliIgIAAGgwEWiwXp6eno168fBEHAwYMHsWfPHpx//vkYOnQoAODgwYPIz8+HUqnEgAED4O/vLwacc3Nz8dVXX+HTTz9FUlIS/vWvf2HIkCEAgOzsbCxatAh6vR7jxo3DzTffjHvvvRd1dXV44IEHMHfuXGi1WigUCqcYg06nA2MMJpMJF1xwgds4zbmgqwxBj+tZ3BH8c3KAf/zjXA/DSzPU1dVh27ZtCA8PFyfBjRs3AgA0Gg1iXOIgoaGhKC4uBmNMzAIJCAgQJzmbzYbc3FxoNBrExsaisLAQ+fn50Gg0MJlMiI6OhtFoxLp169C3b1/U1dUhODgYcrkcMTExONK4YIiJiRH9x1qtFkqlEvn5+Thy5AimTZuGM2fOQKPRQC6Xo66uDr///jsMBgNiY2NF/z8AREZGQhAEhEsL+wBkZWVh3bp1GDx4MGpqarBnzx6MGDECzz77LN566y188sknsFqtyM7ORmZmJmJjY3HDDTdg0qRJGDhwIMrLy8WArCtpaWlYt24d0tLSoFAokJ6ejqysLJSUlMDX1xdGo1F0i0RERKCkpAT+/v5QKBQoLCyEj4+PeN6amhrY7XZERkYiNDQUCoUCBw4cQJ8+fZCcnAyZTIbU1FSsW7cOffr0gZ+fn7jSrqiogJ+fH8Y3ZqOtXr0aBoMBfn5+mDp1qpPve/DgwQgJCUGfPn1E4zRixAiMGDHC7fcmKSkJer0eI0eOxNChQ51W94MGDcLXX38tXi8tLcW8efNQVFSEyy+/HGazGUajEf7+/igtLUV0dDSsVqtotGfMmNGiy+/vQu8xBGVlUFVVUa69l3OG3W6HTqdDIK9obsRgMGDTpk3w9fUVJwVPfoA8RgAARUVFou8coNW21OUSHh4Oi8UCk8kkTsa+vr6IjY1t4qKRy+WIjY112t4LgoC6ujqUl5dj3759UCgUOHjwIIoa8855gFehUIhuKLlcLgZyXV0FRUVFCAsLw8qVK+Hr64vXXnsNfn5+op9ZEAQ8/vjjqK+vxxdffIHAwEDcfffdmDt3Lnx4xhrIwEhhjKGiogKMMdjtdsyYMUPcESUkJCA2NhYnTpyAXq9HYGAgEhISsGvXLuTl5SE9PR1Hjx5FWVkZ+vbtC4vFguLiYvj4+EChUGDWrFlOKZqJiYlizAYAYmNjkZKSgrS0NNTX12P9+vWora1FXFwcxo4dK362F110EXx9fbFly5YmAVCVSiVmQnmCIAiikeVGoL6+HjabzSnWwxk/fjyioqJE19zIkSPRv39//PLLL7BaraKb66KLLvIo1fennyhXoxtkVreb3mMISkthiI+H3yivtt255NSpUzh+/DguvvhiyGQy2Gw21NTUYNu2be3yr/If6sGDB/HQQw9h9OjReOSRR5CTk4PU1FSnuAAA+Pj4OE2i/ByujwOapoqq1Wrk5eWhvLwcMTExYjqlXC4XJyCe3y4lIiICdrsdlZWVWLRoEUaMGIHExES89NJL6NOnD0pLSzFjxgz4+fkBgHjJc+8XLlyIHTt2YOzYseJ9nJKSEtHdYzQaxXTRpKQkDBkyBIWFhTh06BBUKhXiGxMSFAqF6DrhjBs3DoMGDUJMTAx8fX1x4MABKrSSy3Hy5ElYrVb069evSZ6+6ySuUCjEVb+fnx/GjBmD0NBQhIeHO72f/v7+Td7vjhAZGYkzZ86In6PBYEBAQAAqKipE487jMjxHnxuwxMREKBQK9OvXD6dPn4bNZoNKpfLICFitlMENOMp2eiK9xxCkpSHziy8wtauyfry0isFgwP79+2E2m1FbWwulUon169fDZDLB19cXoaGhsNvteOSRR6BSqbB48WIolUrk5eXh448/xuzZszGucdn16aeforKyEs888wwAYNmyZQgMDMT+/ftx3XWUsBYdHY358+ejsrISgYGB6NevHyIjI/Hzzz9jw4YN0Ov1uOaaa3D55Zd7NH5fX18UFBQgKChInPh5FkhLCIIAg8GAxx57DAUFBWJxFs+3b2howCWXXCJOVDyIqtFooNFoUFpaKgZbAXJ58bhISEgIRowYgePHjyMtLQ2hoaHiSpjnxOfl5cFisbS4w1KpVKLrLSkpCdHR0aLRGTasST2oRwiCgEG8Grqd7NxJBc+tZUdLd5g2mw0ymQzTpk3DunXrYDKZoFarYTKZEBoaCrVajdraWphMJgwaNEg0ZgkJCThw4AD69evncdGfJIO1CTk5lO3NpbW6M73HEHjpFBoaGlrNZTabzZDJZKKLhnPkyBEIggCVSoWioiKxfF9aC7Bu3Trs378fAPDss89iypQpeOedd1BbW4stW7ZgwoQJSE9PF6srw8PDxVTSRYsWISgoCAcPHkTfvn3x1ltv4amnnnI7xuHDh8Nut+P999/H5MmT3QYDDQYDNm/ejISEBAwZMgRKpRLh4eFOq1ke9HX3HkgzSz788EMUFBTg5ZdfRn19PQ4dOoS77roLtbW1yM7ORkJCAoqLi6FQKMSsqFmzZiE4OBhr166FXq+HRqOB2WxGRUUFfH19YbFYMGXKFAQHBzdbTyGXyzFu3Djs2bPH48lNEIRWK3Rdycyk0pvDhx26fh3Fbid3S2IiFfy3hPQz0Wq1Yoxi0KBBOHjwINRqNYxGI2JjY8U0XpvN5vS+hYWFISoqCoMGDcJJF50qKRYL6ThqNKSz9+qrwCOPUO0pz17mGn8DBlDpkSfYbKSIci4SGr2GwIvHVFRUYMuWLZg2bVoT36vFYkFRURFOnDiBqqoqAOQa4EVHgiAgNzdXTAE8deoUGhoanAKnRqMRH374IQYPHoxp06bh3XffxbZt2xAREYFPPvkEu3btwscff4xt27Zh0KBBGDhwoGgQZsyYgcmTJ0MQBDGoOHz4cGRnZyMxMRFarRanT59GQUEBRowYgfT0dBQWFuLWW2/Fe++9h3vuuQdBQUGi+2L//v3497//DYPBgPDwcKxYsQJKpdIjl0ZFRQVuvfVWTJ8+HQ8++CAaGhqwbt06XHjhhRg9ejQAYFqjAq2vry9iYmJQXl6OUaNGISUlBTU1NTAajeJ7k5aWhk2bNkEmk6GmpgZTpkxBXFycGNBsjdDQULfpjp3Jiy/S5c6d7TcEL7xAxf4nTlDpTGMxLxqle1rE19dXLKQzGo1ijCE2NlZcWJjNZrHOhFdxS7/HvGo4ICCgRUOQl+cQEH79dRIaACgZkTsc3nyTxt2Su8hoJEEAXl94662keGM0OupGDQaqF+0kVZJm8RoCLx5RW1uLjRs3wmq14ujRo01y2zdt2oSKigoEBgYiOjpaDJDm5uZCJpMhPDwcjDHRn15RUeHkW6+qqsK///1vVFVVYdGiRRg6dCimTZuG2tpaxMbGQqPRIDExEeeffz5+/PFHXH755ejTpw9iYmIwcuRIpKSkiCtpfs7Q0FDRlRQTEyNWmHLi4+Mxd+5cfPfdd1i7di3OP/98PP3007BarXj++ecRHh6Oyy67DG+99RZ+//13zHWjyumOrVu3Qq/XY+XKldDpdBg2bBiMRiMuueQSp8fxHZFarRZTQbmejpSYmBgEBwdDq9XiwgsvFIPDXVVl2h4avV2YPLl9x9fWAv/+N/1fUUH6dr6+JPeVl9f68TxgXFpaCplMJk74gYGBCAoKgsFgEF1ldrsdJpMJsbGxTQyku1iRK1JB4QULgLvuoh1Raird1tBAcmEhIaRTabORaMDu3SQh9tRTtEv48ktSuPntN3rMF19QUiM3AmYz7Toefxx4+eXW34OO4DUEXlqltrYW69evh5+fHzQaDc6cOYPq6mpxwjKZTKioqGiS3qlUKhEVFYWTJ0+ipKTE6UcWFhYmBm3NZjMefvhhlJeX49lnnxVzxyMiIpoUWiUmJuJf//qXWIh0/fXXi/dVVlbCaDQiPj7eYzfIXXfdhbS0NJw4cQIrVqzAzTffLGb/LF26FP369cPmzZuxYsUKXHzxxR5Nvlu3bkVCQgJmz56N9957Dxs3bkS/fv0wZMgQNDQ0oLa2FlFRUaiqqhLHyxhrdhKSyWQYP348ZDJZk2yr7kBlJQmOLlpEYr3tISiIFF4+/5zaAERFkaslOJjUYDwhMjIStbW1OO+885xcdoMGDcLOnTsRHh6OgIAAWCwWyOVy9OnTp11j5YYgNpbGOnMmSYBx145KRVJlAAnQcv77X3Jx3XEHXY+OJkFdxoAtW+g2iUI3lEo6V+OGpkvx9iz20iKMMWRkZECtVsPf31/MiuEBT4AMRXNZPjKZDBqNBrW1tfDz88OePXtQWloKtVqNoqIilJWV4euvv8aZM2ewePFit1W0nmKz2RAfH4+KigqUl5ejuLi4iXqj9LFmsxkApRPecssteP3115GamoqUlBQsXLgQSUlJEAQBN954I6qrq7Fz506358rPzxffD61WiwMHDmDixIm49tpr8eyzz0Kj0eCaa64R0099fHxQW1sLq9WKhIQElJWVISgoqEUjExwc3C2NAECT1fLlpCBeXt6+cwgCcO+99D93BT33HBmGq6+mSdYVu50mYt5QbODAgbj44oubLEiSkpLwz3/+ExdddBEUCgXUajVCQkLaJOBmNjvGkJNDK/U1a6hfzUUXkfjwzJkUzzh5kgyA1AgAwGefkfvoiitIQuy558jYnTxJCjK+vqTYfs89jmMmTyah4q7Gawi8tAivIpX6xoODg1FYWCiKopWVlTUJDEsJCgpCQkICtFotnnjiCbz77rswm824//77cf311+PLL7/EtGnTMIbLa7uhrKwMZWVlTrfx6k8AYgUslwJISEjA0KFDUVJSAoPBALPZjOrqapSUlKC0tBQ1NTUwm80oKSkRzzFs2DAsXrwYzz77LCZMmCA+z8iRIxEQENCsIXj33Xfx1FNPwWq1Yvfu3bDb7Zg0aRIAYNKkSfjtt99w0UUXASB3EC8cS05OxrBhw6DT6ZpVJO0JBARQa4s5c0ilvTnMZmqNIVlDAKBGck8+SSvjRx91tEXIzqayn/fec6/HuGkTCfRywVypq1GKIAhOKcOCICA1NdVtjYE7tm6lMf3xB10/dYqCwKmpwMqVpGP588/AX3/RKp63xKipoZYnn39OAWalEhgyhOoO/vyTFOYB2g388AOJCFdUAB98QK6yb78lA3P8OLVE+b//A44da0ETrQN4XUNeRFwzXUwmE4xGYxMhN67Pk5ubi7S0NBQUFHgURN25cydsNht27dqFrVu3QqvVYvz48aioqMA9kmWQxWJBSUkJAgMDERwcjPr6elGfngu2cdExrVaLmJgYmEwmcdU8e/Zs8XX4+voiLy9PDCBGR0cjMDAQ/v7+kMlk+Ouvv6DT6ZwkF1yRy+U477zzsGvXLtjt9iZup7y8PNTX12Pv3r3YuHEjIiIinOIRro9PTEyEWq1GdHQ0fH190adPnyZFYT2JH36gCc/fn2S8muPAAeCrr2jVu2GD4/ZDh8gHfsklpO7OOXGClOClvXmkcN1IT2IIHL2e+j0leqjEa7XSBKxWk7vKZqOVvOvr5EXd11zjMFpBQbQD+OsviiN8+CFw4410nyCQuntICLmR/vMfSjMNCACef54MwFdf0WN1OlKQ37ED6NOna4L+XkPgBQAFa7du3SoWNp05cwYHG3uzuhZgARSIzcrKQkBAgDgZt8bWrVshl8vR0NCAd999FwEBAVi0aFGT81dXV2PYsGGoqqoSg3+TJk0CYwyrV6+Gv7+/KKmgUChQUVEBAKLOvNTFkpyc3CRILGXgwIHYunWrW0PADaFKpcLYsWPx119/ITs7GykpKdi8eTPMZrOoFw9QbcPx48dx1113uY1RWK1WKJVK+Pr6IokvGwFMnDjRIy397khdHbluXniBJrGWDMF559Fknu6idnPqFF0OGEDHGwwUG8jLI9eISkWT6C23OB/3z3/SpaeGgK/kP/jA4advjU8+oYZvP/1E1yMjgV9/bRoUv+MOGoc0W1kmo53Ejz9SANk1JCGTAe+/TzsIXudqtZLy/Y8/khEZOJAylBISyGBmZFR5NvA24jUEf1Psdjv0er3bCa6+vh5yuRx+fn5iJWpmZiZqampw9OhRhIeHY9u2bQgLC2vW9y+XyxEcHIydO3d61BymoaEBmZmZmDVrFrZu3YrKykrMnj27iRHgPv2UlBSxCIg1tvEDKIDMReUiIyMxZMgQZGRkoKioqF2r6qioKMhkMlgsFiclUoBiHwkJCTh9+jTS09Mhk8mwc+dO9O3bF0uXLoVarRYDjiEhIWITlUsvvVQUZ5O+/3q9XpQ2kNLZVbZnk0OH6HLEiNYNgSDQRNpot0W4zz0ykoxEdDR1WbXbgUmTqPFfQQHw0Uc0ec6bB5SVOVpWe2oIeJvnn3/2zBAwRp1dx48HLr+c+lZVVwPXXUcZQNLSjbg4Mhqu9OtHMQBBcK9uc9VVztcVCrpt+XK6/txzlE3V1VXL3hjB35Tq6moc4r9SCZWVlfjjjz+wcuVKrF69GqdOncKOHTvESe/EiRPYuXMnIiMjnVapdrsdr7zyChYtWiTexnV6oqKiYLfbsX37duTk5Ig+dy4VDQB79uyByWTC5MmTxYCwq4qi1WpFSUkJkpOTRa0d3i6Rw1sJMsYQFhYGtVqNqVOntsnnK0WpVCIpKQm1tbWigB3g6EiWnp4uPk9qaipWrlyJzz77DDqdDpWVleBKuNdeey0AYNasWfD390dNTQ3q6uqcqo6NRqNbYbiu4PRp9wFWV3JzqWV3ezlwgC65Iaivb/6xDz9M8QDXgDJfqQuCIxMnPJy6w06fTgbi0CFqLnjbbcDatTQRA+SGyc/3bKy8sNrTfITqanJj/fOfNDae1FVYSK3CPYEHjJOTqbeUJ1x/PbViBqgmw2YjQ/PSS54d3x68O4K/KQaDAeVuUjhOnjwJlUqF0NBQ6PV67N69G2q1WlwZc2kC1/zqr7/+GmvWrAFA+jZSV1BdXR1eeOEFsWlGbGwsBg4ciCNHjsBkMuGdd97BF198gfDwcIwcORLR0dFQqVQYJdF9MpvNqKqqQlpaWouunIiICBw9ehQKhUJcSavV6hYDza2Rnp6O9PR0rF27VpS7MBgMCAsLg1KpRFhYGAoLC/HQQw/hnnvuwXfffYfIyEiUl5eLssuXX345/Pz8cF5jK1HeDrK2tlaUzrDZbGdFyfL998mvPXMmZba0JNv02mu0Qr711pYf1xwHDtCkFRNDbpGWsmtXraJL13KMsjKHDENcHPnMIyOB+xrbVsXHOzqlvvEGMGsW8MwzZAS++KLl55TCM0obs5OdqKigqmCpRy8sjDKSGmvPAJAh+vhjctl4wvjxFOxuRjjVLZMm0THXXUeGYMAAal/elRtH747gb0p1dbXYhJtjMBiQl5cnTkYajQbR0dEIDg52aizu6q/OycnBRx99hPRG5y6XhuZ8/PHH2L9/Px544AEsWLAAiYmJOHbsGJKTkyGXy/HAAw8gOzsbd911F3x8fJCQkIAHHngACoVCdAVVV1dj9OjRSE1NbTGNkqdQMsY6zaXC9X14ZhNAbhzu9gkPD4fZbEZiYiIeffRRqFQqPPLII2IKbFxcHJRKJS655BInaYaUlBQYjUZUVVWJDWras2tpC7/9RivnMWMoi6e1yf3ECZp829tD5cABmuQEgdwnvJGfK2YzuYCefJImUymZmcCKFfR/bCxNyn/8QTsDwOFbj40FHniAnuuLL+jyuuuaGpbmSEgAHnusqa/+l1/I8Hz2WdNjfHycJR+WLyeXVEsNBaXccAPFT3iQ2FOuuIKCxP37k6EEulbd1Lsj+JvCW+OZTCYAJJlgMBicJIM95ccff4RKpcJ//vMfPPnkk9iwYYNTIdehQ4cwevRosfJW2sVr69ateOaZZ5CamooLL7zQ6bwGgwGlpaWib9+TAh9fX1+xWKizK2u5SigAUZ8fgBicBoDp06dj4sSJUCqVGDx4sKjHL4WLnsXFxSE5ORkKhQIJCQkdbqPpCW+8QavVjRspO6Y1Nm+mFW9BQdMJ0pWTJ4GFC6kqdtcuevzvvzvcQQUFNGm5BoMBcv/YbLS6LSx0Nj6C4FitN7aVwCWXAA8+SK/n6qspQHvZZbRiZ4wCqKNGURbQkSOUltlSt1mAJvUlS8i3z/31mZl0fsAhFcH56iuSzHjzTcdYfXwoQ8pT5HLnAHJb4OGqJUuA+fOBtDTSOOoKvDuCbkxNTY1b905rMMZQU1MDlUoFo9GI2tpa5OTkoL6+vklTlNaoq6vDhg0bMHPmTAQEBGDatGnIzc1FXmOETq/XIz8/Hymuv6JGJk6cKMYWBEGA0WjEqcY0Ea1Wi6FDh6K6uhpJSUkeZc4IgoCYmJgW6xbaS3BwMARBQENDAwRBECt9XQXYuNuMV0C79jHmKpcymQznnXceRo4c2SVGoKKCVv2NPdbBGK3Qp04lI5CZSSvS0lLglVcGQRAoT3/aNJpYa2popW63OyphW+K66+i4khJKbwTIJcQFRl9+mdw27uA+9bVryYA0ho6Qne2Pm25yiMpNnOgIoHI9nxtuoEyiZ5+l64JAAeT582ncY8c6AsctwQPZkvbD+PRTMkLl5XQeKWvW0G6hCzpOton776fPyE3yXqfhNQTdmMLCQnHS9ASuTMnbCMpkMphMJtTX14vNUjzJ8JHy559/wmKx4IorrgBAAV6ZTCa6h7Kzs8EYa9YQABA16QGITWl45s+AAQMwY8aMJvr4LRETE+M2pbWjKBQKxMTEoLq6GmPGjBGNja+vr9g7QQo3BK5NVEwmU4frAsrLaWK1WJp/TGYmFTTNnk2rbIAkCx59lP6vqaFVbU4OsHs3vf+vvkoreh8fhwEBaHLX61ueUE+fpljCqVPk3vnkE1ot84yWlrKG5HJawTfW2YkB45MnA/Dll47JdtAg8osLgnOKpq8vBZBd4TbYk4BxYza0GOAGyMCEhtJrc5WyyMnpGRLSnYHXEHRjKioqnCpfAco8aY6ysjJs2bJFlHzw8fFBfX09ysrKmpVLbo2MjAykpaWJBThhYWFIS0vDxo0bwRgT2zDyHP7WsFqtiI6ORk1NDWQyGYKDgxETE9Mmf3+fPn26THBtxIgRuPTSS9FPog/AheAauBxmI+np6bjtttvEKmKOxWLpcI/b118nf/p77zX/mNJSuqysJI0aQQAGDyb3C+CYJI8fB6qqVKIy5oUX0up64UKqCAbIEPz0E+X6uxRwAyC3DmM0GScl0Qr1+eedV8wBAbTDqK6mVfzq1Y7jL7uMiqL4xMqf48CBYKhUDrcUY8C771IA2pNwSp8+9PxtKSqTkpZGu5jzz6f6Aik8m6k34DUE3RTGGCorK2EymWAwGMTbtmzZIso819TUYNeuXWKaaGFhISorK3Hy5EnY7XYolUrU1taisrKySWcrTyguLkZBQUET/Z/p06ejoKAAJ0+exLFjxxAfH++xDo4gCBgwYADMZjNiYmLavEPh5+gqeNWxK2FhYWK8hePj44ObbrrJrXZ/RwPZPLj4zjs06bpj7lzyYS9ZQnIHq1ZRxhBfN3BDsHkzXd52GwmfffABpUJWVNDj4+LIqHDbl5lJrp8vvnCcSy6nCX7hQnpsXBylnkrz8bmPvr6eNHT+8Q8yFlJ49mx5OT3Hhg1ReOghZwmJsjLP/fBKJQWRPdkRuHsfFywgY+vj45wdZDDIUV7u3RF4OcvY7XZkZWWJq3+9Xi/KGdQ3RuMaGhpQWlqKEydOwGAwYMOGDSgsLMTx48dhMplQVFSE+Ph4FBQUQKVSQaVSoaqqCgaDoV2uFK6tM84lXWHSpEmQy+VYuXIljh071qJbSIrNZoNCoUB0dDRiYmLarf54LoiJiWlxN5aTkwOLxSJ+fh01BEOHklvn2DFy/7gjJIRWsvPnU1D1k0+AV15xrNB9fWni5Y1RkpJo4ktMpDjC0aM06cbE0I6AF0jt2kWaQc8+S+eSFjMJAuXj892INGNHuhb45RdKX122zPF6XniBsnMAEpa76y6gTx8DFi50Pn9REaW0ekp0tPtdjCvczdbYwM4JhcJhCA4cAIqLfREf33t2BN6soW5CfX09srKy0KdPHwQEBIiFWHK5HJWVlYiJiYFOp4OPjw/y8vJgs9nAGEN4eDhKSkpw4sQJWK1W+Pn5Qa/Xw9/fHz4+PjCZTG1eQXNf+I4dOxAfH484nsoBikMEBQVhzJgx+P333wF43srQYDCITcbPP//8du1SzhVRUVGIjo5GXV1dE7lou90OuVwuat6HhIR0OJi9ejUFTuPjSbTsssuaPubnnymL5rLLaLX7ww+Udihl2DDK9klO1iIpyZFWw91EKSm0q/D3d2TSbN8O7NtH2TTz55M76PLLqaBp0SLy47/4IhkEaXx/6lRqrFJURLIMM2ZQcDg/n67LZOTyefllWvE/9hhgtWbD19e55Lat+nuvvupZLQGf6KW7jUsvpV2FQkGGwmbj70O6mN3UG/Aagm5CVVUVysvLUVtbi4CAANGHznvWpqamQqfTiTnvp0+fFpuRBwQEIDc3V5zwXeV1PTEEe/bsga+vL4YOHYonnngCp06dglarxaWXXio+RqfToby8HLGxsXjkkUdw+PBhKJVKseuWwWCAXq93en5poxij0Sj2sD0bhVWdCe98tmbNmiaGwGKxICgoCEajUSzO6wh1deRWeeUVEiyTShnY7bTCnzaNisFUKjIETzxB97skMGH9errMyNiL2Nip4u3p6WRkXnqJql7r6x0BXF4mMm0aTfj9+5NB+eYbapICUPzClcRE+ps/n3LyuWgaLyTr358mXX6OsWOBjIy6drxDzrgUqDfL5MmUyind2JWVUeEYdw1Jym6aiNz9nfG6hjoRq9Uq+u/bSn5+PgIDA1HcWEVTXl4uFndVVVXBYrGgsrISKpUKkZGRiI2NFSd4jUaDqqoqt+6IhoaGVgPFVqsVzz33HBYvXowTJ06IRsFut4vxAZvNhrq6OgwePBharRahoaGYMmUKxo0bB6VSKWYnhYaGoqysDIwxGI1GFBcXo6SkBFarFWazucsLqrqSsLAwBAUFifpHhYWFYIzBYrGIwWtp/UFrWK3uNWSys+ly8GCqCZDmx+/aBVxwAal3St05vHSDr/RbQ6mkLJkbb6TGJ7feSrdLC5+42FlhIfn8AfeZOxytlvz++/bRSnrMGGf9He5mKSwkI1ZS4tlYW2PPHs/y62NiKCby0UeO20wmSh/96CN6D7j76O67T6EN7Qp6PF5D0InU19cjm/+K3WC1Wp0ygDiMMVRUVCAyMhJFRUVoaGhAWVmZqLfDGENZWRkqKyuhVqshCIJTkFUQBCQlJbkNWvbp06fV9nv79u1DfX09KioqsHDhQvj4+OCdd97B6tWrRbmHqqoqDB8+HAMGDBD7vUqpqqrC5MmTMXnyZMTGxqK0tBRarRYzZsxAfHw8SkpKkJ6e3uY6hu5GfHw89Ho9jEYjLBYLzGYzzGYzIiIiRMPsSeCcMVqFLljQ9D4ujjZ4MLlVnnrKodDJK1pzcshPzw3BxIm0ur38cudzrV9Pfvfnnms+jrNrl0Nd87HHKKg8aBBNnHFx5OrhufctlUPk51NcYNs2Oi4ighRDec0AD7xedBE1e3fN0mkv//kP8NBDrT/u9Gl6HdIUV5OJ3FuXX07prdwQKBTNROj/pnhdQ52I1WpFXV3zW13exSotLU18/NGjR2E2m8EYg1KphNlsRmZmptjfF6DAY3Z2Nurq6prNT3d1/xQXFyM0NNSjtNGMjAxoNBrExMQgJycHM2bMEI0Hl6iw2+2Ij48XdfwZY+Jz2mw2+Pj4ICYmBoIgYNKkSTAYDLDb7fD390d4eDiGDRvWo3cDnMjISBw7dgw2mw1hYWFoaGiAxWJBcHAwwsLCoNVq3RpkV/gK251I2/HjZCT69aPdwUsvUbC1f38yICoVia4ZDM5uI3dfDZ7xeuZM8/EYqYJ4//60Oubjio8nA1NcTAqhLdX8SXcu3LeelQWcOUM7Db4eycqiy5kzaSLuKEqls0unOXg6q3TTzncE27dTsHvAACoyA2o6PrAehHdH0IlYLBbodDonxUkptbW12L9/v6ifX1paigMHDkCv1ztN2GfOnHGa8P39/VFeXu40+baEzWbDnXfeiW+//bbJfUuXLhXF4/iYt27divHjx+PWW28VBdRczyeXyxEUFAQfHx+Eh4dDr9eL95tMpiaS1X5+fqKrSqFQdEsjcOCAc1GVJwQFBYnS3SEhIWhoaIDdboefnx9iY2Ph4+PjkYQHb8f4j380ve/4cZqQfHzIf69QOLp6TZxIkzv340sNgTv418jPr3kpUn6Oiy5yyFLwTU1yMgVPjUZHBXFzuDMEzz1HukFffOG4j+sRuZOiaA8+Pi0X3nHcbGRxxRX0ns6bR0FstZpqMvr2NXTO4HoIXWoIBEGYJQhCtiAIOYIgPOHm/iBBEFYKgnBQEISjgiDc2pXj6WqsViuMRqOT0JsUrVaLoKAgbN++HUajEceOHRMVLnnwNCIioolmvSAI4ircEyoqKmAwGHDmzBmn23Nzc7Fq1Sr8wKUcQW4hrVaLadOmYfz48fj111+bVPnqdDqnhvAJCQlOhsBoNHbLDlt//kl6Nc1x003kVmgLPG7Dm59brVYIggCVSoXY2NhWC91++olcOtwQcDfFf/5DPvnSUtLi5751pZLiBFlZtJLl1a9mM/3vLptIysiRFLx9/PHjzT6G7whctfEBSrXct48aw+zd2/JzcUNw5ZWUGgqQe+v0aecA7fff0+vvLJUQpdIzQ+DuMW++Sa4wnj5qNFJrytraLtRz6IZ0mSEQBEEO4H8AZgMYAuA6QRBcdQTuA5DFGEsDMBXAa4Ig9NhYPfcbu1agAhQH0Ov1CAoKgsViwaZNm1BRUdEkwCuXy92mHkZERHg82ZY2Jnm76hT9+eefAIBTp06JuxLuFuKZP+4CzgaDwSmFNDw83GnXY7FYuuWKf9YsymN3t0HLzyexsl9+ads5udZRbGws/P39xSwulUqF4ODgFus1zGaaJMePdxiCr7+myzfeIL+/xULumfPPdxw3bBjtCLhEwvXXU8A4NrZ1jXuFgia72NjmfTA8yamjwVsfH3Jb9e/vyF7iAeJbJUs8jabtKaKtPa8nriFuCNyJwPH00cJCksHYvbv7fZ+7kq7cEYwBkMMYy2WMmQF8A8B1/cIABAi0/PUHUA3AzQauZyANILpiNpvFArHw8HDU19d75EvmyGQyj6tw3RkCm82GdevWiQJpmZmZsFgs2LJli6im2RyCIDiJpnEXEa83EAThnHfZ0mrJB829XtLNU21t08e7bJbaRFJSEgYOHCiK5PH4Tmtw18Tp05RRAziCr7yxu7sau6FDqao3M5Ouv/EGTawvveTZBNgaKhW5Rdy5qQwGMkqCQM/bGkuX0nvPQ2Vct781t1JHePRRh0FtCf7+cyPLG8ovWeJIH+XGQi7v4pZg3YyuNARxAKQyToWNt0l5G0AKgGIAhwE8yBjrseF6rlrpbkfA7+NERka2ms3TXrghqKysFDN81q1bh5qaGtx+++2IiIhAZmYm9uzZA71e36RTmBTGGFQqlZPR4hLLOp2u0yppO8qpUxTgfP11us7VLF9/nUTFXJGuftvaBjA8PBxRUVFOcR1PtI/4RHTLLaRxw/vvMEYTenO25MknKdvl6FFHNs6ff1ILw85yrzz+uHtZB19f2jkBnun5yOVkDBpVUTBmDLBlCzWS6SqGDqVdVmvMm0d1Eby7mclEE79M5nANObKGepch6MqsIXdRTdd39yIABwBMB9AfwDpBELYwxpxyKQRBuAvAXQBVeGbwmvk2otPp2n2sJ2i1WoSHh+Po0aPIyclxus9qtUKlUokVw1KkLR07g4JGR7LdbseZM2ewe/duvPfeexg0aBCGDRuGUaNGYfPmzSgrK4NGo8GgQYOafX6uWeT6vlksFthsNmi1Wvj5+WHr1q2dNn5PcP0sCwp8AZyP887LQkZGOQwGORYuDEVsbD0yMpoa5i1b4gDQcvW337YiKKh9G1G+K9i0aZPbcUkxGuUYPXooEhOL8fnnBpSWjkRRUS3++isLjE1BUdFpZGQ0L5qTnh6ExEQVNmyowMsvU8HA5s3un8uVjnz3AwLGwmBQQ6tteXwA8O67IwEEISsrA9JM6ua+Hp3xm8zJ0aCoyA9TplS0+tgVK0bip59sWLr0UGMcYAIKCk5g3jwdlEqGnTsZgHRYrYYunSvaS1fNYV1pCAoBSDe68aCVv5RbAbzMaFmZIwjCaQCDAWRKH8QYex/A+wCQnp7OWlrBtkRGRkaLq9+OsmbNGmi1WvTv3x8jXHrTFRQUYNu2bW596TqdrlNX1FVVVZDJZLDb7aiursZHH32E0aNH47nnnoNarcY//vEP7NixA9nZ2bjyyitb9O/X1NRg8ODB6O+ivqXX6/Hbb7/BaDRi8ODBYoyhKzCZyM/94IOOBiaunyVftY4YMQRTp1IoKi2NCpduuQUYPtz5nGvX0uWaNcCUKRPRTnFWbN68GRaLRRxLa9+xESOA9etD8e9/0w4mODgC06ZNwYYNQGJiPyQl9Wv2WHen9fT73JHvfkgIpZCed14/TJ3a/PgAx+dwwQVdPy7OqlXU31iqWeSOPXtoVzVtGr1v3D04fHgybr+d/m+U14JGo8bUqe1vf9pVdNUc1pWGYDeAgYIg9ANQBOBaANe7POYMgAsAbBEEIQrAIAC5XTimLsVkMkGtVrtdXev1+jZ3BuMxhbZSWlqKgQMHIjs7G9u2bYPFYsGcOXNEV8bw4cPx66+/Osk/tIQ7F5ZGo0FcXBzCw8MxoIuVuZYtIwmFvn2Bxh7xTeC+8oULKfvlyy8pi+X118nl4WoI/P3JnXDRRR0bW3BwcBNV0pY4eJBcFAC5NJ55hlwT06e37Xnfeafz3EKtwb1entQCJie3XxK6vXgaLObtMLn7h39svI7AYiEV159+AhhrprHC35QuixEwxqwA7gfwJ4BjAL5jjB0VBOFuQRDubnzYcwDGC4JwGMAGAI8zxirdn7H7YLPZmihRMsacGp9zSktLYTAYoNVq26QAWlpailmzZuH48ebT/ux2O/bu3StKHvz444/Iz89HRUUFhjfOfJsbNYjdKYS2ZAT45MYYQ0AzPQAnT56MIUOGeBQobSunT1OaoVZLf3J5y6mSje0SoFBQCuDNNzsCx9ICooICkl9+6inS8fn5Z6CFYvBWCQ4O9jjWU1Li3NP3uutI/99oJB2fNvQgwj33OHoJdDXnnUeXngiwHTnivkiuK/HxIbG45uS6OTxGww1BQABw332U4rp4MS02QkOptiA01IN81L8RXVpHwBhbxRhLZoz1Z4y90HjbcsbY8sb/ixljMxljqYyxYYyxL7tyPJ1FZWUlDkjbHMGh2Onj4yPuCBhj2L17N3JycqDT6do0YXJZ4yxehgkyDtwwlJeX4+GHH8YjjzyCjz/+GMeOHcPbb7+N5557Dna7HX379kVQUBCqq6sRGRnZJmkHs9mMkpISVFRUQC6Xd1kTmJZYvx745z+py9a+faSS2VJVa2goVar6+lIVrN1OE5ggOBuC1asdGvlWK8kot0Xy2JX4+HiPd0SuOQSFhdQroKqKjMJff7V/HF3JBx9QQNuT5uk8hfRswn9WrdUS8Pu5QYiJIZfS6NGO9NGKClpA6HS9S3Shd73aTsJsNuPMmTM477zzxJx/S+O3TKFQwGw2i4HUuro6nDhxAgqFok2yy2WNAutFPOEcwHvvvYeDBw/ixx9/xMcff4zjx49jwIABWLVqlZgqyltbRkdHIzIyEnV1dR73C+CuLa1Wi+TkZJw+fbpNKa6dCdeXX7aMtGvq6qg69aab3D++spJ8wAqFQ0UzJoakkquryUVUUUEuAIOBXDGXXEKrwo7kz7fFdefS6RIffkhGjqc+9ia1y86Eb7QtlpaNEDcAvOubzUZ/Pj6OrKHMTPpevPNO6/2z/054JSbagcFgQG1tLWolCeq8wpRjNptRVFQEpVIJi8UCk8nUJteQO0Nw5swZsaH98ePHMXr0aDz44IPQ6/XYtGkTJk2aJD5HdHQ0ohu1AzzpB9zQ0IAzZ86I2jn9+/fHlClTusTt4wncELz2mmMlnSuJHm3dGo633nJc37OHJvzycsexUVGUp28wAL//TrIM559Pk+/GjbTb4E1ZBg1ypJ52Fa4SB+np5Nvm/m2vIWgfN95IE3hLO0aADEXfvo402fXryXDs2OGoI+CfkbeOwEurcNcPr84FHDsCgAqsKioqcOrUKQQFBUGj0UCv17epQQw3BFyW2m63i0bh0KFDOHPmDJKTkzF06FAMbKzauemmmzBz5kxRqppXInuyI9Dr9UhISEBNTQ0YYwgJCUGfPn063GClvUg7Tu3aRT9Uqe/5mWeGYf58xw+XT6Y8wwUgnZ3sbBJRKyykYq3zz3eszKOjyRD8+itw4gTw8MNd+5r48774Irmv4uJocuJfHa8haB8xMeQGbC3v4bnnqPbi11/pOg/z+fo6XEO9tY7AawjagcFgQEhIiJOWj7RNoZ+fH7Zv3w69Xg+VSoXAwMA2t2Xkrp6SkhLYbDZUVlaKhWqrVq0CYwzJyckQBAH3338/brnlFgwcOBD3338/3n33Xfj4+GDIkCEIDw8XDUVLNDQ0YMCAAbDb7QgLCzsncQEpvIUiQCl94eHOhiA+ngLy3DPDDcHmzZRZtG8fTfQyGfm3CwochoDDDQE/9qWXuvY1aTQUiJwyhZ6XZ7t4dwQd48QJ6jOgbSXRJyGBvh9cslqaNbRwITXS6a2GwBsjaAc6nQ6BgYGoqalBQ0MDVCqVk0Z/QEBAk0ybtrpYysrKRFnq8vJycWcgk8nEQDXvFTB8+HAxS0itVqNfYxfy6dOnY7qHeYmCICAqKgoDBgzwuBF9V/Lpp7RiGzoU+L//I9eN1BDExRkRFeUnGgL+A/bxIakJ3nbx00+pWYrRSJLK0syXmBhK3/zmGwrW8i5fXUXfvg7df8Ahnzx4MO16PLDXXtywYwd9Ry680FkB1ZVVq0hbiid5SXcEPOuM52Z4XUNeWkWv18PHxweMMbEjmaW1lIVWWL58OT5o7NTR0NCAmpoapKamAqA4AXcLjWyc4cLCwpz0f1zhBsQTrFYrFAoF/P39MXLkyC6vC/CEfv2AIUOoXeNrr1HGSlKS4367XcDevdQSEXCsqlNTaXX33Xd0fd8+WuklJDh+7BkZtCqPj6fnOHiwayUQmuPJJ4Fvv6Wdwpgx5Nby0nakweKWePdd2jW4qyPYtYu+MxdeSMWGYWFNq9H/zngNQRuxWq2wWq2Qy+Xw9fUV3UNGo9FjUThXbDYbVq5ciZ9++gkmk0mMD4waNQoAGYKCggKo1WqMa8zh47uB5jAajW4L2xoaGpDnUvGj1+vFpjI+Pj5nLS6QkkJ+W1fMZgrcHjlCHbMefphkmaWum5tvzkNsLNUDMEY/4CefpB/5s88C//sfPY7bylOnHI3dp0whY5CQQNeHDwduv512BV3J7t3k4uK9BAYPJldVQQFlEHlot724wDfbrRWV8U07vxwxgjSWAgNp5/jAA7RLnDEDUKt7rORZu/AagjYiVRYNCAhAfn4+rFYrTCZTuw3BqVOnYDAYYDKZsGvXLnElP2TIEKhUKhQVFaGwsBCxsbFi8/fWDIHFYnGbpVRbWyvuZjgmk0nMMDpb1NRQAxZ3/QBKS2ny37Gj+eOHDq3H009T0DUvj1w+0omcK3ZzwTl3CqRSTCZA0mKhSzCZqGaAB4337ye31KFDVByW37KMj5dm8HRHwO/nl+PGkeqqn58jWHzyJBUyms2eJ3b8HfAagjZiNpvF7B+5XA673Y6qqiro9fp2G4LDjW2yNBoNNm3aJO4IoqOjERsbKxqCPn36YNCgQZgzZw5mzJjR4jm5yJ3NJXmdd9NylcruKiXU5uBNTtx5t6Tpn5yFC0knnnP0aKAYH9i7l7KDpFpc/FhuCCZObFlllOeRdyX8o+Abrq+/Jjlpb7C4Y3haUGa1knuxUR8QWi2lHAOO9NE//qBCxoaG9v2WeyreYHEbaWhocFpNK5VKbN68GWazuc1dun7++Weo1WocPnwYffv2RVpaGtauXYuwsDCxb0FSUhI2b94Mm82GKVOmwMfHBwvcdTx3g4+PD8xms6iQaTKZEBAQgJCQEFRWVoqZQXa7/awXjvHC7JMnm97nzhDU1joEzQDgvfeSxKboOh2lBD7+uON+/lHwy+xsqjJujrNhCPj5uSHgnbW8hqBjTJpEQV4eA2oOi4ViT1wPkrsQDQbH5+/4jLzBYi8twBvNc0JDQxEQEIC4uDj88ccfyPdwf2+z2fDRRx9h6dKlOHjwINLS0nDhhRfCZDLhhx9+QHh4OBQKBe677z4MGzZMlI3wFEEQxGI2Tl1dHfr374+QkBBxR8Abz3vS5L4z+de/yBXiLkDqzhAEBFDWEH/r7XYB3HaZTM7BYumxM2aQ2uSECS2Px9UQlJZSmmGxq15uO1i1igwZ3xHwjSPXyOFBS68haB/+/q1LkAAUDJ4/H1i+nK7zxvVA0zoCb9aQlxZxDQoLggC1Wg2bzYY333wTf/zxh9vjsrKycOedd0LbmOycnZ0NvV4PPz8/WK1WjBw5EqmpqXjppZfQp08fsRo4JCQES5cuxSuvvOJxKiiHt8UEaMK32+3o06cPAgMDRZeR0Whs0nj+bCAIpOlyzz1N73NnCAIDST+I6/nZ7QJiYkh64o47yBAIAu00ysupnSOH1xC0xKWXOgvCbd5M0tfnn+9oHN8eSkqAiy+m2oaYGFIe5bsUPvHz2EQbCs+9SCgtBf77X+fKc3fExdH34557HAaYG4/77ycXo8MQ9K5gsdc15ILdbgdjrFl/v1ardZtVw5u51zcjvXjkyBHk5ORgz549mDZtGvbt2wcAeOONN7BmzRqMHTsWADB27Fjxf45cLscY3tvQQxhjCAsLQ2FhIQASyhs6dCj8/f2d4gMmk6lNO43OoKyMsoFycqgY6N13ne9fsIAmTukKj5c21NdTuqXdTqs4frvZTBOpTEYdvDhGIz2Pq86PK48+6nydv0WlpcCiRRRAbA9cHkOlouwknu4KOCb+OXNo18IL6Ly0jcJC2mEOHOicYuzKhx/S7gxwNKrnO4LYWPr7/ntaULQz3Ndj8e4IXCgsLBSF29xhMBjcFodxQ1DHm7W6UN0YleIGYN++fUhKSsKAAQNwyy23dKprhruueGGY2WyGQqEQM458JTOs1Wo9643nd+8GPv+cfnDu7Kavb9Mf9MCBtGrn2O0CZDLgkUdoZ8FbPcbFOZ9XrQbuuotqCVpDGkweOJBWjq++Sr2E2wsPaLuT0L7pJqpziIkh14Z3R9A+PA0Wv/Ya1Qvwx0pdQ/v3kxLp7beTMGFvw7sjcEGr1Tr1E3Clvr7ebVomz9n3xBA0NDTgyJEjuKwlgf0OYLFYoNFooFKpIAgCdDod+vXrJwaH1Wq12MEMwFkPFPMA8bhxpCwq/UECtEMIDnZOB73gAvrjLFhwAuPGjUZ6Oq22588Hrr7a0buWe7oEwaE22RKXXUYupEY7jfPPd5ajaC98R6BUAj/8QEZl/34qZIuKor/MTHof7r/fawzaA3/PWqsjkBoKq5XchzxraP162qXecgsFnbthl8ouxbsjcKElQ2A2m1FfX9+uHQGvQC4uLsavv/4Ki8UiFox1BJvNJmoQcSwWC/z8/MSJv6GhwalOQBAEBAYGihlQZ7vxfFUVrZR5QZfrruCNN5ylGNyRkqLFiBFkBBoa6Mcr1ctv60uSyZzdR7yR+Zkz1CSnvfDN1i+/ODKEuNshO5s6jf30E9VNtJTe6qV52lpHwP+fO5fiS9JzbN5M3e16G15D4IJOp2u2kXt9fT0EQXAbWPVkR8CF5959910kJSV1uM+vtIEMdwcxxmC1WuHv7w+VSgW73Q5BEJrUCYSEhKCiogIRERFtksfuDKqrKb8/MpKCuNJOj4zR5Osatjh2jFwoK1fS9Z07Q7FjB+0kGhroB/zjj8Bvv1EaaVtj365ZQ8uW0Sr+qqtI5ri9hIfTJJOcLJU4psvMTOqQ1RjG8e4G2klbKouvvJLiUiEhtAPk1dw87PfZZw5Rut5Er3cNFRUVIU6iRKbT6cRMG8YYiouLceTIEUyaNAk1NTXNZtdwQ6DX60XtHinV1dWYOnWq2ALy5ZdfblWIrrq6Glartdn6hIqKCkyYMAFlZWUoKiqC3W6H0WiEr68vNBoNfHx8IJPJoFarm7h/UlJSxFTSs40g0G7gmmvoT0p5ORkG15xwtZoCt3wr/847A3DgAO0ITCZy/+zaRYFhaSzBU1wNAV89+vk5xMnaQ0MDnUuvb1pQJs0a8vFpu/HyQsTG0qTOiwebw2KhRAIu7nf55bS4+P13x2diNPZOg9yrdwQWiwWHDh0SXSs2mw0mkwk2mw0WiwWlpaXYuHEjysvLcebMGZSUlDTbZUy6i3DNHOIupbCwMLz++ut47733ECFNbWkGs9ks+vE52dnZYtaPXC5HQkICUlJSYDAY4Ofnh6ioKNTV1cHf3x+CIMDf39/J0HECAwMRHh7e7mrojvC//zkqi13hMkiuOwJp1hAA2GwC5HIyBNzl0pE8/JYMQUf0BHfvpsu//mq6I+ATjk7nrSHoCAoFCQjyn+ahQ2RUt21zflxWFsUAli4liROj0ZGZxj8LryHohVgsFmi1WjG3X+prN5vNqK2thUqlQlRUFI4fP47y8vJmDYFeIlTj6h6qqakBQMVncXFxbVqFS3cgdrsdcrkcDQ0NsFqtUKvVkMvlCAoKwpgxYzB58mQMHToUAMT4QEhICGK6aV5ibi7l7m/dSv7ytWsdbSNdDQGXF+aGwG6nCfXUKRIMs1g6NpnOnk1yDxzuZlCrO1ZxzL9SMhllBt17r+O18PF6DUHHMJupSphP/LxfFA/8c8LD6fvy2GOOnSdPUrj6arovPNyxO+hN9MKX7MBisUCn06Gurg7h4eGiIRAEAWazGdXV1VAqlVAqlaisrIRMJmt2BS3dEbgaAp4x1JJsdHNIq5gtFotYByCXy52CvDw11M/PD/Hx8aLbKSUl5awHg6XYbMCWLdRPgNujm2+mwO4FF1Dq53XXUQZHQYGjaMx1YlQq6UfrMAS0I5A2punIZCotQAMchsXHp2M7Am5QLriAqpulFc58vP/5D9UYeGkfNhtpUSmV9P7yhnyuK/v//IfciwB9ptIdQWAg/THWO3cEvd4Q2Gw2lJaWon///qIhYIyhoaEBtbW1Yn4/z7JpjuZ2BFarVcwYCm3NiemCIAhOriGz2Qx/f3/odDrI5XKEh4c3OUYul2PcuHHi5H8uYgBSamqoWApwZMX8/DP5c7ks9KlTZAQAKjZrTgj1ppscEhLcECxZQjsDXlDWXhoa6Bx8tT55Mp132jRHXMJTqqpoZfn9944x3Xgj7Szsdkc8YMIESqWNj3dOn/XSNlyDxfwnI2kgCIBahDaul2C1Ou8Ijh+n7+UTT1DBoqRVeK+gV7uGuCAbl31uaGiAIAhgjMFkMkGn04kra39//xZX9DqdTpx8uSFYt24dLr/8clH/v62GwG63Q6FQiHIQUgE5i8XSbCexoKCgc+L7P3iQflBSpJsjo5F+rFotGQI+/Oxsx2NuuIHkAtzx/vuODJ6XXjqMp54itciVK6lqVFq121YeecQ5QD17NvD887SSv/rqtp2r0e6LxgWgif/NNymmwVsqajTAgAEUrOT9E7y0Hbmc3l++c+MFYVLXEGO0c+CeXYuFYgX8sz16lHpbyGS9s1NcrzYEFosFCoUCJpMJRqMRWq0WcrkcCoUClZWVYIx5rMGj1+sRGxsLwBEsXrlyJfR6PVavXg1BENq8OhcEARqNRgwOW61W0RjZbLazXgjWGrNnU0cxKVJDcPQo7RAAkp9Wq+lHLDUEf/3lkAl2B99VDBigQ1KSI300MZEm1fbiGizWamns+flNfc2twTeH/v4Ol89TTzXNGiorox3NCy94DUFH4UqugCMuM2WK437+2XJXkNVKNQRc2pzv3L7/nrrG9TZ6tSEwGo2QyWQQBEEMGvMOXdyd4w7mpvJHp9MhJCQEvr6+qKurQ2lpqdhnoLi4GEFBQW3q/MVVQQMDA8V+yIwxhIaGis9/thVDW+LPPynQ+9dfzrdLDcHBgw43S2goreJGj26antmc9NE//uFwM/35ZxR27nQUlH31Fe0O2ourIXjoIeqX/NJLzmJ0nsCb4Pz8M7kiBg+m3Y9r1lBpKdU8HDjgDRZ3FB8fx+6LG4KbbnLcz43EhRfS93TkSKrwrqyk2/lPc9kyKmjsbTRrCARBWCkIwm/N/Z3NQXYVRqMRCoUCPj4+yMrKEquGuY6/O0wmE6688kr85TLj6fV6+Pv7IygoCHV1ddjY2I8wPT0dQNvdQlarFX5+fvD393eSkg4MDBQb4nQnQ3DoEF0WFjpXyLoaAouFiqt4HGDXLoe7hxeLNWcIGHOoj7711kB8842jjuCll4CPP27/+OXypumj7Q0Wc8NWWkpuopwcGqPrjkAa0/Aago5RWOhoZcoNgfQnzD9bf3/67hmNwKhRjipibx1B87wK4DUApwEYAXzQ+KcDcKSF43oMBoMBCoUCYWFhKC0tRVlZGXx8fODj4yM2qHelvLwcNTU1WLt2rdPtOp0OGo1GNATr16/HkCFDMHfuXABtNwQWiwW+vr5OaqGMMfj6+iIgIAB2u11MEe0O8MCc3e5I3wNIr+eXX4Dp08mNM3w4uYKmTnU85sYbaTUfH0/XmzMEcrljMrXbyZ8bFETn7Wiw2HVHwM/XEUOgUNBOxWql94Sfn2c6SSd/ryHoGEFBtCgAHIZAKtgbEECfyxVXUIZR42ZdDBZLZSp6oyFo1lfBGNsEAIIgPMcYmyy5a6UgCJu7fGRnAb4jAKgtpE6ng0KhEIXa3PXxlYrH8SpexpjTjuDo0aPQ6/V48MEHMWrUKKjVao8KyKqrq0WDwXcEPDjMawiUSiUCAwOdAtndAWk/nuPHHZr7MTEk6Nacvt78+TTpLl/ukAh2U/8GwHmy5llDH35I1/v169hkOn06nZ8xR+DRx6d9ncuuvJImmP79HatSmYzaZT7xhKOC2Lsj6DxefJG+A9ddR9+1t95yXpAIAn0mlZVUc8DXZTzMNmEC1RbMmtU7DYEnMYIIQRBEUWBBEPoBaH1W6wFIDQEAMetHLpeLBVuucENgsViwZ88eABALvLgh0Ov1UKvVmDFjBlQqFZYsWYJ58+a1OBaepcQzjrgh4JO92WxGQECAqBsUwPMcuwn5+RR4W7MGSEtz3H70KE3w3F30xRe0G+Cr5lOnSB4iM5Mm4y+/BFzaMYg47wgEJ834jhaUTZ9OfQf4JM3rEtpbRxAQ4NyG8sYbqVsad18AjvH+97/UatNL+/noI8dCon9/4LbbKGjPP7uaGlp08Ip2HsfhhkCpdPSx8BaUuechABmCIPD+P4kA7uqqAZ1NTCZTs5k3gwcPdns7NwQqlQrbt2/HpEmTxBoC7hoCgAsvvFA8dypPfm8BvV6PhIQEFBUVISgoSDQEXErabDaLu4XAwMCz3my+NRITKfB70UXOt3/6Kem8nzxJgTqLhYwGt7E8hfStt8hItKT9P3s2BfkAh8TEihXA6tUdLyjT6ymeER1Nq/d58yhzKD3d2bB5wh9/0Go0IsJhCO66iyqIzWbHajQykgKXQUGtt1n00jLSYPHRo7SwACg5ISqKJv633qIGNoDDEPBay+Jiytx67TWSCM/KOpujP/e0aAgEQZABCAIwEACfGY8zxpqvrHI+fhaANwHIAXzIGHvZzWOmAngDgA+ASsbYFNfHdAVWqxU2mw0yWdsSp2pqaqBQKDBhwgTs3LkTjDGxqtjf319MEZ0zZ45H55PWCCQnJ8Nms4m7ApVKBaVSKbqe+vfvD4CKxM62Ymhr8BXt1q2UFnnllXS9ro4muuhoiiMYjTRB8pU3/yF6ooLBJYMB4JNPdmPWrDF44w1K9zt9umNFWe++S13KtFoak7R2oK1Vvzt30utbtAj497/ptvp6YPFiWrny6mi5nN6XV14hd1hHVE57O9L00Y8+otoMgFxBUVFN00cTE6k5El+jVVSQeyk9nQx0bzMELc6CjDE7gPsZYw2MsYONf54aATmA/wGYDWAIgOsEQRji8phgAO8AmMMYGwqgjaU77cfSTt2AqqoqhIaGYuTIkaitrUVRUZG4I/D398fs2bPx3HPPYaAHVSkWiwU5OTlOUtHDhg0Tz+fj4yO6hsLCwsRzBgQEdFv9oP/9j7RcONwQKBSUpQE4q0RyV48nL8dmc6z6EhIMiIykAKHVSgqUbgqtPYa7A/iEUVxMPuMzZ6hpSVviBDodGRNBcFRPz55N45e6HaxWkj144gmHW8NL+5C68HiweOFCRz8Ifh83BKGhlF7Kv3f8c3n9dUpu6G14shxeJwjCI4Ig9BEEIZT/eXDcGAA5jLFcxpgZwDcAXEOG1wP4iTF2BgAYY+VtGn0HaK8h4AHdlEZBk2PHjok7Ao1Gg5CQEEycONGjcxmNRsTExIjxCD8/P4SHh0OtVsNisUCpVEImk2H48OGYNGlStwoOS/nzT9J3yc6mldaZM46YADcEAHDeeXQpLYjm/0sb1TfHHXdQ0ZjNBnz/fTwyMx2ZIgsXUk+C9uJqCObOpYnihx/It99C07om6PW0s3jwQVph/uMfNBFZrc6GQBCA556j/7vpR9tjcDUEsbG0I2us8RQ/1+Rk+mymTCENLG40+OeyZQvVf/Q2PDEEtwG4D8BmAHsb//Z4cFwcgALJ9cLG26QkAwgRBCFDEIS9giC0HFHtRCwWi8dVw1K4IUhMTIRarXYyBG0VdzOZTEhKSoIgCIiJiYEgCJDL5Rg0aBDsdrvo/hkyZIhTn+Fzid1OrhiJtBJOnqRMoaAgWoHxxuAAGQI+2fN0PqkL55JL6JL/YFuCB4stFupHsGGD41zPP9+0mK0tuBoCHnPgt7dl3cD1B48dI4XVrCw6r9Xq3BRdKprXzTx9PY7t2ylRAaDJXamkOg4eC+C7MaWSdmu//056Uvx+6fvvDRa7gTHWr53ndjfLupbkKgCMBnABAF8AOwRB2MkYO+F0IkG4C40B6qioKGS0s6GoTqcTj+Ur7ua6kTVHVVUVBgwYAKPRiAEDBuDIkSNObpr6+vpm4w68IpgbIB8fH5SWlkIul6O+vl4cm91uR3BwMDIzM9tlrDoT6XsGAL/8Eos330zGfffl4KqrqLVWRsYA+PjEIitrM8rKYgAMwh9/bEdEhBl33KEBY0BGhh6CoMKsWYm45poCZGTQEttikeH99/1gNBqRkWFzMwIH5eXJMBrDsXHjTgCTkZd3CiEhZoSE9EdNjRKFhbnIyDjT4jma49QpGveWLTsQEdGA2tp0+Psbcfp0DYBkbNq0DaGhrVsDnU6HsrIyAFEoK6vFffcZkZcXg7599SgsrIfNFoKMjJ3i4xWKSTCb5aioKEJGxsl2jd0TXD/H7kJXjKugYAiMxkDExKhxxx25uOEG+k6sWwcYjTJcfXU/FBf7AgjHvn1b4OtrQ1mZCgD1Oq2oKO5V7xfgofqoIAjDQH5+cS3HGPu8lcMKAfSRXI8HUOzmMZWMMT0AfWN9QhoAJ0PAGHsfwPsAkJ6ezqZKq5HaQEZGBvixZ86cwfbt29uk/2Oz2VBfX4/IyEj4+/sjNTUV33//vdjjWKPRoLy8HAOaEb0pKiqCWq0W9YJKS0sxc+ZM7N69G66vqbi4WNQuOpdI3zO73RGwjYsbgKlTB6CignzoM2cC06dPFUv2hwwZj6FDnQvHAODaawGgffGN77/n+fhU1pKc3B//+hcFZAMCgEGDkjB1alIrZ3FPWBgFbi+6aBwCA2nlGBvrjyFDKKdwzJgJYsFbS2RkZGDDhijMmgXU1AQjLCwYAKBSaXDvvRqcPg2nz1qlot1Hv35xmDq1mQKKTkD6OXYnOmtcb79Nu8+nn6bAv1ZLciRBQc7fiZoactVxd+VFF02CTEbf7csuo+9A376x8Pc/8bd+v1xp1TUkCMJCAG81/k0DsASAJykxuwEMFAShnyAISgDXAnCVpvgVwCRBEBSCIPgBOB/AsTaMv03YbDYxEGswGNqcMVRXVwe73S6mcaakpMBqteLrr79GTEwMFApFi+eUy+VilbDNZoNCoWi20U13MAKu5OY6ttLct7poEblCliyh6zNnkkuE28LPPydpic5AoaAtPg8wS3sRAB3zs6emUmohd2NJK4uBtheVcQ0kPraHHiI32AMPOD9OqaRmNa+91v6xe6E41Y8/0v8pKeSGDAtzaAkdO0a1BbmNSfB1dVRDwL9DMhm5Ge12r2uoOa4CrdL3M8ZuFQQhCsCHrR3EGLMKgnA/gD9B6aMfM8aOCoJwd+P9yxljxwRBWAPgEAA7KMW0y+QrLBYLioqKkJycjIqKijZr9XAhOr6i5wFjf39/vPzyy2L1b3NI3TwmkwmhoaHn3PXTFgYMoFaSKpVjgrz7bmDYMMq9BoDgYPoDaMK++WYK5LY1F98dF15IMQhuCORyYMcOagAPdMwQaLWUKZSYSK9v4UKKW6SkUF1AM22j3fLwwxR0nDOHDOfw4cA991AWEs9w4uTn07jbuCbx4oI0WLx2LX03wsMdkuCFhcAnnzjXqUhDekYjZW99/jklB/AWo70FTwyBkTFmFwTBKghCIIByAB7tvxljqwCscrltucv1pQCWejjeDlNWVobk5GRUVla2ObjLi8n4jiA8PBzPPvssBgwYgJiYGLGLWXNIVUuNRiMSEhLa8QrODevW0Vbb9S1LTXXkYgM0oX74IVXq8pfXWbVvl15KfzYb8N13OzBr1jhs3kwqkmvWUPCvvfz5J9UOHD5Mhu3mmx33eeISkvLzzzTOTz8lN4ROR5lUDz9MQfUjkqWOWk3GdO7ctqucenGgVDp2X4sXU5qodEfAd3R8A37VVcDttzuOt1pJeTQhgZra9zY8WYfsacz3/wCUMbQPQGZXDqorKS8vh9FoRENDQ5uLslwNAQBMmjRJDBbzRjKuDeel8EYzFoulXa0rzwXZ2eTyefNNuv7uuxQryM6mnGuTyfFYk8mxIubKo51lCIxGWuHJ5UBERAMCAhzpo35+HavOdc0aOnyYqn5LSmhir60l94K0d0Jz8DoCAHjySVrtp6c3zRoCyLX24YcO9VYv7cM1fVSlAu6/n1xygOM+lYrSdocMIV0hDv/8H3mEYl69jVYNAWPsXsZYbeNKfgaAmxljt7Z2XHdFp9OhQqpG1Qa4IWguwMxTPnm1sBTeXyA0NBTGxtzKc91G0lN4uT5fsR4/ToHbX36hgimp/5xP+rW1jgrazjIEL75IVcn19cBnn/XF/v0OQ/DQQw5FyfbAJ2j+WsaNA159lVwEc+eSb3nIEOot0Bo6HfDOO1Q3MWkSvW8WS9OCMsAhne2tI+gY/v7O6qMqFQV/r7mGbuOfK48zXXghSaBzpGvCnY6krl6DJ8HizwVBuFMQhMGMsTzGWI9euxgMBhQXF7v1zRuNRixduhQ1vI2WC5WVldBoNC3GFuRyuVtDwLWDQkNDUVtbi9DQ0G5TG9Aahw/TRMWLpWNiaDLOyqIKTam7SKmk1XltrWNH0ExHzTbD1UFraoBPP+2HffscP/59+6ieoSPnBhzxB6n6KL8OtB4rsNkcO6Rjx2hSaa6OQIrXEHSMd9+l9xtwGIKyMtIdAmhXFhzs2BE8/jjwzDOO46WfS2+s6fDENfQpKN/vLUEQTgmC8KMgCA927bC6DrvdjsrKSrda/gcOHMCqVatEVVFXjh492mxqKKclQ6BWqxEaGoq6ujr0bU50vxty+DAFTfkPhJdN7NrlvndASAgZgtGjKWOoOTXRtsJ/rNwXLJc7G5mOTKZS1xBjzo1p+O2pqcD48S2fx2yWIy6OJp2GBsoI2rCBjne3I+iMsXtxhhuCl14ieWmA/P41NdQx7sEHKclAuoCRrgt7Y9aQJ66hvwC8AOAZULZQOoB7unhcXYafnx+qq6vdpm2eaeyuUskjTBJqa2tx8uRJseNYc7RkCHx9feHn54fg4GBEeaKp0E04fNg5IMzbNGRnuzcEwcH0o/Pzo4yZzlLMdmcIBg8Gtm2j6x2ZTIcMITnspCSasBlruiM4fJgyUlrC19eGwkJgwQLnCmuLhYzCggXOj+e5BT1kc9ht+eor4JZb6P9Vqyj24ufnXhpkxQq6dBUe5j/73rgjaNX2CYKwAYAGwA4AWwCcdzY1gTobX1/fZpu65OXlAYDbGMK+xg7mo0ePbvbcgiBApVJBq9U2uc9qtUKj0UCj0SA6OrrbyUi3xMqVzpNsfDxNmLm57g3B+vX0I9u3j1Zet9/eMWVQDp+UpYZAer0jhiAujqSiAcfk7a6OwFPNIb7h1GqpYcrDD7vPRhk4kP68yqMd49Ah4OuvKVNr6FC6zc+PDLDFAmRkAO+/T8aef6auGXD8M+6NhsAT19AhAGYAwwAMBzBMEIQeu36Ry+Wie6eiosKpST3fEVRUVECn02H+/PlYt24dAGDv3r3w9/dHcnJyi+f38/NrcUfg6+uLSZMmtbmY7VwycqTjxwXQ/zk55Pt2XeECtGMICKDV8/33O/cw7giTJgEvvODYxstktPPgDe078gPW66lpSW0tGZxPPwUuvph2Qps3k5trwYLWdzcFBb64+GKKoVx6KbkpJk+m9yEnh+owpGzY0DGNJC8ETx9ljCb7XbscqaJGI733P/xARoEvKFx3BM8/T587XxD0JjzRGloAAIIg+AO4FcAnAKIBdJ+Gue3kpcZ2Uf/973/BGEN+Y7/FiooKHD9+HIcPH8bhw4eRk5ODzMxMjBw5ssWCMcYYNBqNW0Mg7THcXVVE3XH4cCDy8kiJU/rSBYHiBu747TcK0pWU0KqrM3YDAMUaxo4l183KlVswc+YkUfxuyRLK9GkvWVlUjfr772QApHUEkybRJXc18HaW7qipUWLVKjIaL7xAxWP19bRivf12ym3n4mgA9VF4+mnqhcBlur20Hb4IsNmowPDxx4E+jQI3BoMj2K9Q0GOTkqh+Q8o33ziKIHsbnmQN3S8IwrcADgC4HMDHoB4DPZ7i4mIcPXoUFosFVVVV0Ov1kMvlKC8vR0EBCaeef/75+O6771BZWYmxHkQ9eQ9jTmFhIerq6iAIQrdrJuMJ69ZF4+GHm1a+Dh9OGUM8M0jK2rXA0qXAiRMUnOus4mmdzrGi9ve3Qal0GBmbrWPVudJgcUMDNdgpL6cdwhdf0PO+8IJD/bQ5jEZ54/jo+pdfUkA7LY0Mgus6YvFimoDKe6yztXvA11ZGI31GKhXtFD/7zNE2FCAjEBRE97nmfVRVUfZRb6sqBjxzDfkC+C+AwYyxCxhjixsDyD0axhiqq6thNpuRm5sr7gaGDh2Kmpoa5Obmws/PDy+99BJWrlyJDz/8EBe59mF0c05pNpLFYoFKpYLBYIAgCD1qJ8A5ccIfaWlNJ/PDh8kt425SDA4mA3H8OBmCzuLLL8nfvn8/sHx5ErKyHL74J5+kdMH2IjUEpaW0C/j9d5KdmDfPOedcKsHtislEM/2GDeQiS0hwrPRNpqYZKY2eR2/WUAcJCqLdF/9sVCr67s2bRy4gqf9/61Zy13HdIQ5fv7ne3hvwJGtoKaiN5E0AIAhCRGMD+x5NfX292JwmKytLNAQ8K+jgwYPo06cPBEGAv78/+vfv36JbCKBgsbTGoLa2VjwHY6zH7AiqqqjnQG0tcPJkAFrqs+OuODokhMS78vOpEUhnwSfRggLg228TkJvrvAtoo6K4E9KCMunqkX9kPCD9j380DTJK4TsCtZoM04oVjvoGd4aATz5eQ9Ax7r4bKCpyvI8qFS1Gtm1zCMz16UPvf0kJuX94tpkrPeRn2ql4kjW0EJQyOggUH/AB8CWACV07tK6FVwkDZAj8/Pzg7++PQY1L2IKCAlx44YUen48Lzkkne4vFgkGDBqGkpAQGg6HHGALe8vGFFwC7XXBrCJYsoYwgd24fLjp36FDbdXpawl36qJSOvL3SgjJpFhK/nWcLTZzY8vMolTYkJzvew0WLHN3XTCZvQVlXw1VxVSraOU6bBmzcSMH6+++n+7jsRHMGvYf8TDsVT1xDV4Bkp/UAwBgrBtBJmeHnDp4tFBISggMHDmDLli0YOHAgIiIixMfEt2EW4xISisaZw2g0Ijg4GMHBwYiNje1RriE+kb/xBiCTMbdB2EcfBX76qeXjGXP0jO0M+KTMf+x8Un33XbrsyNsbE0Or9/Hj3e8I+G7jiy+o0XlzTJ9egexsZ4VRnp2yeDHVEkjh9QPeOoKOsWEDBfkZowyh665zZA25pvz+8QddumYN7dhBl96CMveYGUU/GQAIgqBp5fE9Am4IJk6ciMrKShgMBtx///1OhqAPTztwwW63i30FpLcplUonQ8DbT8bGxsLPz6/HpIzyEofRo4EfftjeZomIOXMop/vbb8lF1Fk0tyPojDqCgADg+utJhpobAumOQCobkZPT+vmkmVIXXkhpibff3rRRz1VXUdxDWrDnpe2UlFAhmcEA9O9PixGpIfjgg6Z1HK47AsboGDeiA397PJmZvhME4T0AwYIg3AlgPUiJtEfDXUMzZsyATCbDI488gqSkJPj7+4tVx83JRNfV1aGoqMjpNqkhYIyhoaEB4Y3+gZCQEES2RdD+HMMnvS1bgICANnZkAa2iV6+mya8zbd/o0SQVzMVfuSF48EHH87YXs5mKjgoLKbXwhx/o+cLCyMVw993Apk302JaKyr7/Ph4XX8y7ndFtQ4aQT/r4ceDUKefHL1nSO4OTnQ3/7AsLgVdeobiM1BAcO9a0XsPVEPz6K6XyTp/e9ePtbrT4MxVIme1bAD8A+BEUJ/gPY+ytszC2LqWqqgp+fn5ITU3FH3/84RQP4LuCuDj3rQOtViv8/f1FFVGgqWuIB5kBalwzhndu7wE0NNCPSK8Hjh1ruxewpoYafLRT5LVZBg2iDl9z5wIbNmSI/QdSUmhl3REpi9pa8if/+isZmiuvJHeRjw8wYgQZhOZcDVLy8/2wbx9VC3/2Gd2m01FgcsYM4OWXnR+/Zg0VnrlRNfHSBvhuMDeXGswcP+78eUkLycaMoVU/F1HkbNxIxYO9kRa9YYwxJgjCL4yx0QDWnaUxnRWqqqrEfgCuaqIREREwGAzNqoPabDZER0ejrKxMfIzNZoNKpYJMJoNCoYDFYkGAZGZS9CDH4w03UHGW1QrExTWVy2gNngnTUr59e9BqKZ9/wADaafDdBm8L2RGk6aMVFVRlPHYsrRo/+IACvvPn02NaMgRGo1z0PYeEkAE4cABiwN01WLx0Ka1UO/u96m3wHQFXd1GpyKD/8AOl7x486HjMwIEkZ+768z5yhD7b3rhD82R22ikIwnmMsb9VmUV1dbVTgxkpN9xwA+rcVUpJiI+PR0lJCRhjEARB7EUAUOWwRqPpUZO/lLffdvyfkdF2fYjgYFq9S2V+O4OMDIo/fPIJ8NNPyejbl/zrBw7QX0eQZg3t3k2Bx507SV7j3ntpx8G9ga0ZgoAASnEdMgT43/8cndqkz8Ph3cp6SPio2xIURKnKvKhfpaJdwpVX0nXpjuDBB4E9e8joSz8P/rnyvty9CU++ftMA7GiUoD4kCMJhQRB6dE8CgBRGm+sQNmLECEyZMqXF40NCQhAeHi66h2w2m5gVpFKpxPhAT4OxjmsDyWS0NZf2h+0M+I/21Clg5crYTnWnSHcE7oLFvFBp6VJyTTUHNwQKBbmE3n3XYUCkz8Ph8hWdJcPRW5k0idRwuewJD/iuX0/yITExjvt+/52Me3PG15s+6p7ZAPoDmA7gUgCXNF72WHhVcXM7gtbgqaAREREwNUZWpVpCvr6+PSo4LEWrpR/IG2+c65E0pbU6go4gNQT8/D4+DhcUNwQjR7bccS0uzoi0NMfEvnOncxtK1zF//jm5LXqQGG23RlpHANBO7v33geeec7SgfPZZunQ1BPw74DUEbmCM5bv7OxuD6yoMBgNMJlO7ewbzKuHQ0FAxjZRnDQFAZGRkj2lD6Qr/IXVHr1ZzdQT33EM+347g40Ny2//8p/OOgN/H6wjee4+yqZrjkUdO4K23nFMQecxg/vymgma8Z4OXjmGzUbC/sJAkQoYNo9ub60ngDt42tDt+97uaXvWSbTYbGGNiDUFHmscrlUr4+fmJLS8ZY6IERUpzspw9AJ462h1dFS3VEXR0FScIwCWX0P9bt9IlP6dCQc81dCj1au7Xz+HSaQ6pIRgxgoKWkya13urSS/uQyynAn5bmqOQGHIbgnnvoMW+/Tcqkf/7Z9BxKJX0+3jqCvzHff/89Lr/8cpSXl4s1BO1xDVmtViiVSshkMmg0GlFpVBCEHhsclsJX293REAwaRKu2IUMAlcomrtwOHiRxuI6yahX5k2fOpImCd2I7dIjSPI8coTTSllaY99wzCq+9RpMO11mKi6Og5b59TesIvHQeERH0GT7zDCm9Ag5DsHMnaV8BlMK7f3/T40+dosVAM3Wkf2t6jSGIjo6G1WpFcXExihtnjWj+S28DvAk9QGmnMplM7D/QmihdT4DvCLrjqig6Grj1VtKMWbNmCwYPpts3bOgcQ3DVVVQEFxtLxoCnFyYlOYrYeH2FOxgDTpwIQE0NXV+2jC4bGsiQzJ5NCqpeuoaICCoke/55x/eYG4Ly8tZ3Y0eOtOz2+zvT7BJWEAQtGmUlXO8ClRi0UXjg3MLF5IqLi2EymeDj49NuQxDYqLkgCAJCQkLQ0LiM/jsYgpAQEuXqTPnozkKvp9W569gCA9FmGQx3KBQULM7OJpntyy4j99A779AkceAAKVk2tyMwGkmkj5ePjB9Pu4CGBkfTnL/BprHbIo0T8YXMsmX0GY4Z07ohWLWKPt+WZMb/rjT7tWSM9XhhOSkRERHw9/dHSUkJdDod4uLi2jVxWywWp8b3oaGhOH36NBhjTQrTeiJxccDrr5/rUbjn5EmaXB96CDh8OAVDh3Y8SCyFG4LffgMee4wCxD4+JFnQ2MUUUVHNGwIeUOaGYPBgkq1++GHHY/4Ga4Vuy6hRlBoKOAzBmDFU6W61tm4IeOlQZ7VW7Ul47BoSBCFSEIQE/teVg+oKBEFAXFwciouLUVBQ0KyOUGtIXUMAGYKKigoMGzYMGlc5wx6IxUIros4Ui+ss+CSanQ1s2BAFicJHp8ANgTR9lN/O2bDBIR3hCq9q5Ro2xcXAhx86rzC9O4KuY/FiYOFC+p9nfGVmUoB/ypSmkhLN0Rs/I09aVc4RBOEkgNMANgHIA7C6i8fVJcTHx6OgoABFRUXtNgQ2m81pwvf390dSUhIGc4d1D2fdOprIumO7Pm4IXNNHO/P8ro1ppJcAVQk3lxkslwPnnVeNxETn27mB4I/x0nU0NDjqPwBy6734IlWl86yw5nD3efcWPNkRPAdgLIATjLF+AC4A0Exvn+5NfHw86urqYLfbm5WY9gRpg5nw8HBMnTq1xzSdaY3unD7KV2pdUVAGAD/+CDzyCJ1foXA03ZF+tL//DrzVjORiYiKwZMkhUQyPwzeQV18NXHFF547Zi4NVq4DvvnOWG2lLHcHjj9Nn3huNtSeGwMIYqwIgEwRBxhjbCGBE1w6ra5A2mmnvjsC1wYwgCE59ins6rpWZ3Qn+A+Ur9s7W5xk/nlI+LRbnyZ8boLFjKX7As4E8JTaW0lH/+1802S146Vxyc513YH5+JCKYkgJImhK6JTi4c1ur9iQ8+SnVCoLgD2AzgBWCILwJoO0i9d0AqSFo746AMdZjOo21h+68I4iOJn/vzJlASIi50325v/9OxWQPPECuBM769RRI3LGDqoSbW2H+9htwzTVjxR7F/Ro7ewcGkvrojh3A6dOdO2YvDri8F29JCTh2Y8ePt55Z9q9/0eN6I54YgssAGAEsALAGwCl4qDUkCMIsQRCyBUHIEQThiRYed54gCDZBEK7y5LzthbeMDAsLa1dg1263QxCEZuWp/w5054IyjYZy/Z99Fvjpp+1op1RUszz6KLl9EhIo24QTFuaYRNy5Gh57jGoEKiuB8nK1GKi86y66lMlIfO6f/yTNey9dA88g27PHcRs3BL6+vTMI7CmeaA3pGWM2xpiVMfYZY2xZo6uoRQRBkAP4H0i0bgiA6wRBGNLM414B4Kbou3NRKpWIjIxscTdQV1eHGl4R5EJ9fT369Onzt4kHuGPUKODf/+6cvPzOpqEBWLvWkcrZ2fCsoc2bydfM+fhjID6e2ky62xEsXUoFYzx9lGcN3Xor1RHIZBR74M/hpWtwl0o8bx4VBLah/XivpNWvpSAIc0ETdSSomMzTgrIxAHIYY7mN5/kGtLvIcnncA6DuZ+e1bejt44477kCstLO4CwaDQdQPcsVoNCLxb+7kHTPGeTXcnaivp/aP06YBDQ3D8NdfnRvL4Ibgo4+oLeU//0m3f/WVQ0r6ggsomCzVsp88mSZ77pvmdQT33Ueuhn37HM/RGwORZwt3m/zYWK8R8ARP1idLAFzKGDvWxnPHASiQXC8EcL70AYIgxAG4AiRx3awhEAThLgB3AUBUVBQypA7cNmA2mzF69Gj4+PhAx5dvEqRFYTqdzklHiN+XnZ2NEydOtOv5m0On07X7NXU2Op0cZrMMoaGWxuvdZ2z19QoAE3HggBk1NeHYtGkzlMrOK3gwGkehrMwCnc4KqzUAGRmZjc87HEAoGDNizJjdWL0a2LLFLmYVlZSMRliYGUeP6qFQxGP7dup3+Ndf41FTo8TWrRkApgIAsrOPIiOjk3t4ekB3+hyldPa4zjtvOLRaBTIyyPqWlKixd286Zs8uQUaG50JPveX94nhiCMraYQQA2jm44lqz9waAxxljtuZW4QDAGHsfwPsAkJ6ezqZOndqO4QBr1qyBVqttViK6vr4eAQEBqKysREhICMrLy2GxWBAXF4eamhpER0djHNcK6EQyMjLQ3tfU2Tz1FPDqq44Uze40Ni4kBpATftq0yZ2a8x0S4pCrCAqC+Lq5mmVgoC8uumhyk+NOnqS/sWPDMG1aqXgc9zBOnz5VfGxa2lCci7ezO32OUjp7XMHBFN/i5/zlFyroe+qpPhg50vMEkd7yfnE8CRbvEQThW0EQrhMEYS7/8+C4QgDSdz4egKs0WDqAbwRByANwFYB3BEG43INzdxpckhogt9CAAQMgk8lgt9tht9uhVqthMplgMBgw0NPSxB6MydQ9U0eBrm1MA1ALzP/9r6msNXcBqVTk5lmwgFISOTfdRJcXXgg89ZQj7SQtzfn8U6bgnBiB3sTateTW40gb2HtpHk8MQSAAA4CZoGwh3qWsNXYDGCgIQj9BEJQArgXwm/QBjLF+jLFExlgigB8A3MsY+8Xz4XcMi8WCyspKsbkMYwwRERHQaDSwNCarJyQkoKioCJGRkR3qX9BTMJm6Z8YQ0LQxTWfXEQweDAwYQHUE0gxhbhQmTiSp4jfeAMrKHPc//zxdlpQ4n2/bNsdtO3ZQN7Je8BU650idC1w/6P33z81YegqtuoYYY7e258SMMasgCPeDsoHkAD5mjB0VBOHuxvuXt+e8nYnZbEZISAh0Oh38/PwQEBAAjUaDwMBA1NbWgjGG5ORk5OfnY9iwYc0Gkf9ONDR03x2Bjw9l5/zxB/DzzwYAfq0e0xZWraLXv3y5o2gNAL74guSjlUp6bsCxwmQMoubRnXeSjzqTQgvQaBwBzKQkOnbmTBL283J24JlEQ5rkK3qR0pIM9WOMsSWCILwFN3LUjLH5rZ2cMbYKwCqX29waAMbYLa2OtpOxWCwICAhAXV0dbDabqBcUGBiIkpISKJVKBAUFYdq0aT229WRb6c47ApmMsoYuugiYOzcTPADbWSxbBtTWUhMTKe7aTnIhOZMJkMpMNSfW99xz1B3rzz+9huBsMnUq9RgYP/5cj6R709Lmmqd57gGw181fj8dsNiMyMhKMMdhsNrE/QUBAAHQ6nTj5h4WFQdbZfohO4MgRYNYs5+boHeX664Enmi39O/f8/DO97q6Ap49+/TXw66+O27/9ltwNt99OwUiADAbgSBnlzef9/Gxuz/32247n8HJ2mTix892Ifzda+lpeA+B3AMGMsTfP0njOKna7HZGRkcjLy4PVahUnfl9fXzQ0NLSrleXZ5NAhWmHOmtV5DdAvvrhzztNVXHMNVYkOGZKKHTs699zcELz2GmUKXXYZ3c6rgWtqSMZAEBwZTDwL+c03gUWLALXavSGQPocXL92NluzkaEEQ+gK4TRCEEEEQQqV/Z2uAXY2vry9iYmIQHR0t9hxWq9Xw8fFBMF/+dVO4LsrmzZ13zvx8R/FUd0Qup0n48OGgTj83NwTNBYtVKnLrWCzAzTfTbdwQaDTUf6C5HYF0/F68dDdaWp8sB2kLJYFcQdJIKWu8vcejVquRmJgIu8S5q1arERAQ4NSApjtyrLG6g2dGdAY33kgT319/dd45OxO+opbJOr+NFDcEdnvz6aOuMsXcELzyCqWdpqbWgWopm38OL166Gy21qlwGYJkgCO8yxu45i2M6a/BqYX8uDtOISqVCYGBgtzMEej1NRnwy4TuCysrOe46GBodEQneET8JdYQhef50m8wkTnN8DbhT4LuHpp0m64L77gL59KchcWEhiZ6NHu9epio+nc6amdvqwvXjpMJ6Izv0tjYDNZoNKpRLdQVJkMhlSUlK6nSG48UYShQMobZFrqFR0omJBdy4oA6Q7gs4/d0wMKY+WlzsLmHFDcF6jCMqqVZTGCpCr6IEHgPR0ul5Z6V6ifMMGKnbqZl8pL14AtKFn8d8Ns9mMgBaWvomJid0uU6imxpGdIgjA6tXUVamysvMabjc0dN/0UYAm4VtvBfr3b6oV1VHWriW56IICkqTmvPACvb933knXw8Mdu7CKCspi4n0cvvqqb7Pn/+knoKpV3V4vXs4+3WumO4uYzWYEdket5RbYtIkap9gk8chHHqEAb2fR3XcEY8aQLPTSpZ2YM9vIjz9Svn9UVMsVwOHhjgn9++/J3TNsGF2/+GJXFRXiySeBBx/s3N2bFy+dRa82BC3tCLozOh01UBk2jFwNMTHOZfUd4aWXgFtu6ZxzdQU//kgFQl2BQgGUlgILFzr3PPjjD3p/X3yRroeFOXYEPFicnEy7htGja92e+6ef6NKbNeSlO9KrDIFMJoOtcTlttVp7rCGorweysmjSqqgAFi/uvBaI11/fvYXRHn2U9P//85+hnX5uHn949lnnAPz+/XTJZSViYsgA22xkCASBahva8hxevHQnep0h4AiCIPYe6AkwBgxtnPu0WmrEHRZG4meLFgFHj3bO8+zeTfnw3RU+kebltb3VqKfnBpyDxdwVx78+Tz9NWUJyORkCjcbz4LV3R+ClO9KrDIEgCKJwXE8zBIIALFlC/2u1VDsQHOyYsDrD98wY+eCXn3M5wObpyvTR5gwBLzFxN9nrdG1Lt/XuCLx0R3qVIZDJZGLXMbvd3qOa0Dc0kPvhvvsoWFlb2/mGgOv8d2f72JWG4Omngeuuo4ld+h64GoKjR4FLLwUOHqQMpjc9EGCZOhUIDQUiIzt92F68dJhetT7huwGLxdJsDUF35dAhYPp04LffgP79SUgrNJTcEipV5xSVcZ3/7mwI+EfWFYrggYE02bs2QeeSU8nJdNnQAPz+O3DHHQ49otb45hvacfWgr5yXXkSv+1oGBARAr9f3uNRR3vZQpaIUz1dfddwXEdHxHcHjjzty4btz+uh331FwvKZGC8C/1ce3hc2bgX79qEuZlAUL6I8THk6XlZXUsUytbl3vPi8PWL8e+Ne/ureh9dI76VWuIYB6DWi12h5nCLjs8UUXAR9+6HzfkSPARx917PxLlpBUAtC9J6rkZGDFCuCxx7I7/dzbtlG3MdfX71qsx2sMKiuB//s/quVojUceAf79b9Iy8uKlu9HrDAHvNdDdlUVdqZFI2FRWkhvjrbfoelBQxyQXuA98wgTS4p82rf3n6mp+/JF6EnQF3G3z7bfOty9bRq4o3u7Qz4+MRVUVBYv9PdiYbN1Kl96sIS/dkV5nCAIDA2Gz2aDRdH76YVciNQQFBZQ5xCeV774Dnnqq/efmVbLXXANcey21VeyuvPEGMHcu8NxzKZ1+bv5+unYo45XbvAmNIAAjR1J8xlNDwPHGCLx0R3qdIfD19YVGo+lRqaMABYqXLgVCQsgQAI5uWdu2NfVrt4WKCooL/PknZSVVV3d4uF0Gn0grKjo/kMHbT7r2I+IGQtqjYPt2qkDW69tmCLw7Ai/dkV63PuGy0z0pdRSg/P4xY8gdxA0BF6CLjaVqY9fVqd1OjdVb2/wMGUKPk8vJH3755cCMGV3yMjqMI32088/NDaCrIfj3vylT6I47nG9nrO07gm6mY+jFC4BeuCPghqCn7QhOnSIXxcMPOyQg+I6gTx+6LCx0Puapp2iSMhpbP78gkKEBundAsysb0/BU0KEu6hVBQRQnkK4d1q0jyeply0gevDXuuqvzxunFS2fTK3cEKSkp3U5iujXuvpt81Dt3AgcO0AqV9yPgl4WFwODBjmN4cLOmpmUtnBUrSC9/wQKKEfDzdUf4jkAu73xDYDLR++RJ0VdiIr3fBkPrqaMA8N//UhczL166Iz1rNuwEBEFAYmLiuR5Gm6mtpfhAbS1NVB9/TN2xAJq4VSrngDLgcEO01spy82YqkLrmGnIxdecuWh99RNW/KSn1nX7uWbNoYh8xovXHDhwIDBpEdQElJa0/XqNx7OC8eOlu9DpD0FOpqSFDcOedlN4pzW3v14/cP1df7XzM77+TG6m1AGVxMXXaArp3m0oAiI4GvvoKuO22vC57Dk+rlseNo8tvvumyoXjxclboda6hngo3BEYjcOIErTD1epq0mpu4xo4FNm5s/dzFxRRw7gn88gvJb0tdYOeK558HcnM9l5nw4qW74t0R9AAYc7iG+Irdz8/ZADz3HGW3SFm8uKmS6Pr1lCYqpScZgm+/Be65B1i6dNC5Hgri4qhrXHeuu/DixRO8hqAHYLcDn38OXHklVRQDjtRRzp49JEgnZdkymjQ//piuHz9OaaHSfrx2O52rp0xm3M2l03k3s168dBbeX1MPQC4HbriB/l+3ji6lxU0ABYxdWzhyWemyMrr8/Xe6lLZhlMnIQPQUulKG2ouX3op3R9CNMBgo958LzHG0WsrsqakBLryQbnMteoqPp/t5dSxjjvoBfr5du+jSYmkqpNZT6Mo6Ai9eeiveHUE3YsMGah5/8CA1TOdkZQFTpgCrVgGzZ1MjFNd6OGlR2aBBVPHKWyxyQ5CZSZcGAxmXwEB6zpdfJkVTno7anenKOoK/GxaLBYWFhTCZTAgKCsKxY8fO9ZCa4B1X2/BkXGq1GvHx8fDx8fH4vF1qCARBmAXgTQByAB8yxl52uf8GAI83XtUBuIcxdrArx9Sd4Y1hVq2ifH4eD+CrfD8/msSnT6eCJimJidSwRqej61ot5a3X1tJfaSm5hEaPBvbupdz3wEBqeLN+fdtkEs4lr71GO52wsHoA0ed6ON2awsJCBAQEIDExETqdDgHdMDdYq9V6x9UGWhsXYwxVVVUoLCxEv379PD5vl7mGBEGQA/gfgNkAhgC4ThAE1xrM0wCmMMaGA3gOwPtdNZ6ewOzZwJdf0v/vvuu43WCgS42G3DupqQ43D2fiRCAnhyZ6gLKAamooMDxmDPUsAByN7vl35PRpykRydTV1VwICgC++AC6/vPhcD6XbYzKZEBYWJnbm8/L3RxAEhIWFwcS7THlIV+4IxgDIYYzlAoAgCN8AuAxAFn8AY2y75PE7AXRjcYOuR6OhoLBeTzUADz5IOjbSHQHXAXriCWD37qbnqKhwbrXIG94DZBg0GkC6Y8zLo91ET5krVq0iFxfXW/LSMl4j0Ptoz2felYYgDkCB5HohgPNbePztAFa7u0MQhLsA3AUAUVFRyMjIaNeAdDpdu4/tSvi4DhwIRk6OBlddVYSsLBWWLRsHlYqn9AzGoUM7UVWlBDAKOl09MjL2OZ1n//5gPPFEKl555TAUCjt++CEe9957CpGRDeJjbDbg11/jMHCgDqmpdTh6NB3R0SZkZBxpcWzdhc8/T8K33yZg//4EABnnejhN6E7vV1BQELSNTRRsNpv4f3fCO6624em4TCZT276HjLEu+QNwNSguwK/fBOCtZh47DcAxAGGtnXf06NGsvWzcuLHdx3YlfFwPPshYYCDdVlnJGMDYsmWM5eUx9uOPjBkMjGm1jCUmMpaR0fQ8ej1j/foxlpLC2Mcf0/FXXcVYfDxjL73E2OuvM2a3M6ZWM/bII3TMnDmMPfdc62PrLjz+OL2uyy4rPNdDcUt3er+ysrLE/+vr68/hSJqnK8d17NgxlpaWxkaMGMFycnLYuHHj2j2um2++mX3//fedPcQ24zquvn37soqKiiaPk372HAB7WDPzalfuCAoB9JFcjwfQxLErCMJwAB8CmM0Yq+rC8XR76uochWK8h4BOR9k80oye06fdH+/nRwqi8+c7YgLh4VRH8OWXwIABwEMPATExDqG0X3/tkpfSZXjTR714yi+//ILLLrsMixcvBgBs3769lSM6htVqhaILW9BZu1AfvisNwW4AAwVB6AegCMC1AK6XPkAQhAQAPwG4iTF2ogvH0iOQGgKVilIldTpKHy0ooMb1rZGWRpdbtpDfv08fqhs4doyyjQBnQ9DT8KaPto/HH38cWVlZrT+wDYwYMQJvvPFGs/fn5eVh1qxZmDhxInbu3Im0tDTceuutWLhwIcrLy7FixQqkpKRAr9fjgQcewOHDh2G1WrFo0SJcdtllyMvLw0033QR9Y5Ds7bffxvjx45GRkYFFixYhPDwcR44cwejRo/Hll186+cZXrVqFN954A3K5HJs3b8bGjRvh7+8vuu6aO/7ZZ5/FypUrodfrMXHiRLz33nst+tynTp2K8ePHY9u2bZgzZw6mTp2Khx9+GDqdDuHh4fj0008hl8sxe/Zs7N27FwcPHsSIESOQn5+PhIQE9O/fH4cPH8aGDRvw/PPPw2w2IywsDCtWrEBUVBQWLVqE4uJi5OXlITw8HC+++CKuvPJKVFRUYMyYMdyj0mG6LGuIMWYFcD+AP0Fun+8YY0cFQbhbEIS7Gx/2HwBhAN4RBOGAIAh7umo8PYHaWochEARK77RYgE8+Aa64wrNz8KYqu3dT+ijPBrLbHbsKbgh+/ZVqDnJzO/NVdC3eVo89i5ycHDz44IM4dOgQjh8/jq+++gpbt27Fq6++ihdffBEA8MILL2D69OnYvXs3Nm7ciEcffRR6vR6RkZFYt24d9u3bh2+//Rbz588Xz7t//3688cYbyMrKQm5uLrZt2+b0vP/4xz9w9913Y8GCBdjoRnmxuePvv/9+7N69G7t27YLRaMTvvBy/BWpra7Fp0ybMnz8fDzzwAH744Qfs3bsXt912G55++mlERkbCZDKhvr4eW7ZsQXp6OrZs2YL8/HxERkbCz89PNJb79+/HtddeiyWSLI+9e/fi119/xVdffYWXX34ZEydOxP79+zFnzhyckcoEdIAurSNgjK0CsMrltuWS/+8AcIfrcb2VujqSWeZUVZFBuPfe1ttNcsLCgK+/Bn78kYrLpBr4CQl0GRNDhWQnT5KSaU9JHQWo8rqwEIiL6/x+BH9nXnnllXOSF9+vXz+kNja4GDp0KC644AIIgoDU1FTk5eUBANauXYvffvsNr776KgAKdJ45cwaxsbG4//77ceDAAcjlcpw44XAajBkzBvGNHZRGjBiBvLw8TJw40eNxNXf8xo0bsWTJEuh0OtTW1mLo0KG49NJLWzzXNddcAwDIzs7GkSNHMKOxz6vNZkNMTAwAiLuGzZs346mnnsKaNWvAGMOkSZMAUM3HNddcg5KSEpjNZqcagDlz5oitdbdv345ffvkFAHDxxRcjJCTE49fcEt7K4m7EX3/RDoDDd6QGg+eGAKAuY9deS/8fPAikpJBriO8Inn0WeOEF4MknyVD0pIYpCgV1XsvIqDjXQ/HiASqVSvxfJpOJ12UymejzZozhxx9/xKBBzoqyixYtQlRUFA4ePAi73e7UXlZ6Xrlc3mb/ubvjTSYT7r33XuzZswfBwcF47bXXPMrH1zT+OBljGDp0KHbs2NHkMZMmTRJ3AZdddhleeeUVCIKASy65BADwwAMP4OGHH8acOXNE15Xr+TldkRLs1RrqJOx24JFHPNP/b46gIArucp57juoA9HoKBHvKmTPAZ59RqmhaGsUYGAPOO4/uDwkht9OXXzoa0vQU/voLuO8+oKHB+9X9u3DRRRfhrbfeEv3d+/fvBwDU1dUhJiYGMpkMX3zxBWxcM6WL4JN+eHg4dDodfvjhhzYdP2jQIFRUVIiGwGKx4OjRowCAyZMn48svv8TAgQMhk8kQGhqKVatWYcKECQDotcY1/hg/++yzZp9j/PjxWLFiBQBg9erVqHFtS9hOvL+mTkKvJ/mDvXvbdzxjtEKXGpJ164DVq9u+I9iwAbjlFuD//o+uW62005AuJAoKSMaipxmCgweBd94BPvigh+hme2mVZ555BhaLBcOHD8ewYcPwzDPPAADuvfdefPbZZxg7dixOnDjRZGXc2QQHB+POO+9Eamoqrr/+epzHV04eolQq8cMPP+Dxxx9HWloaRowYIWYq8fa4kydPBgBMnDgRwcHBomtn0aJFuPrqqzFp0iSES1eDLjzxxBPYvHkzRo0ahbVr1yKB+3s7SnN5pd31r7vWERQVUX57375tP3bjxo3MYKDjX3zRcfvs2YylpzN26BBjmZmen2/3bjpXeDhjZWX0v1LZ9HGbNzOWm9v62LoTb7xBr2fu3IJzPRS3dKf3q7fXEXSEnj6uttYReHcEnQQv9svPb/2xOh3w9tvkTuLwBvPShjP+/rTTSE11uHU8ISWFLvv2dQjX8d4EUiZNcmgO9TR6qoy2Fy/dEa8h6CS46qcnPPEE8MADzm4kd4ZAo6Hz/vorsHOn5+fXaKgd5cqVTeWqezpe6RwvXjofb9ZQJ8GF4TxBJqPJmiuFAu4NQXg47QoefJD6EYwd6/lzzJzp+P9//yMF0r8D3joCL146H68h6CQmTwauvtqzYPHhw8CwYWQQOPWNafFSQ7B0Kf1FRrYta8iVe+9t/7HdjTvuoAK4+PhK9HKxWi9eOg2vIehEBg4kGeiWYAzgooDff0/GA6AWlCaT+xWvXt+2rKG/MyoVZWdlZNSe66F48fK3wRsj6CQ2bgSqq1sXcZMaClfxOJXKIaoGUOroJZdQ+mhHdgR/J3buBObOBUpLVa0/2IsXLx7hNQSdxP79wPLlrWezREY64gHSWpA1a0g1tMHROgBFRY7exd4dAbF3L/Dzz8DXX3dS/rSXc85vv/2Gl19+ufUHSli2bBlSUlJwww03tOt4Kf7dsE9rRkaGWHV8NvC6hjoJnj46cyawdq2zr9+VwEAyCNXVjtt27KCUUqmYI5/8v/6agsVevPwdmTNnDubMmdOmY9555x2sXr1a1ORp6/FtQcy1l3Xdutlms0F+DjMhvDuCToKnj2ZmOlb87njrLQoAh4Y67wjq6qgfr/S7xhcqAweSUJwXb/poR5g6tenfO+/QfQaD+/s//ZTur6xsel9r5OXlYfDgwbjjjjswbNgw3HDDDVi/fj0mTJiAgQMHIjMzEwDw6aef4v777wcA3HLLLZg/fz7Gjx+PpKQktzIPd999N3JzczFnzhy8/vrrHh2v0+lwwQUXYNSoUUhNTcWvrfhw8/PzkZKSgnvvvRejRo1CQUEBli5divPOOw/Dhw/HwoULAQBLlizBsmXLAAALFizA9Eat9w0bNuDGG28EANxzzz1IT0/H0KFDxeMAqjZ+9tlnMXHiRHz//fdYs2YNBg8ejIkTJ+Knn35q/Q3uRLyGoJOQdo/jGUBr1lD/4LlzgX2NXSU//5xy/CdOpEYxnLo6R/EXh+8Ili5tvhlNb8NrCHoWnshQu1JSUoKtW7fi999/xxNPPNHk/uXLlyM2NhYbN27EggULPDperVbj559/xr59+7Bx40b861//alXLPzs7G/PmzcP+/fuRnZ2NkydPIjMzEwcOHMDevXuxefNmTJ48GVu2bAEA7NmzBzqdDhaLBVu3bhWVRV944QXs2bMHhw4dwqZNm3Do0CHxOdRqNbZu3YrLL78cd955J1auXIktW7agtLTUsze4k/C6hjoJ6XeKG4Lt20lK+sgRYM8eYORIkn2+8UbK7ZcibUrD4Qqz334LXH99z60C7kx8fOjS25im7bTUwtbPr+X7w8Nbvr85PJGhduXyyy+HTCbDkCFDUFZW1ubndHc8YwxPPfUUNm/eDJlMhqKiIpSVlSFaqvvuQt++fTG2sXhn7dq1WLt2LUaOHAmAdhgnT57EvHnzsHfvXmi1WqhUKowaNQp79uzBli1bxJ3Cd999h/fffx9WqxUlJSXIysrC8OHDATgkrI8fP45+/fph4MCBAIAbb7wR77//fptfe3vxGoJO4r33gJtvBiZMcBiCoiLqL8Bl1Kur6b7+/Zseb7M17QswciSwaRPFB7zBYmLuXBLM69OnFN46gu6PJzLULR3T2qrd0+NXrFiBiooK7N27Fz4+PkhMTGxVYloqcscYw5NPPon/40qOEhITE/HJJ59g/PjxGD58ODZu3IhTp04hJSUFp0+fxquvvordu3cjJCQEt9xyi9PzSp+jK+SlPcXrGupEIiKogrexhwSKihzqnow5OoH17w+88goVlXF+/ZUmfVd4xbLXEBChocDixcCAAW3Q9PDS66mrq0NkZCR8fHywceNG5HsiCibhoosuwscffwxdYzCwqKgI5eXlAEhR9NVXX8XkyZMxadIkLF++HCNGjIAgCKivr4dGo0FQUBDKysqwevVqt+cfPHgwTp8+jVOnTgEAvv766w682rbjNQSdxGOPAb/9Buza5cjwWb4c+Phj4KOPaCI/fZrcP0lJgNEIHD1KOwGO64JAqwX+8Q/631tHQBw+DIwbB2Rlnf1uW156LjfccAP27NmD9PR0rFixAoMHD27T8TNnzsT111+PcePGITU1FVdddRW0jYHBSZMmoaSkBOPGjUNUVBTUarUYH0hLS8PIkSMxdOhQ3HbbbWL/AVfUajXef/99XHzxxZg4cSL68i5SZ4vmZEm76193laEePJixq692f98PP5B08t69dN1ud8gpV1Yy9vnnO9mcOYzt3+98nMlEjwEYy8npsqG3SHeSVWaMseXL6f24+OKicz0Ut3Sn98srQ91+evq4vDLU5widjtI9R4ygegCTCXj9dWoR+f/tnXtwVFWexz8/A5jBBFhALBQkwQXUYWIeCC5jKHEAeSzDgiKwDJvAbsEgjiAuDoOLsIyUDMujBndLeUjBuMwybklmIuWOKMiAyiwBAqLySGADRlAg44QQYEnw7B/ndNJp0qEDdN9e7u9T1XVvn77d95tzb+7vnsf9/gJjAq7Vh0jtQPA338DRo0nk51/5MFqzZnY5cmRtvmG/o7OGFOXGo4HgBlFRYad/FhXZnARffAHTp0NBQW0gePJJCKQiDQwMf/MNHDvWHBEISdmKiO1Kat++draM3wm4qPbq9aeGN1QUJWI0EFyFs2chdEpvYaG94w9gjA0ESUn2obCzZ+1AMdjB4uRkO5AMNlAApKTA8OF2YPnYseZ06lT/OEB5OSxbpolYAqSn2/GV7OwzXktRlJsGDQRXoUePuk/1HjsGmZnw3HO1ZZcuQYcO1jaiRQsbCEpL7Wcd3AzHiRPtsrNLtdu9O2zYYJfHjzevySoWDu0SqeVmS7ajKF6jgeAqBO7gAy2AQOtgy5babW691QaIZ56xd/8VFXVbBAATJthlIBAE06JFddjEMT/5Se14gqIoSjTQB8oaIDC18+GHa+2he/WCxx+HvXvr/06fPvaO9csvbesg4BcUGCgODPpWVdkg8dxzsHjxPh4JY95SWalTRxVFiS7aImiAwDMnubm1geDbb22KySNH4M9/tmXFxTBwoH2GYOlSePllWLAAgixFaloIXbvaZdOmdqbRRx81rGH16trvKopyJWPGjCEtLY2lS5fy4osv8v7771/T78Ta+jlS5s6dy6JFi6K6D20RNMCdd8L27dYi+sMP7d38Aw/As8/C/Pk2KIDtLnr33brjBs2bQ/AzIbm5MHp03f7tCxdsgvm77+4Q1s1x7lw1nFOUcHz11Vd8/PHHjX5S+Fqprq6mSZPoXTZr5vVH0fK6PrRF4Fi4ECZNqjsbKDHRdgstXGhdQ7duta2AYcNg1qzaKaABC+rkZJgzB+67D2bOtMEhmNBBzsAAcL9+4Y215syptQJWlOsixj7UkdpQ79y5k969e5ORkUHv3r05dOgQAEuWLGGCG1zbv38/3bt35/z583X2MWDAAE6dOkV6ejrbt28nNze3xno6JSWFOXPm1FhPHzx4sMH9hWPNmjWMHDmSoUOHMmDAACorK5kwYQIPPvggGRkZNZbWgwcPrnEWzcjIYN68eQDMnj2bVatWhbXCLikpucLyev78+XTr1o1+/fpdVd+NQAMB9uL/0kuwYgWsX19b/tvfwsaNtjvn8GGbcKZ1a/je92zKybVrrcncunV2+6Qk+1sHD1ovoYbyEoCdhnr4MLRqVRW1v01RvCQSG+p7772Xbdu2UVhYyLx585g1axYA06ZNo7i4mLy8PMaPH8/y5ctpHjJglp+fzz333MPevXtrbB2Cadu2LXv27GHy5Mk13Svh9tcQO3bsYO3atWzZsoX58+fz6KOPUlBQwAcffMCMGTOorKyssaQ+e/YsTZo04SPX7xuwpG7ICjvY8vrMmTOsX7+ewsJCNmzYQEFBwbUfgAjxdddQdbWdn//739uZPosXWwdRsOULFtguni5drI/Q7t0wapRNHpOTY3MKg7WbBtsiCFhJz5xpHyBriAcesEsdA1Biggc+1JHYUJeXl5OTk0NRUREiQlWVvTG65ZZbWLNmDWlpaUyaNCmsT09DjBgxAoCsrKyaZC/h9tcQ/fv3p7XrAti0aRP5+fk1geXixYscP36c7Oxsli1bRmpqKkOGDOG9997j/PnzlJSU0K1bN6qqquq1woa6ltfbt29n+PDhNUEvmtnXAkQ1EIjIQOCXQAKwyhizIORzcZ8PBs4DucaYPdHUVF1t++b37LEX88xMO/2zTRs7/VPE2kKMHm0HhHNz7XMEgWxiY8bY5fTptjwrC6ZMsWUtWsC4cXa6Zz1utYriOyKxoZ49ezZ9+/YlLy+PkpKSOjPoioqKSEpK4sSJE9e1/4SEhIj2F45QS+q33nqLbiFWAJcuXWLXrl107tyZ/v37c+bMGVauXElWVhbQsBX2bSH2wrG2pI5a15CIJAD/BgwC7gfGiMj9IZsNArq410Tg1WjpAVi5MpVWrewF+5FH7KygefPs073jxtXODKqqshf+ykrb3x+Y6TNkSK2zaL9+1lV00iTbvWOMDQAdO8LkyXVTTiqKEp7y8nLucg/crAkaECsvL2fq1Kls27aNsrKyetNW3sj9Rcpjjz3GK6+8UtOtU1hYCECzZs3o2LEjb775Jg899BDZ2dksWrSopssqUivsPn36kJeXx4ULF6ioqODtt9++hr+ycUSzRdATKDbGHAUQkfXAMODzoG2GAb9yznh/FJFWItLeGHMyGoLuvPMCubnW3iEx0bYIkpPtFM1g0tJs7uElS2z3zne+AydO2CQzoYE6IcF2HSmKcm08//zz5OTksGTJkpqcv2BzAD/11FN07dqV119/nb59+9KnTx/atWsXlf1FyuzZs5k2bRppaWkYY0hJSWHjxo2AtaTevHkzzZs3Jzs7m9LS0ppAMHbsWIYOHUqPHj1IT08Pa4WdmZnJqFGjSE9Pp1OnTvWOfdxoxETJxEZEngAGGmP+wb0fB/QyxjwdtM1GYIEx5kP3fjPwU2PMrpDfmohtMXDHHXdkrQ8e0W0E586dIynwhFccEa+6IH61qa6r07JlS/7SJca+fPkyCQkJHiu6EtXVOCLVVVxcTHnIbJW+ffvuNsb0qG/7aLYI6uvkCo06kWyDMWYFsAKgR48eJpI+vfrYunVrRP2BsSZedUH8alNdV+fAgQMkJ9sEPhUVFTXr8YTqahyR6kpMTKzJrxwJ0ezJLgU6Br3vAISO+ESyjaIoihJFohkICoAuIpIqIs2A0UB+yDb5wN+J5SGgPFrjA4riR6LV9avEL9dyzKPWNWSMqRaRp4F3sdNHVxtjPhORH7vPXwPewU4dLcZOHx0fLT2K4jcSExMpKyujTZs2XktRYoQxhrKyMhIb6dUe1ecIjDHvYC/2wWWvBa0bYEo0NSiKX+nQoQOlpaWcPn2aixcvNvriEAtUV+OIRFdiYiIdAolQIsTXTxYrys1M06ZNSU1NBewgdmMGD2OF6moc0dKljz0piqL4HA0EiqIoPkcDgaIois+J2pPF0UJETgPXmoWiLXDmBsq5UcSrLohfbaqrcaiuxnEz6upkjLm9vg/+3wWC60FEdoV7xNpL4lUXxK821dU4VFfj8Jsu7RpSFEXxORoIFEVRfI7fAsEKrwWEIV51QfxqU12NQ3U1Dl/p8tUYgaIoinIlfmsRKIqiKCFoIFAURfE5vgkEIjJQRA6JSLGIzPRQR0cR+UBEDojIZyIy1ZXPFZEvRWSvew32QFuJiOx3+9/lylqLyHsiUuSWfxFjTd2C6mSviJwVkWle1JeIrBaRUyLyaVBZ2PoRkZ+58+2QiDwWY13/IiIHReQTEckTkVauPEVELgTV22thfzg6usIeN4/r6zdBmkpEZK8rj2V9hbs2RP8cM8bc9C+sDfYRoDPQDNgH3O+RlvZApltPBg4D9wNzgX/0uJ5KgLYhZQuBmW59JvALj4/jV0AnL+oL6ANkAp9erX7cMd0H3AqkuvMvIYa6BgBN3PovgnSlBG/nQX3Ve9y8rq+QzxcDL3pQX+GuDVE/x/zSIugJFBtjjhpjLgHrgWFeCDHGnDTG7HHrFcAB4C4vtETIMGCtW18L/I13UvgBcMQYc61Pll8XxphtwJ9CisPVzzBgvTHmf40x/4PNudEzVrqMMZuMMdXu7R+x2f9iSpj6Coen9RVARAR4EviPaOy7IRq4NkT9HPNLILgL+CLofSlxcPEVkRQgA/hvV/S0a8qvjnUXjMMAm0Rkt4hMdGV3GJc1zi3beaArwGjq/oN6XV8Qvn7i6ZybAPxX0PtUESkUkT+ISLYHeuo7bvFSX9nA18aYoqCymNdXyLUh6ueYXwKB1FPm6bxZEUkC3gKmGWPOAq8C9wDpwEls8zTWfN8YkwkMAqaISB8PNNSL2HSnPwT+0xXFQ301RFyccyLyAlANrHNFJ4G7jTEZwHTg1yLSIoaSwh23uKgvYAx1bzZiXl/1XBvCblpP2TXVmV8CQSnQMeh9B+CER1oQkabYA73OGLMBwBjztTHmsjHmW2AlUWoWN4Qx5oRbngLynIavRaS9090eOBVrXY5BwB5jzNdOo+f15QhXP56fcyKSA/w1MNa4TmXXjVDm1ndj+5W7xkpTA8ctHuqrCTAC+E2gLNb1Vd+1gRicY34JBAVAFxFJdXeWo4F8L4S4PsjXgQPGmCVB5e2DNhsOfBr63Sjruk1EkgPr2MHGT7H1lOM2ywF+F0tdQdS5U/O6voIIVz/5wGgRuVVEUoEuwM5YiRKRgcBPgR8aY84Hld8uIgluvbPTdTSGusIdN0/ry9EPOGiMKQ0UxLK+wl0biMU5FovR8Hh4AYOxo/BHgBc81PEwtvn2CbDXvQYDbwD7XXk+0D7GujpjZyDsAz4L1BHQBtgMFLllaw/qrDlQBrQMKot5fWED0UmgCns39vcN1Q/wgjvfDgGDYqyrGNt/HDjHXnPbPu6O7z5gDzA0xrrCHjcv68uVrwF+HLJtLOsr3LUh6ueYWkwoiqL4HL90DSmKoihh0ECgKIriczQQKIqi+BwNBIqiKD5HA4GiKIrP0UCgKI1EROaJSL8b8DvnboQeRbledPqooniEiJwzxiR5rUNRtEWgKICI/EhEdjrP+eUikiAi50RksYjsEZHNInK723aNiDzh1heIyOfORG2RK+vktv/ELe925akiskNECkTk5yH7n+HKPxGRf47136/4Gw0Eiu8RkfuAUVjTvXTgMjAWuA3rb5QJ/AGYE/K91libhO8aY9KAl9xH/wr8ypWtA5a58l8CrxpjHsTmVQj8zgCsPUBPrBlbVjwZ/ik3PxoIFMXmOcgCClxmqh9gLTe+pdaA7N+xFgDBnAUuAqtEZAQQ8PT5K+DXbv2NoO99n1q/pDeCfmeAexVibQzuxQYGRYkJTbwWoChxgABrjTE/q1MoMjtkuzoDasaYahHpiQ0co4GngUfr+X0TZj14/y8bY5Y3Vrii3Ai0RaAo1sjrCRFpBzU5Yjth/z+ecNv8LfBh8Jecb3xLY8w7wDRstw7Ax9jAALaLKfC9j0LKA7wLTHC/h4jcFdCiKLFAWwSK7zHGfC4i/4TNznYL1pVyClAJfFdEdgPl2HGEYJKB34lIIvau/llX/gywWkRmAKeB8a58KjaxyVSs53xg/5vcOMUO60TMOeBHeJf7QfEZOn1UUcKg0zsVv6BdQ4qiKD5HWwSKoig+R1sEiqIoPkcDgaIois/RQKAoiuJzNBAoiqL4HA0EiqIoPuf/AKOab/p5v1TGAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import matplotlib\n", "from matplotlib import pyplot as plt\n", "# static plots\n", "%matplotlib inline \n", "\n", "### plot and examine learning curves\n", "\n", "episodes=list(range(N_episodes))\n", "\n", "plt.plot(episodes, mean_final_reward, '-k', label='mean final reward' )\n", "plt.fill_between(episodes, \n", " mean_final_reward-0.5*std_final_reward, \n", " mean_final_reward+0.5*std_final_reward, \n", " color='k', \n", " alpha=0.25)\n", "\n", "plt.plot(episodes, min_final_reward, '--b' , label='min final reward' )\n", "plt.plot(episodes, max_final_reward, '--r' , label='max final reward' )\n", "\n", "plt.xlabel('episode')\n", "plt.ylabel('final reward')\n", "\n", "plt.legend(loc='lower right')\n", "plt.grid()\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Questions\n", "\n", "0. Try out different batch sizes and hyperparameters (including different network architectures). Can you improve the performance?\n", "\n", "1. Explore the final batch of trajectories. Check the sequence of actions. Can you make sense of the solution found by the agent? Hint: think of the dynamics on the Bloch sphere and try to visualize the trajectory there.\n", "\n", "2. Compare the Policy Gradient method to conventional optimal control: can optimal control give you a control protocol that works for all states? Why or why not? \n", "\n", "3. Take one of the high-reward tranjectories in the final batch of data. Now perturb it manually at a few time steps in the first half of the protocol such that it no longer produces an optimal reward (you would have to add a function to the environment which evalues a given trajectory). Last, use the policy to see how it would react to those perturbations in real time. Will it correct on-line for the introduced mistakes (i.e. before the opisode is over)? \n", "\n", "4. Find ways to visualize the policy. What is a meaningful way to do that? " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Advanced Problems\n", "\n", "1. What is the initial state distribution $p(s_0)$ in the impelementation above? Check the performance of the PG algorith if $p(s_0)$ is \n", " * a delta distribution \n", " * a compactly-supported uniform distribution over some sector of the sphere (say a cap around the south pole)\n", " * a Gaussian distribution with non-compact support \n", " \n", "2. Introduce small Gaussian noise to the rewards, e.g. $r(s,a) \\to r(s,a) + \\delta r$ where $\\delta r \\sim \\mathcal{N}(0,\\delta)$ for some noise strength $\\delta$. Does this lead to a serious performance drop as you vary $\\delta\\in[0,0.5]$? Why or why not?\n", " \n", "2. The loop over the $N_{MC}$ trajectories slows down the algorithm significantly. Consider ways to speed up the evaluation of a single PG iteration. This may include a modification of the environment `QubitEnv2` or the use of parallelization software (see JAX's function [`vmap`](https://jax.readthedocs.io/en/latest/jax.html#vectorization-vmap) and [`pmap`](https://jax.readthedocs.io/en/latest/jax.html#parallelization-pmap)). \n", " \n", "3. Change the environment `QubitEnv2` to define a nonepisodic task. Additionally, introduce a \"stop\" action so that when the agent bring the RL state close to $s_\\mathrm{target}$ the episode comes to an end and the environment is reset. This would require you to also modify the Policy Gradient implementation above because episodes now can have different length. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "RL_class", "language": "python", "name": "rl_class" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" }, "latex_metadata": { "affiliation": "Faculty of Physics, Sofia University, 5 James Bourchier Blvd., 1164 Sofia, Bulgaria", "author": "Marin Bukov", "title": "Reinforcement Learning Course: WiSe 2020/21" } }, "nbformat": 4, "nbformat_minor": 4 }