Overview In this assignment, you will gain experience working with OpenAI Gym, which is a set of problems that can be explored with different reinforcement learning algorithms. This assignment is designed to help you apply the concepts you have been learning about Q-learning algorithms to the “cartpole” problem, a common reinforcement learning problem. Note: The original code referenced in this assignment was written in Python 2.x. You have been given a zipped folder containing an updated Python 3 version of the code that will work in the Apporto environment. To make this code work, some lines have been commented out. Please leave these as comments. Reference: Surma, G. (2018). Cartpole. Github repository. Retrieved from https://github.com/gsurma/cartpole. Prompt Access the Virtual Lab (Apporto) by using the link in the Virtual Lab Access module. It is recommended that you use the Chrome browser to access the Virtual Lab. If prompted to allow the Virtual Lab access to your clipboard, click “Yes”, as this will allow you to copy text from your desktop into applications in the Virtual Lab environment. Review the following reading: Cartpole: Introduction to Reinforcement Learning. In order to run the code, upload the Cartpole.zip folder into the Virtual Lab (Apporto). Unzip the folder, then upload the unzipped folder into your Documents folder in Apporto. Refer to the Jupyter Notebook in Apporto (Virtual Lab) Tutorial to help with these tasks. Note: The Cartpole folder contains the Cartpole.ipynb file (Jupyter Notebook) and a scores folder containing score_logger.py (Python file). It is very important to keep the score_logger.py file in the scores folder (directory). Open Jupyter Notebook and open up the Cartpole.ipynb and score_logger.py files. Be sure to review the code in both of these files. Rename the Cartpole.ipynb file using the following naming convention: <YourLastName>_<YourFirstName>_Assignment5.ipynb Thus, if your name is Jane Doe, please name the submission file “Doe_Jane_Assignment5.ipynb”. Next, run the code in Cartpole.ipynb. The code will take several minutes to run and you should see a stream of output while the file runs. When you see the following output, the program is complete: Solved in _ runs, _ total runs. Note: If you receive the error “NameError: name ‘exit’ is not defined” after the above line, you can ignore it. Modify the values for the exploration factor, discount factor, and learning rates in the code to understand how those values affect the performance of the algorithm. Be sure to place each experiment in a different code block so that your instructor can view all of your changes. Note: Discount factor = GAMMA, learning rate = LEARNING_RATE, exploration factor = combination of EXPLORATION_MAX, EXPLORATION_MIN, and EXPLORATION_DECAY. Create a Markdown cell in your Jupyter Notebook after the code and its outputs. In this cell, you will be asked to analyze the code and relate it to the concepts from your readings. You are expected to include resources to support your answers, and must include citations for those resources. Specifically, you must address the following rubric criteria: Explain how reinforcement learning concepts apply to the cartpole problem. What is the goal of the agent in this case? What are the various state values? What are the possible actions that can be performed? What reinforcement algorithm is used for this problem? Analyze how experience replay is applied to the cartpole problem. How does experience replay work in this algorithm? What is the effect of introducing a discount factor for calculating the future rewards? Analyze how neural networks are used in deep Q-learning. Explain the neural network architecture that is used in the cartpole problem. How does the neural network make the Q-learning algorithm more efficient? What difference do you see in the algorithm performance when you increase or decrease the learning rate?

Standards are designed to ensure consistency. Without them, no structure would exist.Go to Basic Search: Strayer University Online Library to locate and integrate at least two quality, academic resources (in addition to your textbook) on the purpose of the ISO/IEC 27000 Suite of Security Standards. You may also use government websites, such as Cybersecurity from the National Institute of Standards and Technology.Please respond to the following in a post of at least 200 words:Describe the purpose of the ISO/IEC 27000 Suite of Security Standards.Justify the value they bring to cybersecurity.Provide full citations and references, formatted according to Strayer Writing Standards.For assistance and information, please refer to the Strayer Writing Standards link in the left-hand menu of your course. Check with your professor for any additional instructions.In 60 to 75 words, please respond to at least one other post. Choose to respond to those who have few or no responses.Student's response:Professor and Peers,Ed here. In this week’s discussion, we’re going to dig a little bit dipper into the ISO/IEC 27000 Suite of Security Standards. Specifically, ISO stands for – International Standards Organization., while IEC stands for – International Electrotechnical commission. ISO/IEC fall into what I would refer as computer/cyber security management, which is a primal necessity, which should be considered seriously by organizations. It identifies suitable countermeasures to mitigate security risks in order to offer safety for computer and cyber resources.What is the purpose of the ISO/IEC 27000 Suite of Security Standards?ISO 27000 series security standards designed as ‘ISO 27000 family of standards’ or ‘ISO 27k’ are the most prominent international standards adapted by enterprises. It provides guidelines for information security and presents both physical and security practices and procedures. It includes in particular ISO 27001, ISO 27002, ISO 27005, etc. standards, which vary in scope and purpose as well as in depth, and level of detail.Our textbook tells us on subsection 1.4 that the ISO/IEC 270001 family of standards, also known as the ISO 27000 series, is a series of best practices to help organizations improve their information security.Additional readings tell us that the ISO/IEC 27000 Suite of Security Standards consists of 46 individual standards, including ISO 27000, which provides an introduction to the family as well as clarifying key terms and definitions (Irwin, 2020). Now, let’s take a quick look at a couple of these standards, not all.ISO 27000 SeriesISO27001ISMS RequirementsISO27002ISMS controlsISO27003ISMS implementation guidelinesISO27004ISMS MeasurementsISO27005Risk managementISO27006Guidelines for ISO 27000 accreditation bodies(Vanderburg & Johnson, 2011).The summation of all these, is that, The ISO 27000 series provides recommendations for “establishing, implementing, operating, monitoring, reviewing, maintaining, and improving an Information Security Management System” (http://www.27000.org).ISO 27001We’re told that this is the central standard in the ISO 27000 series, containing the implementation requirements for an ISMS, - which is the information security management system. In essence, the ISMS is a systematic approach to risk management, containing measures that address the three pillars of information security: people, processes and technology. I think it would be important to point out here, that ISO IEC 27001: 2013 is the only standard in the series that organizations can be audited and certified against. And that’s because it contains an overview of everything organizations must do to achieve compliance (Irwin, 2020).ISO 27002We’re told that this is a supplementary standard that provides an overview of information security controls that organizations might choose to implement. Organizations are only required to adopt controls that they deem relevant – something that only becomes apparent during a risk assessment. This standard contains a more comprehensive overview, explaining how each control works, what its objective is and how organizations can implement it (Irwin, 2020).ISO 27017 and ISO 27018These are supplementary standards that were introduced in 2015, explaining how organizations should protect sensitive information in the Cloud, especially now that a lot of organizations are moving much of their sensitive information to the cloud. 27017 is a code of practice for information security, providing extra information about how to apply controls to information stored in the Cloud. While 27018 works in essentially the same way but with extra consideration for personal data (Irwin, 2020).The latest standard in all of these, we’re told, is the ISO 27701, which covers what organizations must do when implementing a privacy information management system (PIMS).What is the value the ISO/IEC 27000 Suite of Security Standards bring to cybersecurity?We can tell, professor and peers, that in course of our studies, and in real life, we’ve all come across that data breaches are one of the biggest information security risks that organizations face. Sensitive data is used across all areas of businesses these days, increasing its value for genuine and dishonest use. An unbelievable number of instances have occurred, and are continuing to occur every month, in the form of cyber criminals hacking into a database or employees losing or misappropriating information, or one country hacking into another. Wherever the data goes, the financial and reputational damage caused by a breach can be devastating. That’s why organizations are increasingly investing heavily in their defenses, using ISO 27001 as a guideline for effective security. These standards can be applied to organizations of any size and in any sector, and the framework’s broadness means its implementation will always be appropriate to the size of the business.My discussion for week 3, professor and peers.Ed.References:Stallings, W. (2019). Effective Cybersecurity: A Guide to Using Best Practices and Standards. Pearson Education (1stedition).Irwin, L. (2020). What is the ISO 27000 series of standards? Retrieved from(https://www.itgovernance.co.uk/blog/what-is-the-iso-27000-series-of-standards).Vanderburg, E. & Johnson, N. (2011). Information Security Compliance: ISO 27000. Retrieved from(https://www.tcdi.com/iso-27000-certification-history-overview/).Meriah, Ines; Arfa Rabai, Latifa Ben. (2019). Comparative Study of Ontologies Based ISO 27000 Series SecurityStandards. Procedia Computer Science Volume: 160 (2019) ISSN: 1877-0509 Online ISSN: 1877-0509. Retrieved from Basic Search: Strayer University Online Library: & (https://www.sciencedirect.com/science/article/pii/S187705091931662X).