Space exploration is one of the most expensive and sophisticated industries in the world, usually funded by country governments. SpaceX, a private space exploration company, aims to reduce the cost of space exploration by landing rockets on the ground and reuse them instead of abandoning them into the atmosphere. However, achieving this goal was not easy for SpaceX. Their rockets crashed many times before the successful landing of Falcon 9. In this project, we try to improve a reinforcement learning algorithm based on a neural network to land a rocket in a simulated environment. Our aim is to reduce the number of SpaceX landing failures. To achieve this goal, we have studied many algorithms and came up with Proximal Policy Optimization (PPO) algorithm to attack this problem. PPO is an algorithm that learns the policy (i.e. the function that maps a state to action) and the value of a state by using an artificial neural network. In this project, we tried to modify the neural network part to achieve better results, also we tried to modify the PPO algorithm to adopt RNN to consider the previous steps in each episode which is a new idea.
You can see the steps of the training of our best model and how it performs in different stages of training in the following link:
https://youtu.be/UaHEVTesnkk
Developed an algorithm in Matlab for the complex and novel problem of inferring missing SNPs in Genome based on belief propagation algorithm. Results of this project is submitted to IEEE/ACM Transactions on Computational Biology and Bioinformatics. (Link to paper)
Environment: MatlabDeveloped a program in C# language which composes a music that stimulates the given feeling in humans like happiness, sadness and etc. using selected combination of instruments. The program First finds the music from a database we gathered, according to user selections of music feeling and instruments. Then it uses an edited Hidden Markov Model algorithm to learn the structures of these music and create a new music accordingly.
Environment: C#, SQL server.This project is a very simple simulation of world which consists of some creatures and food which distributed randomly. Every creature has its own mind which is a simple neural network and has random number of neurons and it can eat food or attempt to eat other creatures. The creatures learn to go after food and it shows how they interact with each other and their environment.
Environment: C#, BrainNetiman at cs.umass.edu
Computer Science Department
University of Massachusetts Amherst
Amherst, MA 01003, US