Hossein Khazaei, PhD

Hossein Khazaei, PhD

Passionate about science!

Projects

User-User Collaborative Filtering

Published:

Summary: The goal is to desing a User-User Collaborative Filtering that we can use to predict how would a given user would rate a given movie that he/she has not seen before. The idea is to identify users with the most similar interactions profile (nearest neighbors) to that user and see how they rated that movie, and based on the similarity coefficients (for between users) that we calculate, we then predict the ratings. We can later on use such ratings to recommend new movies to the users. We use the 20 million MovieLens data set available on Kaggle. Though, for practical implementation on a pc we shrink this dataset.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Item-Item Collaborative Filtering

Published:

Summary: The goal is to desing a Item-Item Collaborative Filtering that we can use to predict how would a given user would rate a given movie that he/she has not seen before. The idea is to find items similar to the ones the user already “positively” interacted with. This method is said to be “item-centered” as it represents items based on interactions users had with them and evaluate distances between those items. We use the 20 million MovieLens data set available on Kaggle. Though, for practical implementation on a pc we shrink this dataset.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Matrix Factorization using Alternating Least Squares

Published:

Summary: The goal is to derive latent representation of the user and item feature vectors. The (predicted) ratings that a user gives to an item is the inner product of user’s latent vector and the item’s latent vector. We use alternating least squares for training. We use the 20 million MovieLens data set available on Kaggle. Though, for practical implementation on a pc we shrink this dataset.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Matrix Factorization using Embedding Layers

Published:

Summary: The goal is to derive latent representation of the user and item feature vectors. The (predicted) ratings that a user gives to an item is the inner product of user’s latent vector and the item’s latent vector. The idea here is to use the technique that we often used when we want to convert categorical variables into numerical vectors: Embedding. Keras has embedding layers that automatically learns the latent representation of such cases. We use the 20 million MovieLens data set available on Kaggle. Though, for practical implementation on a pc we shrink this dataset.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Matrix Factorization using Auto-Encoder

Published:

Summary: The goal is to derive latent representation of the user and item feature vectors. The (predicted) ratings that a user gives to an item is the inner product of user’s latent vector and the item’s latent vector. The idea here is to use autoencoder to get the latent representations. We use the 20 million MovieLens data set available on Kaggle. Though, for practical implementation on a pc we shrink this dataset.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Bayesian Bandit Recommender System.

Published:

Summary: We have several bandits that we do not know the success rate of any of them. We want to implement an algorithm that automatically balances the exploration-exploitation, and achieves finds the optimal bandit for us. The idea here is to use Bayesian Bandit algorithm to for choosing the bandits in a series of experiments.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Control of Cart Pole with a Policy Search

Published:

Task: The goal is to design a control policy to keep a Cart Pole leveled.
Solution: We assume a simple linear policy. We use a simple algorithm to find the optimal policy by brute-force policy search.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Control of Cart Pole with State-Action Value Functions

Published:

Task: The goal is to control a Cart Pole using value functions, particularly, Q-values.
Solution: We find state-action value functions (known also as Q-values) and use those for finding the optimal policy.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Control Policy for Mountain Car with Q-value Function Approximator

Published:

Task: The goal is to design a control policy for a car in a valley to get on the top of the valley as soon as possible.
Solution: We first use RBF kernels to map the features to higher feature space. We then assume a simple linear policy, and learn the state-action value functions (Q-values) to learn the optimal policy.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Finding Control Policy for Mountain Car using TD(n) for finding the Q-value Function Approximator

Published:

Task: The goal is to design a control policy for a car in a valley to get on the top of the valley as soon as possible.
Solution: Here we use the TD(n) for estimating the state-action values for different pairs of (state, action). Using such estimated values as training samples, we train a (linear) function approximator to estimate the state-action values (Q-values). Note that we use RBF kernels to map the features to higher feature space. This increases the performance of the model. After learning the state-action value functions (Q-values) we derive the optimal policy.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Finding Control Policy for Mountain Car using TD(λ) and elibility traces for finding the Q-value Function Approximator

Published:

Task: The goal is to design a control policy for a car in a valley to get on the top of the valley as soon as possible.
Solution: Here we use the TD(λ) for estimating the state-action values for different pairs of (state, action). The Eligibility Traces allow us to use a Backward-View implementation, which means that we do not need to wait until the end of episodes to calculate the updates. Using such estimated values as training samples, we train a (linear) function approximator (with some kernels to shift the feature space to a higher dimension space) to estimate the state-action values (Q-values).
For more information on this project, please see link. To see the Github repository for this project, see Github.

Policy Gradient with Baseline

Published:

Task: The goal is to directly model the policy function and try to optimize it. This approach is called policy gradient.
Solution: Here we model the policy function using a neural network, and try to optimize it. The policy gradient approach typically has hagh variance, and hence we use a technique called baseline to decrease the variance of the model. The baseline requires the calculating of the state value functions V(s). We use another neural network to calculate the value functions. The challenge of this project is to calculate the gradient when trying to optimize the policy function. Here we show a trick to do that.
For more information on this project, please see link. To see the Github repository for this project, see Github.

Policy Gradient with Baseline in Continuous Action Space

Published:

Task: The goal is to directly model the policy function and try to optimize it (i.e. policy gradient). The challenge is that we have continuous action space (i.e. infinite number of actions available to us), and hence the option of training one model per action is not available to us.
Solution: The trick we play is that we assume that the action comes from a Gaussian probability distribution function, and we estimate the parameters of that distribution function. We also use the baseline to help improving the performance of the model, and to do that, we use another neural network to estimate the state value functions V(s).
For more information on this project, please see link. To see the Github repository for this project, see Github.

Prediction model for COVID-19 Pandemic

Published:

In this project we develop a predicting model based on SEIR epidemiological model coupled with the replicator dynamics model. The Hospitalization data is used to make predictions on the pandemic.
For more information and some of the results, see link. To see the corresponding publication on the theoretical aspects of the model we used here, see link.

Machine Learning Algorithms for Solving Optimization Problems

Published:

In this project we use the domain knowledge + machine learning to solve large quadratic optimization problems with many inequalities. Knowing that the regions for the input parameters corresponding to each active set (class) are linearly separable, we leverage the rich body of classification algorithms to predict the active set at optimum, and use that to find the optimum. We use different classification algorithms, including LDA, neural networks, SVM, and XGBoost. Our simulations show high accuracies for LDA and neural networks for this classification task.
For more information, see link. To see the pdf file for the corresponding publication for the model, see link.
To see the Github repository for this project, see Github.

Simulating Pandemics with Multiple Waves

Published:

This project consists of the simulations we performed for the model we proposed for the pandemics. Watching COVID-19 pandemic we realized that the classic epidemiological models, such as SIS, SIR, SEIR, etc., fail to capture the dynamics of the pandemics, specifically, the potential for multiple waves of the disease. Here we modified the SEIR model to include the behavior of the individuals in the society as a part of the model.
For more information, see link. To see the pdf file for the corresponding publication for the model, see link.
To see the Github repository for this project, see Github.

Time-Series Forecast + Scenario Reduction + Stochastic Optimization for household with PV pandels and energy storage

Published:

For a house with photovoltaic (PV) panels and energy storage participating actively in buying/selling energy in two-settlement market, the problem it how to deal with the uncertainty of power generation of PV panels and the market prices. The solution is stochastic optimization. This approach characterizes the uncertainties as scenarios then tries to minimize the expected cost. In this project,

  • LSTM is used to generate single forecasts about unknown market prices for the coming hours.
  • CNN and is used to generate single forecasts about unknown demand for the coming hours.
  • SARIMA is used to generate the benchmark for the forecasts of market prices and demand.
  • The joint probability distribution function (pdf) of the forecast errors (during the training period) is used for generating forecast errors, and add them to the forecasts generated by CNN, LSTM, and SARIMA to generate the overall scenarios.
  • A Simultaneous Backward Scenario Reduction is used to decrease the number of scenarios while keeping the most important information in the scenarios unchanged.
  • with each new observation of the data, the conditional joint pdf of the forecast errors are calculated, and the new pdf is used to generate scenarios.

For more information on this project, please see link. To see the pdf file for a corresponding publication (where we published some of the results of this project), see link.

Iterative Approach to Solve Distributed Optimization for Coupled Intra-Market of Energy

Published:

Task: Imagine a intra-market between several regional markets, where the players of the intra-market are the operators of the regional markets. The goal of this intra-market is to facilitate trade between markets.
Challenge: How to clear this market such that the regional market operators reveal as little as possible information.
Solution: We formulate the problem as a distributed optimization problem, look at the minimum information needed to be exchanged in the market, especially the coupled regional markets, and through an iterative mechanism ensure that the outcome of this iterative approach makes the results of the intra-market converge to the optimal results.
For more information on this project, please see link. To see the Github repository for this project, see Github.