Markov reward processes are often studied in the context of Markov decision processes which are the mathematical basis for reinforcement learning. As a first step to developing spiking algorithms for Markov decision processes, we developed a spiking algorithm that estimates the state value function of a Markov reward process. The algorithm is composed of circuits that perform streaming binary arthimetic. A "1" is represented by a spike, and a "0" is represented by no spike. For each state there is a dot product, multiplcation, and addition circuit that perform each step in computing the Bellman equation: \[V(s) = R(s)+\gamma\sum_{s'\in S}P_{ss'}V(s').\] Currently, we are performing scaling studies on the Loihi 2 architecture. Future work will involve optimizing the current algorithm, developing algorithms for Markov models with unknown rewards and transition probabilities, and adding actions for Markov decision processes. The figure to the left shows an example diagram of the algorithm (A) for a three state Markov reward process (B).

Grid cells have been shown to encode physical locations with periodic phase-space representations. This project took elevation trajectories from a map and mapped them to a grid cell based phase-space coordinate system. Decoding the resulting phase-space coordinates showed that this method of encoding was able to localize position in space. For this project I worked on developing a metric for calculating the distance between two points in the phase-space coordinate system and a spiking algorithm for a neuromorphic implementation of the algorithm. The figure to the left shows coincidence maps of overlapping the activity patterns of different grid modules.

Spindles are oscillatory events that occur in electroencephalogram (EEG) recordings of neural activity in the brain during sleep. The goal of this project was to characterize spindles through the extraction of certain features and applying different levels of analysis - linear/correlation, principal component analysis (PCA), and fuzzy c-means clustering - to discover potential spindle classes. Five features were extracted: the amplitude of the K-Complex, which is a large downward deflection in the local field potential, the width of the K-Complex, the average spindle amplitude, the standard deviation of the amplitude, and the average frequency of the oscillations. Using PCA, I found that the majority of the variance was explained by the amplitude. I then clustered based on the amplitude using fuzzy c-means clustering. The results indicated that spindle amplitude could be clustered in to five different classes. The figure to the left is a plot of the spindles in the PCA space with the colors corresponding to the different clusters found with fuzzy c-means.

With the rise in brain training games like Lumosity, we were interested in whether the claims being made of widespread transferable benefits were true. We compared a no- contact control group of participants with two Lumosity groups and two active control groups. One Lumosity group focused on training cognitive flexibility and the other foced on training working memory, while the active control groups did either Sudoku puzzles (alternative task) or online trivia games (crystallized intelligence). The cognitive flexibility, working memory and fluid intelligence of the participants were tested before and after six weeks of training three to five days a week. Every group except the control showed significant improvement after the six weeks, but no training group was significantly better than the others. The figure to the left shows the pre- and post- measures of each group for four different tests: matrix reasoning (general intelligence), paper folding (general intelligence), Stroop (cognitive flexibility), and memory span (working memory).

Previous work has shown that an image representation based on a neurobiological model of simple-complex cells (Hubel and Wiesel model) is selective and invariant to group transformations and reduces the sample complexity of a learning task. I constructed an invariant image representation for illumination transformations and compared invariance and selectivity of this representation for two sets of images of faces under varying illumination conditions to a fake representation which was constructed from random images in the set. Invariance and selectivity were measured using the Euclidian distance between the image under each transformation with the invariant representation. We found that the image representation was invariant compared to the fake representation, but not selective. The figure to the left shows examples of the image dataset (left), the Euclidian distance between each transformed image in a group with the invariant signature (blue) compared to the fake orbit (orange) (middle), and the Euclidian distance between the invariant representations of each group (right).