MarI/O and NeuroEvolution of Augmenting Topologies

Date: March 13th 2017 at 4:00pm
Location: Rm 4421, The CS Thesis Room
Presenter: Thomas Flynn

The topology of artificial neural networks has typically been designed by human engineering, but there exists the potential for discovering better network topologies through automated search. Genetic algorithms give a biologically plausible method for starting with a minimally viable network and evolving its topology to accomplish a given task. This week we’ll be looking at MarI/O, an implementation of NeuroEvolution of Augmenting Topologies, for beating a level of the popular video game Super Mario Brothers 3 which is known to be an NP-Hard game.

Generative Adversarial Networks

Topic: Generative Adversarial Networks
Date: February 27th 2017 at 4:00pm
Location: Rm 4421, CS Thesis Room
Presenter: Allan Zelener
Links: Paper, Videos

Generative Adversarial Networks [Goodfellow et al., 2014] is a recent seminal paper combining generative models, adversarial training, and neural networks. There has been much interest in GANs over the past few years due to their potential for learning with limited training data and ground truth labels compared to traditional discriminative neural networks. Many open research issues exist for GANs such as stabilizing the training procedure. For the first meeting of the ML Student Seminar in 2017 we will be viewing and discussing videos from the NIPS 2016 Workshop on Adversarial Training including Introduction to GANs by Ian Goodfellow.

New Format: This year the MLSS (formerly MLRG) will adopt a new format where each member is assigned a date to lead the discussion and may choose to reschedule or cancel the meeting on that date if they are unavailable. The goal of this format is to prevent the need for finding volunteers week to week. It is also encouraged for discussion leaders to select videos from conferences as an alternative to preparing a full presentation themselves if the meeting would otherwise be canceled.

Session #07: AlphaGo – Mastering the game of Go with neural networks and tree search

Topic: AlphaGo – Mastering the game of Go with neural networks and tree search
Date: March 11th 2016 at 11:00am
Presenter: Allan Zelener
Links: AlphaGo Website, Paper (Nature 2016), Slides
Paper Abstract: The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses ‘value networks’ to evaluate board positions and ‘policy networks’ to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

Session #06: LSTM Recurrent Neural Networks

Topic: Long-Short Term Memory Recurrent Neural Networks
Date: February 19th 2016 at 4:30pm
Presenter: Leo (Siyu Liao)
Links: LSTM Paper (Neural Computation 1997), Understanding LSTMs
Description: LSTMs have recently gained popularity as an implementation for recurrent neural networks that specifically addresses the vanishing gradient problem found with recurrent backpropagation. They have been applied to a variety of sequence-to-sequence prediction tasks such as image captioning, machine translation, and generative modeling of documents.

Session #05: Deep Neural Networks with Multitask Learning

Presenter: Félix Grèzes
Date: February 5 2016 at 4:30pm
Location: CS Lab – Rm 4435
Links: A Unified Architecture for Natural Language Processing:Deep Neural Networks with Multitask Learning (Collobert and Weston, ICML 2008)
Paper Abstract: We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel form of semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in stateof-the-art performance.

Session #04: Pegasos: primal estimated sub-gradient solver for SVM

Presenter: Xing Su
Date: Friday January 29 2016 at 4:15pm
Location: CS Lab – Rm 4435
Materials: Mathematical Programming (2011), ICML 2007
Paper Abstract: We describe and analyze a simple and effective stochastic sub-gradient descent algorithm for solving the optimization problem cast by Support Vector Machines (SVM). We prove that the number of iterations required to obtain a solution of accuracy ϵ is O~(1/ϵ), where each iteration operates on a single training example. In contrast, previous analyses of stochastic gradient descent methods for SVMs require Ω(1/ϵ^2) iterations. As in previously devised SVM solvers, the number of iterations also scales linearly with 1/λ, where λ is the regularization parameter of SVM. For a linear kernel, the total run-time of our method is O~(d/(λϵ)), where d is a bound on the number of non-zero features in each example. Since the run-time does not depend directly on the size of the training set, the resulting algorithm is especially suited for learning from large datasets. Our approach also extends to non-linear kernels while working solely on the primal objective function, though in this case the runtime does depend linearly on the training set size. Our algorithm is particularly well suited for large text classification problems, where we demonstrate an order-of-magnitude speedup over previous SVM learning methods.

Session #03: Multimodal Learning with Deep Boltzmann Machines

NOTICE: To resolve upcoming conflicts with meetings and courses, we are changing our meeting time to Fridays at 4:15pm. The location is still the CS Lab (although we may start getting a room if there are distractions).

Presenter: Thomas Flynn
Date: Friday January 22 2016 at 4:15pm
Location: CS Lab – Rm 4435
Materials: JMLR 2014, NIPS 2012, Paper Website (Includes code)
Paper Abstract: A Deep Boltzmann Machine is described for learning a generative model of data that consists of multiple and diverse input modalities. The model can be used to extract a unified representation that fuses modalities together. We find that this representation is useful for classification and information retrieval tasks. The model works by learning a probability density over the space of multimodal inputs. It uses states of latent variables as representations of the input. The model can extract this representation even when some modalities are absent by sampling from the conditional distribution over them and filling them in. Our experimental results on bi-modal data consisting of images and text show that the Multimodal DBM can learn a good generative model of the joint space of image and text inputs that is useful for information retrieval from both unimodal and multimodal queries. We further demonstrate that this model significantly outperforms SVMs and LDA on discriminative tasks. Finally, we compare our model to other deep learning methods, including autoencoders and deep belief networks, and show that it achieves noticeable gains.

Session #02: Playing Atari With Deep Reinforcement Learning

Title: Playing Atari With Deep Reinforcement Learning
Presenter: Anoop Aroor
Date: Thursday January 14th 2016 at 1:00pm
Location: CS Lab – Rm 4435
Materials: Paper on ArxivSlides

Paper Abstract: We present the first deep learning model to successfully learn control policies directly from high-dimensional sensory input using  reinforcement learning. The model is a convolutional neural network, trained with a variant of Q-learning, whose input is raw pixels and whose output is a value function estimating future rewards. We apply our method to seven Atari 2600 games from the Arcade Learning Environment, with no adjustment of the architecture or learning algorithm. We find that it outperforms all previous approaches on six of the games and surpasses a human expert on three of them.

Session #01: Deep Learning With Neural Networks

Title: Deep Learning With Neural Networks: The Structure and Optimization of Neural Networks
Presenter: Allan Zelener
Date: Thursday January 7th 2016 at 1:00pm
Location: CS Lab – Rm 4435, The Graduate Center
Materials: Slides

This presentation will give an overview of recent trends in deep learning and neural network research, provide a theoretical and practical understanding of the basic structure of neural networks, discuss optimization techniques for training neural networks, and introduce the commonly used convolutional and recurrent variants of neural networks. The goal of this talk is to serve as a jumping off point for future meetings of the machine learning reading group and discussion of state-of-the-art research.

This will be an introductory talk on neural networks targeted for computer science researchers with familiarity of machine learning fundamentals, however any student is welcome to attend.