PyBrain

This project explores deep artificial neural networks and their use with Google’s open-source library TensorFlow. We begin by laying the theoretical foundations of these networks, covering their motivation, techniques used and some mathematical aspects of their training. Special attention is paid to various regularisation methods which are applied later on. After that, we delve into the computational approach, explaining TensorFlow’s operation principles and the necessary concepts for its use, namely the computational graph, variables and execution sessions. Through the first example of a deep network, we illustrate the theoretical and TensorFlow-related elements described earlier, applying them to the problem of classifying flowers of the Iris species. We then pave the way for the problem of image classification: we comment several higher-level TensorFlow wrappers (focusing on Slim, a library born within Google itself which is used in the last part of the project), describe the basic principles of convolutional networks and introduce the MNIST problem (automatic handwritten digit recognition), outlining its history and current state of the art. Finally, we create three convolutional networks to tackle MNIST, detailing how such a task is approached with TensorFlow and the workflow followed. All three networks reach over 98% classification accuracy, going as far as 99.52% in the case of the best one. We conclude with an explanation of the obtained results, relating the structures of the different networks with their performance and training cost.

Frontiers in Neuroinformatics, 2021

More than half of the Top 10 supercomputing sites worldwide use GPU accelerators and they are becoming ubiquitous in workstations and edge computing devices. GeNN is a C++ library for generating efficient spiking neural network simulation code for GPUs. However, until now, the full flexibility of GeNN could only be harnessed by writing model descriptions and simulation code in C++. Here we present PyGeNN, a Python package which exposes all of GeNN's functionality to Python with minimal overhead. This provides an alternative, arguably more user-friendly, way of using GeNN and allows modelers to use GeNN within the growing Python-based machine learning and computational neuroscience ecosystems. In addition, we demonstrate that, in both Python and C++ GeNN simulations, the overheads of recording spiking data can strongly affect runtimes and show how a new spike recording system can reduce these overheads by up to 10×. Using the new recording system, we demonstrate that by using PyGeNN on a modern GPU, we can simulate a full-scale model of a cortical column faster even than real-time neuromorphic systems. Finally, we show that long simulations of a smaller model with complex stimuli and a custom three-factor learning rule defined in PyGeNN can be simulated almost two orders of magnitude faster than real-time.

TensorFlow [1] is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition , computer vision, robotics, information retrieval, natural language processing, geographic information extraction, and computational drug discovery. This paper describes the Ten-sorFlow interface and an implementation of that interface that we have built at Google. The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.

Python for Scientific Computing and Artificial Intelligence, 2023

Python for Scientific Computing and Artificial Intelligence is split into 3 parts: in Section 1, the reader is introduced to the Python programming language and shown how Python can aid in the understanding of advanced High School Mathematics. In Section 2, the reader is shown how Python can be used to solve real-world problems from a broad range of scientific disciplines. Finally, in Section 3, the reader is introduced to neural networks and shown how TensorFlow (written in Python) can be used to solve a large array of problems in Artificial Intelligence (AI). This book was developed from a series of national and international workshops that the author has been delivering for over twenty years. The book is beginner friendly and has a strong practical emphasis on programming and computational modelling. Features: • No prior experience of programming is required. • Online GitHub repository available with codes for readers to practice. • Covers applications and examples from biology, chemistry, computer science, data science, electrical and mechanical engineering, economics, mathematics, physics, statistics and binary oscillator computing. • Full solutions to exercises are available as Jupyter notebooks on the Web.

Frontiers in Neuroinformatics, 2018

The development of spiking neural network simulation software is a critical component enabling the modeling of neural systems and the development of biologically inspired algorithms. Existing software frameworks support a wide range of neural functionality, software abstraction levels, and hardware devices, yet are typically not suitable for rapid prototyping or application to problems in the domain of machine learning. In this paper, we describe a new Python package for the simulation of spiking neural networks, specifically geared toward machine learning and reinforcement learning. Our software, called BindsNET 1 , enables rapid building and simulation of spiking networks and features user-friendly, concise syntax. BindsNET is built on the PyTorch deep neural networks library, facilitating the implementation of spiking neural networks on fast CPU and GPU computational platforms. Moreover, the BindsNET framework can be adjusted to utilize other existing computing and hardware backends; e.g., TensorFlow and SpiNNaker. We provide an interface with the OpenAI gym library, allowing for training and evaluation of spiking networks on reinforcement learning environments. We argue that this package facilitates the use of spiking networks for large-scale machine learning problems and show some simple examples by using BindsNET in practice.

BMC …, 2007

Computational neuroscience has produced a diversity of software for simulations of networks of spiking neurons, with both negative and positive consequences. On the one hand, each simulator uses its own programming or configuration language, leading to considerable difficulty in porting models from one simulator to another. This impedes commu- nication between investigators and makes it harder to reproduce and build on the work of others. On the other hand, simulation results can be cross-checked between different simulators, giving greater confidence in their correctness, and each simulator has different optimizations, so the most appropriate simulator can be chosen for a given modelling task. A common programming interface to multiple simulators would reduce or eliminate the problems of simulator diversity while retaining the benefits. PyNN is such an interface, making it possible to write a simulation script once, using the Python programming language, and run it without modification on any supported simulator (currently NEURON, NEST, PCSIM, Brian and the Heidelberg VLSI neuromorphic hardware). PyNN increases the productivity of neuronal network modelling by providing high-level abstraction, by promoting code sharing and reuse, and by providing a foundation for simulator-agnostic analysis, visualization, and data-management tools. PyNN increases the reliability of modelling stud- ies by making it much easier to check results on multiple simulators. PyNN is open-source software and is available from http://neuralensemble.org/PyNN.

Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algo- rithms for medium-scale supervised and unsupervised problems. This package focuses on bring- ing machine learning to non-specialists using a general-purpose high-level language. Emphasis is put on ease of use, performance, documentation, and API consistency. It has minimal dependen- cies and is distributed under the simplified BSD license, encouraging its use in both academic and commercial settings. Source code, binaries, and documentation can be downloaded from http://scikit-learn.sourceforge.net.

Cornell University - arXiv, 2013

Pylearn2 is a machine learning research library. This does not just mean that it is a collection of machine learning algorithms that share a common API; it means that it has been designed for flexibility and extensibility in order to facilitate research projects that involve new or unusual use cases. In this paper we give a brief history of the library, an overview of its basic philosophy, a summary of the library's architecture, and a description of how the Pylearn2 community functions socially.

Deep learning allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction. These methods have dramatically improved the state-of-the-art in speech recognition, visual object recognition, object detection and many other domains such as drug discovery and genomics. Deep learning discovers intricate structure in large data sets by using the backpropagation algorithm to indicate how a machine should change its internal parameters that are used to compute the representation in each layer from the representation in the previous layer. Deep convolutional nets have brought about breakthroughs in processing images, video, speech and audio, whereas recurrent nets have shone light on sequential data such as text and speech. In this paper, main topics about deep learning have been covered. The relationship between artificial intelligence, machine learning and deep learning has been mentioned briefly. Detailed information about deep learning has been given, ie. History and future of deep learning. Artificial neural networks has been reviewed. The importance of GPU and deep learning in big data have been shown deeply. Using areas of deep learning have been explained. Benefits and weaknesses of deep learning have been covered. The informations about deep learning algorithms, libraries and tools have been given.