pyds


Awesome Python Data Science


Probably the best curated list of data science software in Python

Machine Learning

General Purpouse Machine Learning

  • Shogun - Machine learning toolbox.
  • xLearn - High Performance, Easy-to-use, and Scalable Machine Learning Package.
  • mlpack - A scalable C++ machine learning library (Python bindings).
  • dlib - Toolkit for making real world machine learning and data analysis applications in C++ (Python bindings).
  • pyGAM - Generalized Additive Models in Python.

Time Series

  • Prophet - Automatic Forecasting Procedure.
  • PyFlux - Open source time series library for Python.
  • bayesloop - Probabilistic programming framework that facilitates objective model selection for time-varying parameter models.
  • luminol - Anomaly Detection and Correlation library.

Automated Machine Learning

  • MLBox - A powerful Automated Machine Learning python library.

Ensemble Methods

Imbalanced Datasets

Random Forests

Extreme Learning Machine

Kernel Methods

Gradient Boosting

Deep Learning

PyTorch

  • PyToune - A Keras-like framework and utilities for PyTorch.

TensorFlow

Keras

MXNet

Chainer

  • Chainer - A flexible framework for neural networks.
  • ChainerCV - A Library for Deep Learning in Computer Vision.
  • ChainerMN - Scalable distributed deep learning with Chainer.

Theano

WARNING: Theano development has been stopped

Others

  • CNTK - Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit.
  • Neon - Intel® Nervana™ reference deep learning framework committed to best performance on all hardware.
  • Tangent - Source-to-Source Debuggable Derivatives in Pure Python.
  • autograd - Efficiently computes derivatives of numpy code.
  • Myia - Deep Learning framework (pre-alpha).
  • nnabla - Neural Network Libraries by Sony.
  • Caffe - A fast open framework for deep learning.
  • Caffe2 - A lightweight, modular, and scalable deep learning framework (now a part of PyTorch).

Data Manipulation

Data Containers

  • pandas - Powerful Python data analysis toolkit.
  • xpandas - Universal 1d/2d data containers with Transformers .functionality for data analysis by The Alan Turing Institute.
  • Arctic - High performance datastore for time series and tick data.
  • swifter - A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner.
  • pandas_flavor - A package which allow to write your own flavor of Pandas easily.
  • pandas-log - A package which allow to provide feedback about basic pandas operations and find both buisness logic and performance issues.

Pipelines

  • pdpipe - Sasy pipelines for pandas DataFrames.
  • SSPipe - Python pipe (|) operator with support for DataFrames and Numpy and Pytorch.
  • Dataset - Helps you conveniently work with random or sequential batches of your data and define data processing.
  • meza - A Python toolkit for processing tabular data.
  • Prodmodel - Build system for data science pipelines.

Feature Engineering

General

Feature Selection

Visualization

  • Matplotlib - Plotting with Python.
  • seaborn - Statistical data visualization using matplotlib.
  • Bokeh - Interactive Web Plotting for Python.
  • HoloViews - Stop plotting your data - annotate your data and let it visualize itself.
  • prettyplotlib - Painlessly create beautiful matplotlib plots.
  • python-ternary - Ternary plotting library for python with matplotlib.
  • missingno - Missing data visualization module for Python.
  • chartify - Python library that makes it easy for data scientists to create charts.
  • physt - Improved histograms.
  • animatplot - A python package for animating plots build on matplotlib.
  • plotly - A Python library that makes interactive and publication-quality graphs.

Model Explanation

  • Alibi - Algorithms for monitoring and explaining machine learning models.
  • anchor - Code for "High-Precision Model-Agnostic Explanations" paper.
  • aequitas - Bias and Fairness Audit Toolkit.
  • ELI5 - A library for debugging/inspecting machine learning classifiers and explaining their predictions.
  • L2X - Code for replicating the experiments in the paper Learning to Explain: An Information-Theoretic Perspective on Model Interpretation.
  • PDPbox - Partial dependence plot toolbox.
  • PyCEbox - Python Individual Conditional Expectation Plot Toolbox.
  • Skater - Python Library for Model Interpretation.
  • AI Explainability 360 - Interpretability and explainability of data and machine learning models.
  • Auralisation - Auralisation of learned features in CNN (for audio).
  • CapsNet-Visualization - A visualization of the CapsNet layers to better understand how it works.
  • lucid - A collection of infrastructure and tools for research in neural network interpretability.
  • Netron - Visualizer for deep learning and machine learning models (no Python code, but visualizes models from most Python Deep Learning frameworks).
  • FlashLight - Visualization Tool for your NeuralNetwork.
  • tensorboard-pytorch - Tensorboard for pytorch (and chainer, mxnet, numpy, ...).

Reinforcement Learning

  • OpenAI Gym - A toolkit for developing and comparing reinforcement learning algorithms.
  • Coach - Easy experimentation with state of the art Reinforcement Learning algorithms.
  • garage - A toolkit for reproducible reinforcement learning research.
  • OpenAI Baselines - High-quality implementations of reinforcement learning algorithms.
  • Stable Baselines - A set of improved implementations of reinforcement learning algorithms based on OpenAI Baselines.
  • RLlib - Scalable Reinforcement Learning.
  • Horizon - A platform for Applied Reinforcement Learning.
  • Dopamine - A research framework for fast prototyping of reinforcement learning algorithms.
  • ChainerRL - A deep reinforcement learning library built on top of Chainer.

Distributed Computing

  • Veles - Distributed machine learning platform.
  • Jubatus - Framework and Library for Distributed Online Machine Learning.
  • DMTK - Microsoft Distributed Machine Learning Toolkit.
  • PaddlePaddle - PArallel Distributed Deep LEarning.
  • Distributed - Distributed computation in Python.

Probabilistic Methods

  • PyMC - Bayesian Stochastic Modelling in Python.
  • sampled - Decorator for reusable models in PyMC3.
  • PyStan - Bayesian inference using the No-U-Turn sampler (Python interface).
  • pgmpy - A python library for working with Probabilistic Graphical Models.
  • emcee - The Python ensemble sampling toolkit for affine-invariant MCMC.
  • hsmmlearn - A library for hidden semi-Markov models with explicit durations.
  • pyhsmm - Bayesian inference in HSMMs and HMMs.

Genetic Programming

  • DEAP - Distributed Evolutionary Algorithms in Python.
  • monkeys - A strongly-typed genetic programming framework for Python.

Optimization

  • Spearmint - Bayesian optimization.
  • scikit-opt - Heuristic Algorithms for optimization.
  • SMAC3 - Sequential Model-based Algorithm Configuration.
  • Optunity - Is a library containing various optimizers for hyperparameter tuning.
  • hyperopt - Distributed Asynchronous Hyperparameter Optimization in Python.
  • Bayesian Optimization - A Python implementation of global optimization with gaussian processes.
  • SafeOpt - Safe Bayesian Optimization.
  • scikit-optimize - Sequential model-based optimization with a scipy.optimize interface.
  • Solid - A comprehensive gradient-free optimization framework written in Python.
  • PySwarms - A research toolkit for particle swarm optimization in Python.
  • Platypus - A Free and Open Source Python Library for Multiobjective Optimization.
  • POT - Python Optimal Transport library.
  • Talos - Hyperparameter Optimization for Keras Models.
  • nlopt - Library for nonlinear optimization (global and local, constrained or unconstrained).

Natural Language Processing

  • NLTK - Modules, data sets, and tutorials supporting research and development in Natural Language Processing.
  • CLTK - The Classical Language Toolkik.
  • gensim - Topic Modelling for Humans.
  • PSI-Toolkit - A natural language processing toolkit.
  • pyMorfologik - Python binding for Morfologik.
  • Phonemizer - Simple text to phonemes converter for multiple languages.
  • flair - Very simple framework for state-of-the-art NLP.
  • spaCy - Industrial-Strength Natural Language Processing.

Computer Audition

  • librosa - Python library for audio and music analysis.
  • Yaafe - Audio features extraction.
  • aubio - A library for audio and music analysis.
  • Essentia - Library for audio and music analysis, description and synthesis.
  • LibXtract - A simple, portable, lightweight library of audio feature extraction functions.
  • Marsyas - Music Analysis, Retrieval and Synthesis for Audio Signals.
  • muda - A library for augmenting annotated audio data.
  • madmom - Python audio and music signal processing library.

Computer Vision

  • OpenCV - Open Source Computer Vision Library.
  • scikit-image - Image Processing SciKit (Toolbox for SciPy).
  • imgaug - Image augmentation for machine learning experiments.
  • imgaug_extension - Additional augmentations for imgaug.
  • Augmentor - Image augmentation library in Python for machine learning.
  • albumentations - Fast image augmentation library and easy to use wrapper around other libraries.

Statistics

  • statsmodels - Statistical modeling and econometrics in Python.
  • stockstats - Supply a wrapper StockDataFrame based on the pandas.DataFrame with inline stock statistics/indicators support.
  • weightedcalcs - A pandas-based utility to calculate weighted means, medians, distributions, standard deviations, and more.
  • scikit-posthocs - Pairwise Multiple Comparisons Post-hoc Tests.
  • Alphalens - Performance analysis of predictive (alpha) stock factors.

Distributed Computing

  • Veles - Distributed machine learning platform.
  • Jubatus - Framework and Library for Distributed Online Machine Learning.
  • DMTK - Microsoft Distributed Machine Learning Toolkit.
  • PaddlePaddle - PArallel Distributed Deep LEarning.
  • Distributed - Distributed computation in Python.

Experimentation

  • Sacred - A tool to help you configure, organize, log and reproduce experiments.
  • Xcessiv - A web-based application for quick, scalable, and automated hyperparameter tuning and stacked ensembling.
  • Persimmon - A visual dataflow programming language for sklearn.

Evaluation

  • recmetrics - Library of useful metrics and plots for evaluating recommender systems.
  • Metrics - Machine learning evaluation metric.
  • AI Fairness 360 - Fairness metrics for datasets and ML models, explanations and algorithms to mitigate bias in datasets and models.

Computations

  • numpy - The fundamental package needed for scientific computing with Python.
  • bottleneck - Fast NumPy array functions written in C.
  • CuPy - NumPy-like API accelerated with CUDA.
  • scikit-tensor - Python library for multilinear algebra and tensor factorizations.
  • numdifftools - Solve automatic numerical differentiation problems in one or more variables.
  • quaternion - Add built-in support for quaternions to numpy.
  • adaptive - Tools for adaptive and parallel samping of mathematical functions.

Spatial Analysis

  • PySal - Python Spatial Analysis Library.

Quantum Computing

  • PennyLane - Quantum machine learning, automatic differentiation, and optimization of hybrid quantum-classical computations.
  • QML - A Python Toolkit for Quantum Machine Learning.

Conversion

  • sklearn-porter - Transpile trained scikit-learn estimators to C, Java, JavaScript and others.
  • ONNX - Open Neural Network Exchange.
  • MMdnn - A set of tools to help users inter-operate among different deep learning frameworks.

Contributing

Contributions are welcome! 😎
Read the contribution guideline.

License

This work is licensed under the Creative Commons Attribution 4.0 International License - CC BY 4.0

Deprecated Libs    Waiting Room