Scikit-learn is a free and open-source software library with a number of supervised and unsupervised algorithms for machine learning.
Scikit-learn is built on NumPy, and makes use of other libraries like Pandas, Scipy and Matplotlib.
Scikit-learn is a powerful machine learning library for Python that uses NumPy extensively to deliver fast linear algebra and array operations. Some of its core algorithms are written in Cython, which improves performance.
It's designed to make the process as simple as possible, with a minimal learning curve.
Features of Scikit-Learn
Scikit-learn offers a range of supervised and unsupervised learning algorithms for
- Dimensionality reduction
- Model Selection
Scikit-learn has been designed to be easy-to-use, accessible to everyone and reusable in various contexts.
Uses of Scikit-Learn
- Predictive and statistical modelling.
- Data analysis
- It is used to create, investigate, and evaluate models.
- Numerical optimization
- Data mining
pip install scikit-learn
TensorFlow is a platform that allows machine learning to be done openly. It provides a wide range of flexible tools, libraries, and community resources that allow researchers to push the current state of ML research and allows developers to quickly build and deploy ML-powered apps.
pip install tensorflow
PyTorch is an open-source end-to-end machine learning framework, based on the Torch library. It was developed by Facebook's AI Research lab and it is free and distributed under BSD license. PyTorch is implemented in Python and C++.
The PyTorch framework includes all the standard features from Torch, such as nn modules, autograd, and Tensor computation graphs, while also providing Pythonic interfaces and features such as callbacks and Tensor decorators.
PyTorch's user-friendly front-end, distributed training, and ecosystem of tools and libraries allow quick, scalable experimentation and efficient production.
It provides a high-level neural scripting language that is used to define deep learning models and train them. The deep neural networks that we build with PyTorch are mostly forward-only and do not have cycles.
The famous Tesla Autopilot software was built using Pytorch. Uber's Pyro, HuggingFace's Transformers, PyTorch Lightning, and Catalyst are just a few examples of deep learning applications designed with PyTorch..
1. Tensor computing with strong acceleration via GPU
2. Production Ready
3. Fast and Lean
4. Distributed Training
5. Robust Ecosystem
6. Deep neural networks built on a type-based automatic differentiation system.
7. Cloud Support
Uses of PyTorch
- Computer Vision
- Natural Language processing
If you already have python installed, you can install PyTorch via package managers like pip or conda. To install via pip, please make sure you have NumPy already installed then run:
pip install torch
To install using conda, just run:
conda install -c pytorch pytorch
For more detailed installation guidelines, visit Getting started with PyTorch.
Theano is a python library and an optimizing compiler for manipulating and evaluating mathematical expressions involving multi-dimensional arrays. It is built on Numpy and named after a Greek mathematician and philosopher.
In Theano, computations are expressed using a syntax very similar to NumPy's and compiled to run efficiently on either CPU or GPU architectures.
Theano was developed by the Montreal Institute for Learning Algorithms (MILA), the University of Montreal in 2007 and It has been powering large-scale computationally intensive scientific research since then. It is one of the most mature deep learning frameworks in existence and is open-sourced under the BSD license for all to use.
Developed specifically to handle the types of computation required for large neural network algorithms used in Deep Learning.
Features of Theano
1. It has tight integration with Numpy.
2. Performs efficient symbolic differentiation.
3. Fast to write and execute.
4. Parallelism on GPU. (performs computation much faster on a GPU)
5. Includes tools for extensive unit testing and self-verification.
6. Evaluates expressions faster by the dynamic generation of C code.
7. Stability optimizations
Uses of Theano
Here are some uses of theano:
1. Theano is used to implement deep learning models.
2. Used for regular mathematical computation and research.
You can easily install Theano by running
pip install Theano
Django is a free and open-source python web framework. It follows the model-template-view (MTV) pattern and it's designed for rapid development and clean, pragmatic design. It has been designed to take care of much of the hassle of creating a web application particularly, complex, database-driven websites.
Django’s key distinguishing feature is its "batteries-included" philosophy. The framework includes many pieces of functionality that are almost always needed in Web development so that developers don't have to choose between reinventing the wheel and spending time on the framework's basic features.
It started in 2003 when a small team of developers decided to build a new Web framework that would advance Python as an alternative to PHP, which was then dominant.
Since then, it has grown into a powerful and mature Web framework that has been powering web projects of all shapes and sizes. We've seen people from all over the world take Django into production in environments ranging from small startups to large-scale e-commerce systems.
Django has become a very popular choice among beginners and web developers because it is powerful, simple and versatile such that you can easily get a web app up and running.
Features of Django
1. Rapid Development
2. Very Secure
Django is strengthened by its built-in security features which handle most of the security issues pertaining to web applications. It also comes with powerful encryption libraries which help to protect user passwords and sensitive data.
3. Scalable and Reliable
It has all you need to scale as web applications grow in popularity and number of users.
4. Well Documented
Django has amazing documentation which makes it super easy to use. You can learn a lot about Django by looking at the documentation and the tutorials provided in it.
5. Stable and Large Community
Django is one of the most popular Web frameworks in the world, with a vibrant community of developers who contribute to its open-source codebase.
1. Django is mainly used for creating web applications.
2. Used with other libraries to create APIs.
You can install Django by running:
pip install Django
or via conda:
conda install -c anaconda django
Websites or Organizations using Django
Here are a few of the large organizations using Django on their sites or in various projects:
- Knight Foundation
- MacArthur Foundation
- National Geographic
- Open Knowledge Foundation
- Open Stack
Django is used by countless organizations and companies, large and small, around the world.
Flask is a lightweight, yet powerful micro web framework written in Python. It is based on the Pocoo projects, Werkzeug and Jinja2 and It's very flexible and can be used to build any type of website from simple to complex. Flask makes use of extensions that can be used to add features and extra functionality to your application.
It's designed to make getting started quick and easy, with the ability to scale as your application grows.
Armin Ronacher of Pocoo, an international community of Python enthusiasts established in 2004, developed Flask. According to Ronacher, the concept started out as an April Fool's joke that grew in popularity enough to be turned into a serious framework. It started out as a simple wrapper for Werkzeug and Jinja and has since grown into one of the most popular Python web application frameworks.
Flask and the libraries it uses are now maintained and supported by Armin's Pallets organization also known as Pallets Projects.
Flask is used to create everything from a small applications to a large web apps with multiple users and complex system requirements
Some features of the flask framework include:
1. Jinja templating.
2. Comes with in-built Debugging and development server.
3. Compatible with many databases.
4. Support for unit testing.
5. Highly extensible.
6. Compatible with Google App Engine.
7. Unicode-based and WSGI complaint.
8. Supports extensions to enhance features and functionality.
Who Uses Flask
Flask is a popular choice for Python projects of all shapes and sizes and is used by a wide range of companies and organizations such as Pinterest and LinkedIn.
pip install flask
A Basic Flask App
A minimal Flask application looks something like this
from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): return 'Hello, World!'
Visit the Quickstart Guide to understand what this code does and how to run it.
Bottle is a super-simple, fast and lightweight WSGI micro-framework for small web applications. Other than the Python Standard Library, it is distributed as a single file module with no dependencies.
Bottle is built on top of the simplicity and minimalism of the web. It offers a fast and lightweight structure to build simple web applications and services.
Bottle was developed by Marcel Hellkamp in 2009 and it is intended to be fast, convenient, and lightweight, and to reduce the complexity of developing web applications.
Bottle is not as full-featured as other frameworks, but then it doesn’t have as big a learning curve either, also does not require any kind of configuration, and is perfect for quickly hacking together a simple app that requires a database.
It tries to make the best trade-off between simplicity, efficiency, and capability.
Bottle includes the following features:
- Built-in template engine.
- Built-in HTTP development server (WSGI server)
- Support for JSON client data
- Plugins for popular databases and other features
Stable releases of bottle.py are available on the Python Package Index and can be installed via pip (recommended)
pip install bottle
To install the bottle framework with conda run:
conda install -c conda-forge bottle
This tutorial assumes you already have Bottle installed. Let's begin with a simple "Hello World" example:
from bottle import route, run @route('/hello') def hello(): return "Hello World!" run(host='localhost', port=8080, debug=True)
This is how simple a bottle app can be. Just run the script and open http://localhost:8080/hello and you will see “Hello World!” in your browser.
Web2py is an open-source full-stack web framework written in Python. It’s designed for the rapid development of database-driven web applications. Web2py is scalable, comes with a web-based IDE for creating and managing applications. It is easy to install and configure and supports multiple protocols.
pip install web2py
Pyramid is an open-source WSGI web framework implemented in python. It is based on the Model-View-Controller (MVC) architectural pattern and makes it easy for python developers to create web apps.
Pyramid makes it easier to create complex software as your application grows.
pip install pyramid
This is a basic example of a pyramid web app:
from wsgiref.simple_server import make_server from pyramid.config import Configurator from pyramid.response import Response def hello_world(request): return Response('Hello World!') if __name__ == '__main__': with Configurator() as config: config.add_route('hello', '/') config.add_view(hello_world, route_name='hello') app = config.make_wsgi_app() server = make_server('0.0.0.0', 6543, app) server.serve_forever()
FastAPI is a modern and high-performance Python web framework for building APIs. It makes use of python's standard type hints and was designed to simplify the development experience for developers so you can write simple code and build production-ready APIs using best practices.
- It is fast to run
- Fast and easy to code
- Fewer number of bugs when coding
- Very Intuitive
Pygame is an open-source and cross-platform set of Python modules for creating video games. It also includes sound libraries and computer graphics designed for use with the Python programming language.
pygame serves as a python wrapper for the SDL library, (Simple DirectMedia Layer). The SDL provides cross-platform access to your system's underlying multimedia hardware components, such as sound, video, mouse, keyboard, and joystick.
To install pygame on your system, use the pip command:
pip install pygame
Pyglet is a powerful and easy-to-use library that allows you to develop games and other visually rich applications. It is written in python and provides a friendly pythonic API to allow users to create games, simulations, and other graphical applications without having to worry about the underlying windowing toolkit or operating system. Pyglet can be used on Windows, Mac OS and Linux.
pip install pyglet
Panda3D is a game engine. It provides a framework for 3D rendering, game development and programming in Python and C++.
Panda3D was designed for commercial game development. This engine should emphasize four key areas: power and speed, completeness and error tolerance.
Panda3D is free and open-source, making it available for commercial projects.
Installation with pip
pip install panda3d
NumPy is a python library for high-performance, numerical computing. It is the fundamental package needed for scientific computing with python and performing advanced data science and analytics using tools like pandas, matplotlib, scikit-learn, and many others.
It extends Python by adding support for powerful n-dimensional arrays and matrices along with a large collection of high-level mathematical functions to operate on these arrays. It includes optimized code for computationally expensive linear algebra operations, basic linear algebra primitives such as matrix multiplication, inversion, determinants, eigenvalues and singular values; interpolation; minimization problems including least-squares problems; polynomial approximations, etc.
pandas is an open-source python library designed to make it easier to work with data. It provides fast, flexible and expressive data structures for data analysis and manipulation.
It has tools and operations for working with relational and labelled data, numerical tables, time-series, as well as statistical computation and graphics.
pip install pandas
Scrapy is a fast, powerful and open-source web scraping and web crawling framework written in python. It is used to crawl websites and extract structured data from their pages.
It can be used for a wide range of purposes, from data mining to monitoring, automated testing and even building web spiders.
pip install scrapy
Beautiful Soup is a Python library for parsing and pulling data out of HTML and XML files/structures. In other words, It is a popular python library used for web scraping. It can be used to either extract or access specific HTML tags and attributes, like the title of a webpage or find all links on a page and follow them sequentially.
Its different features make it easier to address the challenges we might face when we scrape websites and parse the data we find there. Beautiful Soup comes with excellent documentation and is also a very popular choice among python developers for web scraping.
pip install beautifulsoup4
Beautiful Soup Resources
It provides a collection of language-specific bindings called WebDrivers to drive a browser. With selenium's WebDriver, you can create robust, browser-based regression automation suites and tests, scale and distribute scripts across many environments.
Selenium excels at cross-platform compatibility and web-based functional testing. Sometimes, selenium is also used for web scraping.
pip install selenium
Requests is one of the most popular python libraries due to its simplicity and ease of use. It is an elegant and simple HTTP library used for making all kinds of HTTP requests in python.
The requests library is an easy-to-use library with a lot of features ranging from passing parameters in URLs to sending custom headers and SSL Verification.
pip install requests
ScientificPython is an open-source python library made up of a collection of Python modules that are useful for scientific computing.
Features of ScientificPython
1. Advanced mathematical tools for differentiation, numerical integration and many more
2. It has support for parallel computing.
3. It has support for several input/output interfaces.
pip install ScientificPython
Statsmodels is a Python module that allows you to explore statistical models and make predictions using data. It provides classes for linear regressions, generalized linear models, time series analysis, classical parametric and non-parametric tests, clustering, smoothing, decomposition and for conducting other statistical tests, and statistical data exploration.
pip install statsmodels
CuPy is a NumPy-compatible array library accelerated by CUDA. CUDA, short for Compute Unified Device Architecture is an architecture and programming model invented by NVIDIA that enables dramatic increases in computing performance by harnessing the power of the graphics processing unit (GPU) and parallel computing.
CuPy allows for the execution of NumPy-style computations on CUDA-capable GPUs, with a wide range of data transfer mechanisms to make the most of the available hardware.
NumPy is a library for managing arrays of numbers efficiently in Python. It is a great library for managing arrays of numbers, but it doesn’t have built-in support for CUDA.
CuPy library is designed to take advantage of the computational resources of CUDA-enabled GPUs.
In short, CuPy is a python package that provides fast and efficient operations on NumPy arrays accelerated by CUDA.
- High performance with CUDA
- Compatible with NumPy
- Easy to install
- Easy custom kernel implementation.
- For enhanced array computing
PyRobot is a lightweight, high-level interface that provides hardware-independent APIs for robotic manipulation and navigation.
It is a Python tool for benchmarking and executing robot learning experiments. The purpose of this project is to easily isolate low-level controls for individual robots from high-level motion generation and learning. Using PyRobot allows you to run robots without having to deal with robot-specific software, as well as perform better comparisons.
pip install pyrobot
Open Source Computer Vision (OpenCV) is an open-source computer programming library designed to assist computer vision applications. It offers hundreds of functions for the acquisition, analysis, and manipulation of visual data and can alleviate some of the difficulties that programmers experience while designing computer vision applications.
pip install opencv-python
Kivy is a cross-platform C++/Python software framework for the development of mobile apps and other multitouch application software with a natural user interface (NUI). It runs on Linux, Windows, Mac OS X, Android, iOS and even Raspberry pi.
Kivy is a free and open-source software, released under the MIT License. It is designed to work with a variety of input devices and provides a fully-featured, user-friendly, and customizable interface builder, along with a long list of supported UI controls.
It allows you to write the same app for Android, iOS, and Windows Phone with a single codebase.
- Multitouch support.
- GPU accelerated
- Open-source and business friendly.
- Stable and well documented
BeeWare is an open-source Python framework for building cross-platform applications, such as desktop applications and mobile apps. It is basically a collection of tools and libraries for building and distributing native applications in Python. It consists of a number of components that help to ease the development of applications that function identically on different platforms and devices. (One Codebase. Multiple Apps)
Applications written with BeeWare can easily be packaged for a number of different platforms.
Uses of BeeWare
- Used to build cross-platform apps with a rich, native user interface.
- It contains tools for GUI development for desktop, mobile and web platforms.
- It contains a set of tools for packaging a Python project for multiple platforms.
pip install beeware
The matplotlib is a library used for creating static, animated and interactive visualizations in Python. It allows users to create plots, histograms, power spectrums, bar charts, scatter plots, and many more graphical representations of data.
Matplotlib offers a multitude of options for visualization which makes easy things easy and hard things possible.
pip install matplotlib
To learn more, checkout the resources below.
Python-tesseract is a Python-based optical character recognition (OCR) tool. In other words, it will recognize and "read" text encoded in images.
Python-tesseract is a library that wraps Google's Tesseract-OCR Engine. It can also be used as a standalone tesseract invocation script because it can read any image types supported by the Pillow and Leptonica imaging libraries, such as jpeg, png, gif, bmp, tiff, and others. Furthermore, when used as a script, Python-tesseract will print the detected text rather of writing it to a file.
pip install pytesseract
Bokeh is a Python data visualization package that generates high-performance interactive charts and graphs. It allows for the beautiful and concise creation of adaptable images, as well as high-performance interactivity across big or streaming datasets.
pip install bokeh
A multinational team of developers created Biopython, a suite of freely available tools for biological computation written in Python.
It is a widespread collaborative effort to create Python modules and apps to meet the needs of present and future bioinformatics work. The source code is distributed under the Biopython License, which is incredibly permissive and compatible with nearly every license in the world.
pip install biopython
PySB is a Python modeling framework for creating, simulating and analyzing biological models with a high-level, action-oriented vocabulary that encourages clarity, extensibility, and reusability. In other words, PySB is a Python library that implements a variety of mathematical models of biochemical systems in living cells. It also includes tools to generate, visualize, and explore the models.
PySB models are mathematically similar to the ordinary differential equations often used to model natural phenomena. In particular, they are based on the equations of mass action kinetics and conservation of mass, energy, charge, and volume.
PySB also works with standard scientific Python libraries including NumPy, SciPy, and SymPy to simulate and analyze models.
Members of the Lopez Lab at Vanderbilt University and the Sorger Lab at Harvard Medical School are the primary developers and maintainers of PySB:
Here are a few features of the PySB:
- Free and open source
- Optimized simulation
- Resuble modular specification
- Integration with scientific python
Though PySB is primarily used for modeling biological models, It also interoperates other standard scientific Python libraries for
- Compatibility with SBML(Systems Biology Markup Language) tools
- Efficient array and matrix operations
- Developing scientific algorithms, e.g., ODE integration, statistics, and optimization
- Symbolic manipulation of mathematical expressions
- Layout and rendering of node‐edge graphs
- General‐purpose scientific computing
For detailed instructions on installation processes please visit http://pysb.org/download. Anyways, the installation is very straightforward with conda - run the the following in a terminal:
conda install -c alubbock pysb
This a code example from pysb.org.
from pysb import * from pysb.simulator import ScipyOdeSimulator from pylab import linspace, plot, xlabel, ylabel, show # A simple model with a reversible binding rule Model() # Declare the monomers Monomer('L', ['s']) Monomer('R', ['s']) # Declare the parameters Parameter('L_0', 100) Parameter('R_0', 200) Parameter('kf', 1e-3) Parameter('kr', 1e-3) # Declare the initial conditions Initial(L(s=None), L_0) Initial(R(s=None), R_0) # Declare the binding rule Rule('L_binds_R', L(s=None) + R(s=None) | L(s=1) % R(s=1), kf, kr) # Observe the complex Observable('LR', L(s=1) % R(s=1)) if __name__ == '__main__': # Simulate the model through 40 seconds time = linspace(0, 40, 100) print "Simulating..." sim_result = ScipyOdeSimulator(model, time).run() # Plot the trajectory of LR plot(time, sim_result.observables['LR']) xlabel('Time (seconds)') ylabel('Amount of LR') show()
Keywords from google
programming biological models in python using pysb
IPython short for "Interactive Python" is a command shell for interactive computing originally developed for the python programming language. It provides a rich toolkit to help you make the most out of using Python interactively.
Apart from a powerful interactive shell, Ipython also comes with a kernel for Jupyter and support for interactive data visualization.
pip install ipython
The Jupyter Notebook is a free and open-source web application that lets you create and share documents with live code, equations, visualizations, and narrative text. Data cleansing and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and many other applications are possible.
It is very popular among data science and machine learning practitioners.
pip install notebook
Jupyter Notebook Resources
Impacket is a Python module that allows you to work with network protocols. Currently, the library focuses on supporting low-level socket programming, TCP/IP, and a variety of higher-level protocols. It is a Python module that allows programmers to easily construct and decode network packets.
Many hackers and penetration testers utilize Impacket to create unique network-based intrusion attacks such as man in the middle and session hijacking. It was created as an internal tool to aid in testing, but it has evolved into a strong hacker's tool that can be used against networks.
pip install impacket
cryptography is a python package that provides cryptographic primitives and cryptographic recipes to developers. It is intended to become your "cryptographic standard library".
Cryptography includes both high-level recipes and low-level interfaces to common cryptographic algorithms such as message digests, symmetric ciphers and key derivation functions.
Python-nmap is a Python package that aids in the use of the nmap port scanner. It enables simple manipulation of nmap scan results and will be an ideal tool for system administrators looking to automate scanning tasks and reports. It also supports the output of nmap scripts.
pip install python-nmap
Shapely is a python package for manipulation and analysis of planar geometric objects. It provides a set of classes to represent different geometric objects like points, lines, or polygons. It also provides methods to manipulate these objects, such as computing the intersection or union of objects.
It is also designed to interoperate with NumPy and to provide efficient operations on geometric objects. Shapely includes a powerful set of methods for operating on point sets, including union, intersection, difference, and containment tests. Shapely also implements the following geometric primitives: Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, and GeometryCollection.
This BSD licensed package is built on the GEOS library and is used in GIS and CAD applications, but it can be used in any application which needs to manipulate and analyze geometry.
Uses of the shapely python package
Take a look at some of the functionality the shapely package provides:
1. Manipulating geometric objects in many different ways
2. Creating geometric objects from scratch
3. Converting between different geometries
4. Geoprocessing (e.g. buffer, union, intersection, difference)
5. Plotting of geometry
6. Querying geometries with OGC standards
Shapely is provided by popular Python distributions like Canopy, Python Package Index and Anaconda therefore can be installed via package management tools like
conda. If you use the Conda package manager to install Shapely, be sure to use the conda-forge channel.
To install shape using pip
pip install shapely
To install the shapely package with conda run following:
conda install -c conda-forge shapely
Sympy is an open-source Python library for symbolic mathematics. It has been developed to provide a more robust alternative to Python’s built-in math library. It provides several features that are missing from the default library, such as functions to compute limits, find derivatives or find integrals, among others.
Sympy is written entirely in python and it aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible.
This toolkit is designed to make it easier for engineers, scientists, and programmers to work with quantum computing, without having to worry about the underlying technology. Qiskit’s goal is to make quantum computing accessible to a broader range of people.
The world of quantum computing might seem a little intimidating to someone who hasn’t had any experience with it. But, the Qiskit Toolkit is a great place to get started – and it’s open-source so anyone can use it.
In this post, I’ll give you an introduction to quantum computing and introduce you to the Qiskit Toolkit in detail.
Bitcoinlib is a python cryptocurrency library, it is used to create and manage wallets for bitcoin, litecoin, dash, etc. This library can be used at a high level to build and control wallets from the command line, or at a low level to create your own custom transactions, files, keys, or wallets.
It supports most of the features that are required for building applications using Bitcoin, Bitcoin transactions, bitcoin addresses, private keys and multiple other cryptocurrencies based on the same protocol. It uses pure python implementation; so many applicable platforms are available to use (cross-platform).
Using simple and straightforward Python code, you can create and manage transactions, addresses/keys, wallets, mnemonic password phrases, and blocks.
To know more about what this library can do for you or your organization please refer to the documentation page. The library comes with a fully functioning wallet, complete with multiple signatures, multiple currencies, and multiple accounts.
spaCy is an open-source software library written in Python and Cython for advanced natural language processing. The library was created by Matthew Honnibal and Ines Montani, the founders of the software company Explosion, and is available under the MIT license.
Unlike NLTK, which is widely used in education and research, spaCy focuses on providing software for industrial use.
pip install spacy
Gensim is an open-source library that uses modern statistical machine learning to perform unsupervised topic modeling and natural language processing.
For performance, Gensim is written in Python and Cython. Gensim is intended to handle large text collections using data streaming and incremental online algorithms, which distinguishes it from most other machine learning software packages that are only intended for in-memory processing.
pip install gensim
Cirq is a Python library for writing, manipulating, and optimizing quantum circuits and running them against quantum computers and simulators. It's available on PyPI and GitHub.
Cirq is a Python library that allows you to write, manipulate, and optimize quantum circuits which you can then run them on quantum computers or quantum simulators. Cirq provides useful abstraction for dealing with noisy, intermediate-scale quantum computers.