Instagram's experimental performance oriented greenfield implementation of Python.

Overview

Welcome to Skybison!

Skybison is experimental performance-oriented greenfield implementation of Python 3.8. It contains a number of performance optimizations, including: small objects; a moving GC; hidden classes; bytecode inline caching; type-specialized bytecode; an experimental template JIT.

Is this supported?

No.

Even though the project is no longer under active development internally, we've made Skybison publicly available in the hopes that people might find its history and internals interesting, or maybe even useful. It has not been polished or documented for anyone else's use.

We cannot commit to fixing external bug reports or reviewing pull requests. That said, if you have experience in dynamic language runtimes and have ideas to make Skybison faster; or if you work on CPython and want to use Skybison as inspiration for improvements in CPython, feel free to open a Pull Request.

Instagram is now developing Cinder.

How do I build it?

See INSTALL.md for an overview. However, this project was built in an internal development environment. We cannot guarantee that this project will build or run on any arbitrary environment.

What's here?

The project is structured in the following component directories:

./runtime

The core of the runtime resides here. In it is all of the core types, core modules, bytecode interpreter, and template JIT.

  • objects.[h|cpp] - All object types. See object-model.md.
  • *-builtins.[h|cpp] - Basic functionality of builtin types
  • *-module.[h|cpp] - Implementation of built-in modules and types correlated to a module.
  • interpreter.[h|cpp] - Implementation of all opcodes. See execution-model.md.
  • runtime/interpreter-gen-x64.cpp - Implementation of the assembly interpreter and template JIT.
  • heap.[h|cpp] - Class that manages managed objects. See garbage-collection.md.
  • runtime.[h|cpp] - The Runtime. This is where everything starts.

./library

The Python library code resides here. In it is a Skybison-specific set of library files and their tests. It is meant to be overlaid on top of the CPython standard library so that imports fall back on standard library code.

./capi and ./ext

Our C-API emulation layer resides here. Since any C code that interfaces with Python's C-API expects PyObject* not to move, and the runtime has a moving GC, we have built a layer of handle objects and C-API functions.

The structure inside this ./ext is designed to mimic the CPython repo. For example, CPython defines PyLong_FromLong in Objects/longobject.c. In Skybison, this function should be implemented in ext/Objects/longobject.cpp.

./third-party

We vendor several open-source projects for both testing and core functionality. They all reside here. Included in this list is a version of CPython, which we use for its Python and C library code.

./util

Development tools and helper scripts for building and debugging.

./benchmarks

Set of standard Python benchmarks with an accompanying test-runner.

./doc

This folder contains documentation.

Other notes

The repository has the name "pyro" sprinkled throughout the codebase. This is a holdover from an old internal codename.

License

Please see LICENSE.

You might also like...
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more
Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

Apache MXNet (incubating) for Deep Learning Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to m

Tensors and Dynamic neural networks in Python with strong GPU acceleration
Tensors and Dynamic neural networks in Python with strong GPU acceleration

PyTorch is a Python package that provides two high-level features: Tensor computation (like NumPy) with strong GPU acceleration Deep neural networks b

Implement yolov5 with Tensorrt C++ api, and integrate batchedNMSPlugin. A Python wrapper is also provided.
Implement yolov5 with Tensorrt C++ api, and integrate batchedNMSPlugin. A Python wrapper is also provided.

yolov5 Original codes from tensorrtx. I modified the yololayer and integrated batchedNMSPlugin. A yolov5s.wts is provided for fast demo. How to genera

Python to CLike language transpiler

Python to many CLike language transpiler Currently supports C++ and Rust. Preliminary support for Julia, Kotlin, Nim, Go and Dart. The main idea is th

Upload and changes to Python 0.9.1 release (from 1991!) so that it would compile

This is Python, an extensible interpreted programming language that combines remarkable power with very clear syntax. This is version 0.9 (the first

ParaMonte: Plain Powerful Parallel Monte Carlo and MCMC Library for Python, MATLAB, Fortran, C++, C.
ParaMonte: Plain Powerful Parallel Monte Carlo and MCMC Library for Python, MATLAB, Fortran, C++, C.

Overview | Installation | Dependencies | Parallelism | Examples | Acknowledgments | License | Authors ParaMonte: Plain Powerful Parallel Monte Carlo L

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library,  for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

eXtreme Gradient Boosting Community | Documentation | Resources | Contributors | Release Notes XGBoost is an optimized distributed gradient boosting l

An open source python library for automated feature engineering
An open source python library for automated feature engineering

"One of the holy grails of machine learning is to automate more and more of the feature engineering process." ― Pedro Domingos, A Few Useful Things to

In-situ data analyses and machine learning with OpenFOAM and Python

PythonFOAM: In-situ data analyses with OpenFOAM and Python Using Python modules for in-situ data analytics with OpenFOAM 8. NOTE that this is NOT PyFO

Comments
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 1
Owner
Meta Experimental
These are Meta projects that are not necessarily used in production but are being developed in the open nevertheless.
Meta Experimental
Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI

High-Performance-Computing-Experiments Experimental and Comparative Performance Measurements of High Performance Computing Based on OpenMP and MPI 实验结

Jiang Lu 1 Nov 27, 2021
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Dec 31, 2022
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Dec 23, 2022
An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support

An Out-of-the-Box TensorRT-based Framework for High Performance Inference with C++/Python Support

手写AI 1.5k Jan 5, 2023
[3DV 2021] DSP-SLAM: Object Oriented SLAM with Deep Shape Priors

DSP-SLAM Project Page | Video | Paper This repository contains code for DSP-SLAM, an object-oriented SLAM system that builds a rich and accurate joint

Jingwen Wang 368 Dec 29, 2022
APFS module for linux, with experimental write support (out-of-tree repository)

Apple File System ================= The Apple File System (APFS) is the copy-on-write filesystem currently used on all Apple devices. This module pro

APFS for Linux 260 Jan 4, 2023
BayesOpt: A toolbox for bayesian optimization, experimental design and stochastic bandits.

BayesOpt: A Bayesian optimization library BayesOpt is an efficient implementation of the Bayesian optimization methodology for nonlinear optimization,

Ruben Martinez-Cantin 341 Nov 29, 2022
Experimental OpenCL SPIR-V to OpenCL C translator

spirv2clc spirv2clc is an experimental OpenCL SPIR-V to OpenCL C translator currently targeting OpenCL 1.2 support. It can generate OpenCL C code equi

Kévin Petit 19 Oct 1, 2022
Python and C++ implementation of "MarkerPose: Robust real-time planar target tracking for accurate stereo pose estimation". Accepted at LXCV Workshop @ CVPR 2021.

MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimation This is a PyTorch and LibTorch implementation of MarkerPose: a

Jhacson Meza 47 Nov 18, 2022