Croquis: plot graphs 100x faster on Jupyter Notebook

Related tags

Graphics croquis
Overview

Croquis: plot graphs 100x faster on Jupyter Notebook

Croquis is a lightweight Python library for drawing interactive graphs really fast on Jupyter Notebook. It lets you effortlessly browse and examine much larger data than other similar libraries.

banner

Install croquis by running:

pip install croquis

👉 Please look at the tutorial for more example.


You might be wondering: there are already many mature and feature-rich plotting libraries — what's the appeal of a new, experimental library? Well, croquis is built with the following goals in mind:

  • Fast: Croquis contains a multi-threaded C++ plotting engine which automatically parallelizes graph generation, easily handling gigabytes of data.
  • Simple: Croquis provides a unified, simple API, regardless of data size. Whether your data contains ten data points or fifty million, you can make the same three function calls (or as many calls as you wish in a loop, if you prefer). The library will handle the rest.
  • Interactive: Since the C++ engine is alive as long as your figure is visible, at any moment you can zoom in, move around, or select a different subset of your data, and the figure will update accordingly.

As an example, here's hourly ground temperature data of 2020 from the world's weather stations, downloaded from NOAA website. The data set contains 127 million points.

croquis-example.mp4

The word "croquis" means a quick, sketchy drawing - it's from French croquis which simply means "sketch." (The final -s is silent: it's French, after all.)

Requirements

  • 64-bit Linux/Mac OS running on x86 with AVX2 instruction set support. (Intel: Haswell (2013) or later; AMD: Excavator (2015) or later.)
    • Windows support is under work.
    • Sorry, other architectures aren't supported yet.
  • Python 3.6 or later.
  • Jupyter Notebook.
  • A modern browser (if it can run Jupyter Notebook, it's probably fine).

How to install

pip install croquis

For building from the source, see DEVELOPMENT.md.

To test if it's working correctly, try this inside Jupyter Notebook:

# Paste into a Jupyter cell.

import croquis
import numpy as np

N = 1000000
X = np.random.normal(size=(N, 1))
Y = np.random.normal(size=(N, 1))
labels=['pt %d' % i for i in range(N)]

fig = croquis.plot()
fig.add(X, Y, marker_size=3, labels=labels)
fig.show()

It should generate a plot like this:

Gaussian distribution example

For documentation, see the tutorial and the (very short) reference.

By the way, this library is of course open source (MIT License) and totally free to use, but just in case you really liked it for some reason, the author could use a cup of coffee or two... :)

buy me a coffee

Limitations

Croquis is still experimental: as of version 0.1, we only support the absolute bare minimum functionality. In particular:

  • Only line plots are supported, nothing else: no bars, pie charts, heatmaps, etc.
  • All lines are solid: no dotted/dashed lines.
  • All markers are solid circles: no other shapes are currently supported.
  • No subplots: each Jupyter cell can contain only one graph.
  • Very few options to customize the plot. No titles, axis labels, or secondary axes.
  • No support for mobile browsers.
  • No dark mode.
  • As you can see, the UI is rather primitive.

If croquis seems useful to you, but some features are missing for your use case, then please feel free to file an issue. (Of course I can't guarantee anything, but it will be nice to know that someone's interested.)

FAQ

Is it really 100 times faster?

With large data, croquis can be several hundred times faster than other popular libraries. With very small data, there's less difference, as fixed-size overheads start to dominate.

Can we use it outside of Jupyter Notebook?

No, croquis is currently tied to Jupyter's message passing architecture, and all computation is done in the backend, so it needs an active Jupyter Python kernel.

How does it work?

Unlike most other similar libraries, croquis works by running a C++ "tile server," which computes fixed-sized "tiles" which is then sent back to the browser. If you have used Google Maps, the idea should be familiar. This has an important advantage:

  • The browser only has to know about tiles. Hence, the size of the data the browser needs to know is independent of the data set size.

As a result, the browser stays lean and "snappy" even with massive data. (As explained in the reference, we support copy_data=False option that even eliminates data copies altogether.)

Moreover, unlike the browser's single-threaded javascript code, the C++-based tile server can draw multiple tiles in parallel, which allows even more speedup.

(On the other hand, there are drawbacks - we have to basically re-implement every graph drawing algorithm inside this tile server, not being able to use any javascript API, except for very trivial things like coordinate grids.)

You might also like...
👾 Jupyter Notebook + Space Invaders!?
👾 Jupyter Notebook + Space Invaders!?

Train Invaders Jupyter Notebook + Space Invaders!? Why? • Getting started • How it works • FAQ • Drawbacks • Contribute • Thanks to • You may also lik

A faster,smaller,Address Sanitizer,200X Faster,95% Smaller.

FirASAN(Fir Address Sanitizer) A faster,smaller,Address Sanitizer 200X Faster,95% Smaller. FirASAN 结论: 内存消耗 CPU消耗 ASAN原版 +100-150% +200%以上 FirASAN +5%

This is a C port of Stefan Hirschmann's NoteBook FanControl.

It provides the same utilities with the same interfaces as the original NBFC, although the implementation differs.

Chia Plot Graph
Chia Plot Graph

Chia Plot Graph Monitor for Chia Plotting Introduction The chiaplotgraph tool will graph Chia Plotting activity in a linux terminal. Use a 24-bit colo

Plot dynamic 3d functions z = f(x, y, t)
Plot dynamic 3d functions z = f(x, y, t)

A tool for plotting 3D functions Controls Button Description Arrows Rotate view F Enable/Disable filling surface faces G Enable/Disable showing coordi

vplot is a wrapper for GNU Plot (gnuplot_i)
vplot is a wrapper for GNU Plot (gnuplot_i)

vplot vplot is a wrapper for GNU Plot (gnuplot_i). The source of gnuplot_i I have downloaded from this link. Files listed on gnuplot_i are taken from

GnuPlotScripting - A simple C++17 lib that helps you to quickly plot your data with GnuPlot
GnuPlotScripting - A simple C++17 lib that helps you to quickly plot your data with GnuPlot

Easy Creation of GnuPlot Scripts from C++ Table of contents What is it? News Contributors Installation Examples Plot and fit data Ascii matrix data (N

An improved plot widget for Dear ImGui, aimed at displaying audio data
An improved plot widget for Dear ImGui, aimed at displaying audio data

imgui-plot An improved plot widget for Dear ImGui, aimed at displaying audio data TOC Screenshots Rationale Usage Installation FAQ Screenshots Display

Alpha Plot is a free application for Scientific Data Analysis and Visualization for Windows, Linux and Mac OS X
Alpha Plot is a free application for Scientific Data Analysis and Visualization for Windows, Linux and Mac OS X

Alpha Plot is a free application for Scientific Data Analysis and Visualization for Windows, Linux and Mac OS X (probably BSD also). Web Link Website

Plot airfield from a file containing the list of airfield of Germany and their position (latitude/longitude)
Plot airfield from a file containing the list of airfield of Germany and their position (latitude/longitude)

Plot aerodromes from a file containing the list of aerodromes of Germany and their position (latitude/longitude)

PlotFS is a fuse filesystem for efficient storage of Chia plot files.

PlotFS PlotFS is a fuse filesystem for efficient storage of Chia plot files. PlotFS is not a traditional filesystem. It is mounted read only for farmi

A library for enabling task-based multi-threading. It allows execution of task graphs with arbitrary dependencies.

Fiber Tasking Lib This is a library for enabling task-based multi-threading. It allows execution of task graphs with arbitrary dependencies. Dependenc

Tool to check C++ #include dependencies (dependency graphs created in .dot format)
Tool to check C++ #include dependencies (dependency graphs created in .dot format)

Read Me for Dependency Checker Copyright (C) 2012-2017, TomTom International BV. All rights reserved. The tool cpp-dependencies creates #include depen

Directed Acyclic Graph Execution Engine (DAGEE) is a C++ library that enables programmers to express computation and data movement, as task graphs that are scheduled concurrently and asynchronously on both CPUs and GPUs.

Directed Acyclic Graph Execution Engine (DAGEE) is a C++ library that enables programmers to express computation and data movement, as tasks in a graph structure, where edges represent task dependencies

LLVM meets Code Property Graphs

llvm2cpg llvm2cpg is a tool that converts LLVM Bitcode into Code Property Graph (CPG). The CPG can be further analyzed via Joern or Ocular. To get sta

🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

Graphs the disk IO in a linux terminal.
Graphs the disk IO in a linux terminal.

diskgraph Monitor for disk IO Introduction The diskgraph tool will graph disk IO under linux, in a terminal. Examples: $ ./diskgraph /dev/nvme0n1 $ ./

Graphs the activity of a chia harvester in a linux terminal.
Graphs the activity of a chia harvester in a linux terminal.

Chia Harvest Graph Monitor for Chia Harvesting Introduction The chiaharvestgraph tool will graph Chia Harvesting activity in a linux terminal. Use a 2

Graphs the progress of block height in your Chia Full Node.
Graphs the progress of block height in your Chia Full Node.

Chia Height Graph Monitor for Chia Block Height Introduction The chiaheightgraph tool will graph Chia Block Height changes in a linux terminal. Use a

Owner
Yongjik Kim
Yongjik Kim
vplot is a wrapper for GNU Plot (gnuplot_i)

vplot vplot is a wrapper for GNU Plot (gnuplot_i). The source of gnuplot_i I have downloaded from this link. Files listed on gnuplot_i are taken from

Erdet Nasufi 13 Aug 15, 2022
Alpha Plot is a free application for Scientific Data Analysis and Visualization for Windows, Linux and Mac OS X

Alpha Plot is a free application for Scientific Data Analysis and Visualization for Windows, Linux and Mac OS X (probably BSD also). Web Link Website

Arun Narayanankutty 169 Nov 20, 2022
Sketch-Based Streaming Anomaly Detection in Dynamic Graphs

AnoGraph Implementation of Sketch-Based Streaming Anomaly Detection in Dynamic Graphs. Siddharth Bhatia, Mohit Wadhwa, Philip S. Yu, Bryan Hooi Existi

Stream-AD 54 Oct 6, 2022
A library in Javascript to create graphs in the browser similar to Unreal Blueprints.

A graph node engine and editor written in Javascript similar to PD or UDK Blueprints, comes with its own editor in HTML5 Canvas2D. The engine can run client side or server side using Node. It allows to export graphs as JSONs to be included in applications independently.

Javi Agenjo 3.2k Nov 29, 2022
Selfies but in C++. Celfies is for HPC. Robust representation of semantically constrained graphs, in particular for molecules in chemistry.

celfies Selfies but in C++. Why? Because RDKit in C++ and other speed reasons. Follow the originators of the ideas & code @MarioKrenn, @AlstonLo, @Sey

sevenTM 3 Nov 24, 2021
This Project Implement an interactive camera for 3D model using Quaternion. It have some advantages over eulerian camera like no gimbal lock and faster to compute.

Quaternion-Camera This Project Implement an interactive camera for 3D model using Quaternion. It have some advantages over eulerian camera like no gim

Phan Sang 8 Nov 10, 2022
Create a Jupyter Kernel for 8085 Microprocessor assembly language that can interpret assembly-level programs right from the Jupyter notebook.

Create a Jupyter Kernel for 8085 Microprocessor assembly language that can interpret assembly-level programs right from the Jupyter notebook.

Sarita Singh 4 Oct 5, 2022
A drop-in replacement for std::list with 293% faster insertion, 57% faster erasure, 17% faster iteration and 77% faster sorting on average. 20-24% speed increase in use-case testing.

plf_list A drop-in replacement for std::list with (on average): 293% faster insertion 57% faster erasure 17% faster iteration 77% faster sorting 70% f

Matt Bentley 117 Nov 8, 2022
A drop-in replacement for std::list with 293% faster insertion, 57% faster erasure, 17% faster iteration and 77% faster sorting on average. 20-24% speed increase in use-case testing.

plf::list A drop-in replacement for std::list with (on average): 293% faster insertion 57% faster erasure 17% faster iteration 77% faster sorting 70%

Matt Bentley 117 Nov 8, 2022
xeus-wren is a Jupyter kernel for wren based on the native implementation of the Jupyter protocol xeus.

xeus-wren is a Jupyter kernel for wren based on the native implementation of the Jupyter protocol xeus. Installation xeus-wren has not been packaged f

Thorsten Beier 4 Mar 9, 2022