In-situ data analyses and machine learning with OpenFOAM and Python

Overview

PythonFOAM:

In-situ data analyses with OpenFOAM and Python

Using Python modules for in-situ data analytics with OpenFOAM 8. NOTE that this is NOT PyFOAM which is an automation tool for running OpenFOAM cases. What you see in this repository, is OpenFOAM calling Python functions and classes for in-situ data analytics. You may offload some portion of your compute task to Python for a variety of reasons (chiefly data-driven tasks using the Python ML ecosystem and quick prototyping of algorithms).

Contents

  1. PODFoam/: A pimpleFoam solver with in-situ collection of snapshot data for a streaming singular value decomposition. Python bindings are used to utilize a Python Streaming-SVD class object from OpenFOAM.

  2. APMOSFoam/: A pimpleFoam solver with in-situ collection of snapshot data for a parallelized singular value decomposition. While the previous example performs the SVD on data only on one rank - this solver performs a global, but distributed, SVD. However, SVD updates are not streaming.

To compile and run

Use standard procedure to compile a new solver in OpenFOAM, i.e., use wmake from within PODFoam/. To run cases, it is assumed that you have a Python virtual environment that is linked to during compile and run time. The relevant lines are

-I/gpfs/fs1/home/rmaulik/OF8/OFPYENV/include/python3.6m/ \
-I/gpfs/fs1/home/rmaulik/OF8/OFPYENV/lib/python3.6/site-packages/numpy/core/include \

within EXE_INC and

-L/gpfs/fs1/home/software/spack-0.10.1/opt/spack/linux-centos7-x86_64/gcc-7.3.0/python-3.6.7-7eq7ubsfsxwib5oi7yk5ek7edv3cr7vt/lib \
-lpython3.6m

within EXE_LIBS. Replace these with the include/lib paths to your personal Python environments. The Python module within Run_Case/ directories require the use of numpy, matplotlib, and tensorflow so ensure that your environment has these installed. The best way to obtain these is to pip install tensorflow==2.1 which will automatically find the right numpy dependency and then pip install matplotlib to obtain plot capability. You will also need to install mpi4py which you can using pip install mpi4py.

Points of contact for further assistance - Romit Maulik ([email protected]). This work was performed by using the resources of the Argonne Leadership Computing Facility, a U.S. Department of Energy (Office of Science) user facility at Argonne National Laboratory, Lemont, IL, USA.

LICENSE

MIT

You might also like...
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

SHARK - High Performance Machine Learning for CPUs, GPUs, Accelerators and Heterogeneous Clusters

SHARK Communication Channels GitHub issues: Feature requests, bugs etc Nod.ai SHARK Discord server: Real time discussions with the nod.ai team and oth

An Open Source Machine Learning Framework for Everyone
An Open Source Machine Learning Framework for Everyone

Documentation TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, a

Distributed machine learning platform

Veles Distributed platform for rapid Deep learning application development Consists of: Platform - https://github.com/Samsung/veles Znicz Plugin - Neu

An open source machine learning library for performing regression tasks using RVM technique.

Introduction neonrvm is an open source machine learning library for performing regression tasks using RVM technique. It is written in C programming la

Feature Store for Machine Learning
Feature Store for Machine Learning

Overview Feast (Feature Store) is an operational data system for managing and serving machine learning features to models in production. Please see ou

Machine Learning Platform for Kubernetes
Machine Learning Platform for Kubernetes

Reproduce, Automate, Scale your data science. Welcome to Polyaxon, a platform for building, training, and monitoring large scale deep learning applica

CNStream is a streaming framework for building Cambricon  machine learning pipelines
CNStream is a streaming framework for building Cambricon machine learning pipelines

CNStream is a streaming framework for building Cambricon machine learning pipelines

SecMML: Secure MPC(multi-party computation) Machine Learning Framework
SecMML: Secure MPC(multi-party computation) Machine Learning Framework

SecMML 介绍 SecMML是FudanMPL(Multi-Party Computation + Machine Learning)的一个分支,是用于训练机器学习模型的高效可扩展的安全多方计算(MPC)框架,基于BGW协议实现。此框架可以应用到三个及以上参与方联合训练的场景中。目前,SecMM

Comments
  • cleanup, refactoring, get working with different OpenFOAM versions

    cleanup, refactoring, get working with different OpenFOAM versions

    • the refactored python interfaces are now in two files (PythonCreate.H, PythonComm.H) and the solvers (which were adjusted from pimpleFoam) are now identical

    Next stage would be a function object wrapping of the details, which would eliminate need to dedicated solvers.

    opened by olesenm 1
  • Unable to find Urec_ in solution folders of AEFoam/Run_Case

    Unable to find Urec_ in solution folders of AEFoam/Run_Case

    Hello Romit, Many thanks for this GitHub page and for the wonderful workshop (ALCF PythonFOAM Workshop). The workshop was really very detailed and clear. I tried your steps but when I was trying with the AEFoam/Run_Case using ./All run, I found that the sims keep on going for the whole night, and on top of it, I could not see the variable Urec_ in any of the time folders. Am I missing and/or doing something wrong. Any comment/suggestion would be a great help, thanks in advance.

    opened by chandukec 0
Owner
Argonne Leadership Computing Facility - ALCF
ALCF is a DOE Office of Science User Facility that provides leadership supercomputing resources to advance scientific research.
Argonne Leadership Computing Facility - ALCF
Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and deploy without Python.

Python Inference Script(PyIS) Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and d

Microsoft 13 Nov 4, 2022
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

This is the Vowpal Wabbit fast online learning code. Why Vowpal Wabbit? Vowpal Wabbit is a machine learning system which pushes the frontier of machin

Vowpal Wabbit 8.1k Nov 17, 2022
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Nov 25, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.8k Nov 26, 2022
An open-source, low-code machine learning library in Python

An open-source, low-code machine learning library in Python ?? Version 2.3.6 out now! Check out the release notes here. Official • Docs • Install • Tu

PyCaret 6.6k Nov 28, 2022
A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms.

iNeural A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms. What is a Neural Network? Work on

Fatih Küçükkarakurt 5 Apr 5, 2022
Deep Scalable Sparse Tensor Network Engine (DSSTNE) is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models

Amazon DSSTNE: Deep Scalable Sparse Tensor Network Engine DSSTNE (pronounced "Destiny") is an open source software library for training and deploying

Amazon Archives 4.4k Nov 17, 2022
Implementation of Univaraint Linear Regresion (Supervised Machine Learning) in c++. With a data set (training set) you can predict outcomes.

Linear-Regression Implementation of Univaraint Linear Regresion (Supervised Machine Learning) in c++. With a data set (training set) you can predict o

vincent laizer 1 Nov 3, 2021
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

eXtreme Gradient Boosting Community | Documentation | Resources | Contributors | Release Notes XGBoost is an optimized distributed gradient boosting l

Distributed (Deep) Machine Learning Community 23.5k Nov 22, 2022
A lightweight C++ machine learning library for embedded electronics and robotics.

Fido Fido is an lightweight, highly modular C++ machine learning library for embedded electronics and robotics. Fido is especially suited for robotic

The Fido Project 412 Sep 19, 2022