In-situ data analyses and machine learning with OpenFOAM and Python

Related tags

Overview

PythonFOAM:

In-situ data analyses with OpenFOAM and Python

Using Python modules for in-situ data analytics with OpenFOAM 8. NOTE that this is NOT PyFOAM which is an automation tool for running OpenFOAM cases. What you see in this repository, is OpenFOAM calling Python functions and classes for in-situ data analytics. You may offload some portion of your compute task to Python for a variety of reasons (chiefly data-driven tasks using the Python ML ecosystem and quick prototyping of algorithms).

PODFoam/: A pimpleFoam solver with in-situ collection of snapshot data for a streaming singular value decomposition. Python bindings are used to utilize a Python Streaming-SVD class object from OpenFOAM.
APMOSFoam/: A pimpleFoam solver with in-situ collection of snapshot data for a parallelized singular value decomposition. While the previous example performs the SVD on data only on one rank - this solver performs a global, but distributed, SVD. However, SVD updates are not streaming.

To compile and run

Use standard procedure to compile a new solver in OpenFOAM, i.e., use wmake from within PODFoam/. To run cases, it is assumed that you have a Python virtual environment that is linked to during compile and run time. The relevant lines are

-I/gpfs/fs1/home/rmaulik/OF8/OFPYENV/include/python3.6m/ \
-I/gpfs/fs1/home/rmaulik/OF8/OFPYENV/lib/python3.6/site-packages/numpy/core/include \

within EXE_INC and

-L/gpfs/fs1/home/software/spack-0.10.1/opt/spack/linux-centos7-x86_64/gcc-7.3.0/python-3.6.7-7eq7ubsfsxwib5oi7yk5ek7edv3cr7vt/lib \
-lpython3.6m

within EXE_LIBS. Replace these with the include/lib paths to your personal Python environments. The Python module within Run_Case/ directories require the use of numpy, matplotlib, and tensorflow so ensure that your environment has these installed. The best way to obtain these is to pip install tensorflow==2.1 which will automatically find the right numpy dependency and then pip install matplotlib to obtain plot capability. You will also need to install mpi4py which you can using pip install mpi4py.

Points of contact for further assistance - Romit Maulik ([email protected]). This work was performed by using the resources of the Argonne Leadership Computing Facility, a U.S. Department of Energy (Office of Science) user facility at Argonne National Laboratory, Lemont, IL, USA.

LICENSE

MIT

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

Light Gradient Boosting Machine LightGBM is a gradient boosting framework that uses tree based learning algorithms. It is designed to be distributed a

14.5k Jan 5, 2023

Comments

cleanup, refactoring, get working with different OpenFOAM versions
the refactored python interfaces are now in two files (PythonCreate.H, PythonComm.H) and the solvers (which were adjusted from pimpleFoam) are now identical

Next stage would be a function object wrapping of the details, which would eliminate need to dedicated solvers.
opened by olesenm 1
Unable to find Urec_ in solution folders of AEFoam/Run_Case

Hello Romit, Many thanks for this GitHub page and for the wonderful workshop (ALCF PythonFOAM Workshop). The workshop was really very detailed and clear. I tried your steps but when I was trying with the AEFoam/Run_Case using ./All run, I found that the sims keep on going for the whole night, and on top of it, I could not see the variable Urec_ in any of the time folders. Am I missing and/or doing something wrong. Any comment/suggestion would be a great help, thanks in advance.

opened by chandukec 0

In-situ data analyses and machine learning with OpenFOAM and Python

Related tags

Overview

PythonFOAM:

In-situ data analyses with OpenFOAM and Python

Contents

To compile and run

LICENSE

You might also like...

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

SHARK - High Performance Machine Learning for CPUs, GPUs, Accelerators and Heterogeneous Clusters

An Open Source Machine Learning Framework for Everyone

Distributed machine learning platform

An open source machine learning library for performing regression tasks using RVM technique.

Feature Store for Machine Learning

Machine Learning Platform for Kubernetes

CNStream is a streaming framework for building Cambricon machine learning pipelines

SecMML: Secure MPC(multi-party computation) Machine Learning Framework

Comments

cleanup, refactoring, get working with different OpenFOAM versions

Unable to find Urec_ in solution folders of AEFoam/Run_Case

Owner

Argonne Leadership Computing Facility - ALCF

Python Inference Script is a Python package that enables developers to author machine learning workflows in Python and deploy without Python.

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

An open-source, low-code machine learning library in Python

A library for creating Artificial Neural Networks, for use in Machine Learning and Deep Learning algorithms.

Deep Scalable Sparse Tensor Network Engine (DSSTNE) is an Amazon developed library for building Deep Learning (DL) machine learning (ML) models

Implementation of Univaraint Linear Regresion (Supervised Machine Learning) in c++. With a data set (training set) you can predict outcomes.

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

A lightweight C++ machine learning library for embedded electronics and robotics.