Efficient in-situ workflow planning for geographically distributed heterogeneous environments

F Li, F Song - Future Generation Computer Systems, 2023 - Elsevier
In-situ workflows are a particular class of scientific workflows where different components
(such as simulation, visualization, machine learning, and data analysis) run concurrently. In …

CAPIO: a middleware for transparent I/O streaming in data-intensive workflows

AR Martinelli, M Torquati, M Aldinucci… - 2023 IEEE 30th …, 2023 - ieeexplore.ieee.org
With the increasing amount of digital data available for analysis and simulation, the class of
I/O-intensive HPC workflows is fated to quickly expand, further exacerbating the …

BeeSwarm: enabling parallel scaling performance measurement in continuous integration for HPC applications

J Tronge, J Chen, P Grubel, T Randles… - 2021 36th IEEE/ACM …, 2021 - ieeexplore.ieee.org
Testing is one of the most important steps in software development–it ensures the quality of
software. Continuous Integration (CI) is a widely used testing standard that can report …

Towards Highly Compatible I/O-Aware Workflow Scheduling on HPC Systems

Y Dai, R Wang, Y Dong, K Lu - SC24: International Conference …, 2024 - ieeexplore.ieee.org
Scientific workflows on High-Performance Computing (HPC) consist of multiple data
processing and computing tasks with dependencies. Efficiently scheduling computing …

INSTANT: A Runtime Framework to Orchestrate In-Situ Workflows

F Li, F Song - European Conference on Parallel Processing, 2023 - Springer
In-situ workflow is a type of workflow where multiple components execute concurrently with
data flowing continuously. The adoption of in-situ workflows not only accelerates mission …

[BOOK][B] Mars: Multi-scalable actor-critic reinforcement learning scheduler

B Baheri - 2020 - search.proquest.com
In this thesis we introduce a new scheduling algorithm MARS based on a cost-aware multi-
scalable reinforcement learning approach, which serves as an intermediate layer between …

An HPC-Container Based Continuous Integration Tool for Detecting Scaling and Performance Issues in HPC Applications

J Tronge, J Chen, P Grubel, T Randles… - IEEE Transactions …, 2023 - ieeexplore.ieee.org
Testing is one of the most important steps in software development–it ensures the quality of
software. Continuous Integration (CI) is a widely used testing standard that can report …

Shared-memory communication for containerized workflows

T Hobson, O Yildiz, B Nicolae, J Huang… - 2021 IEEE/ACM 21st …, 2021 - ieeexplore.ieee.org
Scientific computation increasingly consists of a workflow of interrelated tasks.
Containerization can make workflow systems more manageable, reproducible, and portable …

BEE orchestrator: Running complex scientific workflows on multiple systems

J Tronge, P Grubel, T Randles… - 2021 IEEE 28th …, 2021 - ieeexplore.ieee.org
In this paper, we propose a workflow orchestration system that is able to run workflows on
both HPC systems and in the cloud using HPC containers. Most existing workflow …

Hydra: Brokering Cloud and HPC Resources to Support the Execution of Heterogeneous Workloads at Scale

A Alsaadi, S Jha, M Turilli - Proceedings of the 14th Workshop on AI and …, 2024 - dl.acm.org
Scientific discovery increasingly depends on middleware that enables the execution of
heterogeneous workflows on heterogeneous platforms. One of the main challenges is to …