Autonomy and intelligence in the computing continuum: Challenges, enablers, and future directions for orchestration

H Kokkonen, L Lovén, NH Motlagh, A Kumar… - arxiv preprint arxiv …, 2022 - arxiv.org
Future AI applications require performance, reliability and privacy that the existing, cloud-
dependant system architectures cannot provide. In this article, we study orchestration in the …

[HTML][HTML] TITAN: A knowledge-based platform for Big Data workflow management

A Benítez-Hidalgo, C Barba-González… - Knowledge-Based …, 2021 - Elsevier
Abstract Modern applications of Big Data are transcending from being scalable solutions of
data processing and analysis, to now provide advanced functionalities with the ability to …

Enabling real time big data solutions for manufacturing at scale

A Cakir, Ö Akın, HF Deniz, A Yılmaz - journal of Big Data, 2022 - Springer
Today we create and collect more data than we have in the past. All this data comes from
different sources, including social media platforms, our phones and computers, healthcare …

[HTML][HTML] Distributed workflows with Jupyter

I Colonnelli, M Aldinucci, B Cantalupo… - Future Generation …, 2022 - Elsevier
The designers of a new coordination interface enacting complex workflows have to tackle a
dichotomy: choosing a language-independent or language-dependent approach. Language …

Efficient in-situ workflow planning for geographically distributed heterogeneous environments

F Li, F Song - Future Generation Computer Systems, 2023 - Elsevier
In-situ workflows are a particular class of scientific workflows where different components
(such as simulation, visualization, machine learning, and data analysis) run concurrently. In …

Big data workflows: Locality-aware orchestration using software containers

AA Corodescu, N Nikolov, AQ Khan, A Soylu… - Sensors, 2021 - mdpi.com
The emergence of the edge computing paradigm has shifted data processing from
centralised infrastructures to heterogeneous and geographically distributed infrastructures …

Flow-Bench: A dataset for computational workflow anomaly detection

G Papadimitriou, H **, C Wang, R Mayani… - arxiv preprint arxiv …, 2023 - arxiv.org
A computational workflow, also known as workflow, consists of tasks that must be executed
in a specific order to attain a specific goal. Often, in fields such as biology, chemistry …

Accelerating scientific workflows on HPC platforms with in situ processing

TMA Do, L Pottier, O Yildiz, K Vahi… - 2022 22nd IEEE …, 2022 - ieeexplore.ieee.org
Scientific workflows drive most modern large-scale science breakthroughs by allowing
scientists to define their computations as a set of jobs executed in a given order based on …

MLOps approach in the cloud-native data pipeline design

I Pölöskei - Acta Technica Jaurinensis, 2022 - acta.sze.hu
The data modeling process is challenging and involves hypotheses and trials. In the
industry, a workflow has been constructed around data modeling. The offered modernized …

Towards Efficient Workflow Scheduling Over Yarn Cluster Using Deep Reinforcement Learning

J Xue, T Wang, P Cai - GLOBECOM 2023-2023 IEEE Global …, 2023 - ieeexplore.ieee.org
Hadoop Yarn is an open-source cluster manager responsible for resource management and
job scheduling. However, data-driven applications are typically organized into workflows …