Application-level optimization of big data transfers through pipelining, parallelism and concurrency

E Yildirim, E Arslan, J Kim… - IEEE Transactions on …, 2015 - ieeexplore.ieee.org
In end-to-end data transfers, there are several factors affecting the data transfer throughput,
such as the network characteristics (eg, network bandwidth, round-trip-time, background …

Towards optimizing large-scale data transfers with end-to-end integrity verification

S Liu, ES Jung, R Kettimuthu, XH Sun… - … Conference on Big …, 2016 - ieeexplore.ieee.org
The scale of scientific data generated by experimental facilities and simulations on high-
performance computing facilities has been growing rapidly. In many cases, this data needs …

Big data transfer optimization through adaptive parameter tuning

E Arslan, BA Pehlivan, T Kosar - Journal of Parallel and Distributed …, 2018 - Elsevier
Obtaining optimal data transfer performance is of utmost importance to today's data-
intensive distributed applications and wide-area data replication services. Tuning …

Predicting short-term variations in end-to-end cloud data transfer throughput using neural networks

E Yildirim, A Akon - IEEE Access, 2023 - ieeexplore.ieee.org
Predicting the data transfer throughput of cloud networks plays an important role in several
resource optimization applications, such as auto-scaling, replica selection, and load …

[PDF][PDF] A Comprehensive Study of Wide Area Data Movement at a Scientific Computing Facility.

Z Liu, R Kettimuthu, IT Foster, Y Liu - ICDCS, 2018 - lzhengchun.github.io
Wide-area data transfer is central to distributed science. Network capacity, data movement
infrastructure, and tools in science environments continuously evolve to meet the …

Energy-efficient data transfer optimization via decision-tree based uncertainty reduction

H Jamil, L Rodolph, J Goldverg… - … Conference on Computer …, 2022 - ieeexplore.ieee.org
The increase and rapid growth of data produced by scientific instruments, the Internet of
Things (IoT), and social media is causing data transfer performance and resource …

An elegant sufficiency: load-aware differentiated scheduling of data transfers

R Kettimuthu, G Vardoyan, G Agrawal… - Proceedings of the …, 2015 - dl.acm.org
We investigate the file transfer scheduling problem, where transfers among different
endpoints must be scheduled to maximize pertinent metrics. We propose two new …

A two-phase dynamic throughput optimization model for Big Data transfers

MDSQZ Nine, T Kosar - IEEE Transactions on Parallel and …, 2020 - ieeexplore.ieee.org
The amount of data transferred over dedicated and non-dedicated network links has been
increasing much faster than the increase in the network capacity. On the other hand, the …

Improving data transfer throughput with direct search optimization

P Balaprakash, V Morozov, R Kettimuthu… - 2016 45th …, 2016 - ieeexplore.ieee.org
Improving data transfer throughput over high-speed long-distance networks has become
increasingly difficult. Numerous factors such as nondeterministic congestion, dynamics of …

SMURF: Efficient and Scalable Metadata Access for Distributed Applications

B Zhang, T Kosar - IEEE Transactions on Parallel and …, 2022 - ieeexplore.ieee.org
In parallel with big data processing and analysis dominating the usage of distributed and
Cloud infrastructures, the demand for distributed metadata access and transfer has …