摘要
科学工作流为科学计算提供了工作流定义、流程管理和任务并行化等支持,高性能计算为大规模数据处理提供了集群管理、任务管理、资源调度等机制。如今正进入一个"大数据"时代,将科学工作流系统与高性能计算结合实现高性能计算平台上大规模并行计算具有重要意义。集成中间件与上层工作流系统和底层高性能计算平台进行交互,提供任务提交与状态监控功能。同时,集成方案为分布式集群中计算平台提供新的参考实现。基于上述分析以Swift科学工作流与Windows高性能计算平台集成方案为例,通过NASA MODIS图片处理工作流来分析并验证集成方案的可行性和性能。
Scientific workflow provides scientific computing with workfiow specification, workflow process management, task parallelism, etc. High performance computing provides mechanisms and development interfaces such as cluster management, task management, task scheduling, etc. to scientific computing. While we are entering into a ＂big data＂ era, it is necessary to integrate scientific workflow with high performance computing to implement the large scale parallel computing on high performance computing platform. The integration middleware interact with upper workflow systems and underlying HPC platform provides the support for task submission and status monitoring. The integration architecture will be a reference solution to the construction of computing platforms in distributed cluster environment. Taking Swift scientific workflow system and Windows HPC platform integration solution as references, a case study by using a NASA MODIS image processing workflow is presented to analyze and demonstrate the capability of the integrated system.
出处
《电子科技大学学报》
EI
CAS
CSCD
北大核心
2014年第3期457-463,共7页
Journal of University of Electronic Science and Technology of China
基金
国家自然科学基金(61034005
61073175)
关键词
计算平台
分布式集群
高性能计算
大规模并行计算
科学工作流
computing platform
distributed cluster
high performance computing
large scale parallel computing
scientific workflow
作者简介
赵勇（1971-），男，博士。教授，主要从事云计算、网格计算、高性能计算、大数据处理和工作流等方面的工作．
