Design and Implementation of a Web Service-Oriented Gateway to Facilitate Environmental Modeling using HPC Resources
ISBN: 978-88-9035-744-2 http://www.iemss.org/sites/iemss2014/papers/iemss2014_submission_254.pdf
Abstract
Environmental researchers, modelers, water managers, and users often require access to highperformance computing (HPC) resources for running data and computationally intensive models without being an HPC expert. To address these challenges, we have developed a gateway to HPC storage and computational resources. This gateway software (that we have named HydroGate) is a CGI based REST web service that takes input via HTTP methods then transmits commands to the HPC system using SSH. The gateway abstracts away many details and complexities involved in the use of HPC systems including authentication, authorization, data and job management - transferring the data back and forth as well as creation, monitoring and scheduling of the jobs without installing any third-party software on the HPC systems. We demonstrate the use of these web services to provide a web based interface to the TauDEM hydrologic terrain analysis tools. This prototype implementation allows a user with a HPC account and resource allotment to upload a digital elevation model (DEM), execute TauDEM functions on this DEM to delineate a watershed and derive other TauDEM products. The TauDEM tools are executed on the HPC system taking advantage of parallel methods. The user can then prepare (offline) the inputs for a model, such as the Utah Energy Balance (UEB) Snowmelt model, for a delineated watershed and submit the UEB input files and execute the model on the HPC system. The contribution of this study is the realization of the gateway service exposing an interface to the client applications that require access to the resources and services on the HPC centers in a secure and straightforward manner. The design and implementation are described, and the computational experience gained while developing the gateway is reported.
