High Performance Computing of Hydrologic Models Using HTCondor
Location
ECC 216
Event Website
http://water.usu.edu/
Start Date
4-9-2013 10:40 AM
End Date
4-9-2013 11:00 AM
Description
“Big Iron” super computers and commercial cloud resources (Amazon, Google, Microsoft) are considered the most prominent resources considered for high performance computing (HPC) needs. These resources have many advantages; however the limited access of supercomputers and the cost associated with cloud resources may prohibit many water resource engineers and planners from pursuing HPC methods to improve design and decision-making. The goal of this presentation is to provide a model of HPC for water resource stakeholders who would benefit from an autonomous pool of free and accessible computing resources. To demonstrate this concept, a system called HTCondor was used at Brigham Young University in conjunction with the scripting language, Python, to parallelize intensive stochastic computations done with Gridded Surface Subsurface Hydrologic Analyst (GSSHA) models. HTCondor has been included in the image of each CPU in all of computer labs associated with the BYU Department of Civil and Environmental Engineering so that the pool of idle resources can contain from 100 to 200 processors. HTCondor is open source software developed by the University of Wisconsin - Madison, which provides access to the processors of idle computers for performing computational jobs on a local network. The GSSHA model is a fully distributed physics-based model that can predict runoff in a watershed. Stochastic simulations with GSSHA could overwhelm any lone CPU, but in the HTCondor environment, all of the heavy lifting is done in parallel and distributed among potentially hundreds of onsite processors. With HTCondor, computation times can be reduced by 90% over a single computer. HTCondor could be a viable solution to many computational needs under different circumstances using various water resource modeling software.
High Performance Computing of Hydrologic Models Using HTCondor
ECC 216
“Big Iron” super computers and commercial cloud resources (Amazon, Google, Microsoft) are considered the most prominent resources considered for high performance computing (HPC) needs. These resources have many advantages; however the limited access of supercomputers and the cost associated with cloud resources may prohibit many water resource engineers and planners from pursuing HPC methods to improve design and decision-making. The goal of this presentation is to provide a model of HPC for water resource stakeholders who would benefit from an autonomous pool of free and accessible computing resources. To demonstrate this concept, a system called HTCondor was used at Brigham Young University in conjunction with the scripting language, Python, to parallelize intensive stochastic computations done with Gridded Surface Subsurface Hydrologic Analyst (GSSHA) models. HTCondor has been included in the image of each CPU in all of computer labs associated with the BYU Department of Civil and Environmental Engineering so that the pool of idle resources can contain from 100 to 200 processors. HTCondor is open source software developed by the University of Wisconsin - Madison, which provides access to the processors of idle computers for performing computational jobs on a local network. The GSSHA model is a fully distributed physics-based model that can predict runoff in a watershed. Stochastic simulations with GSSHA could overwhelm any lone CPU, but in the HTCondor environment, all of the heavy lifting is done in parallel and distributed among potentially hundreds of onsite processors. With HTCondor, computation times can be reduced by 90% over a single computer. HTCondor could be a viable solution to many computational needs under different circumstances using various water resource modeling software.
https://digitalcommons.usu.edu/runoff/2013/AllAbstracts/13