Solar Radiation Methods

Methodology Overview

This project's goal is to create a high quality solar suitability map for the entire State of South Carolina. This is created through the use of a high resolution, 1/3 arc-second (about 9.8 meter) Digital Elevation Model (DEM) and the Area Solar Radiation tool within ArcGIS's spatial analysis extension. The product can be used to determine the solar potential in a given part of South Carolina. This analysis has many helpful and diverse applications such as solar energy generation, civil engineering, and even agriculture.

There are many challenges with this project. One of them is the computational power required for the solar radiation calculation. The run-time complexity of the algorithm is exponential. Meaning that as the study area increases linearly, the computational time needed will increase exponentially. Another problem we faced is that the solar radiation tool from ArcGIS is designed to be used in a small study area and not in a state-wide application. Both of these problems can be solved by using parallel processing and splitting the raster data into smaller pieces. By computing solar radiation in parallel, we were able to reduce processing time from one week to just under 3 hours!

To tackle these problems, we followed a 4 step process which consisted of:

Cleaning the DEM data and getting rid of inconsistencies
Splitting our DEM into thousands of smaller DEMs
Distributing our data to our GalaxyGIS cluster for calculating solar radiation in parallel
Mosaicing our data back together to obtain our final solar radiated DEM

This DEM was retrieved from the USGS

Step 1: DEM Modifications

The solar radiation tool from ArcGIS requires a DEM in order to calculate the solar potential of a given area. We obtained our 1/3 arc-second DEM from the United States Geological Survey (USGS). One thing we noticed is that the DEM is not perfect. There are some artifacts scattered around the map and this will impact the solar radiation tool's consistency.

We decided to use the Focal Statistics Tool to smooth out the DEM. This tool takes a defined neighborhood (5x5) around a cell and calculates the mean value. Then it replaces the value of the center cell with the mean. This process is repeated for every cell in the raster. One downside of this process is that it will decrease the accuracy of DEM thus also decreasing the accuracy of the resulting solar radiation, but the loss of accuracy is minimal.

Data Splitting

Step 2: Data Splitting

Running solar radiation on a DEM can be a computationally and time intensive process. Running solar radiation on an entire state would take days to run and could crash your computer. For that reason, we decided to split our data and process it in parallel. When splitting our data, we decided on 37,186 pieces with a buffer of 50 meters on all sides. The DEM has a height of 37,186 pixels, so we were able to split the data into just as many rows, each with a pixel height of 1.

ESRI's Area Solar Radiation tool is designed to process data with a area that spans no greater than 1 degree latitude in height. It takes a raster, computes the mean latitude for that area, and performs solar radiation based on that value. Therefore, any large scale processing is simply inaccurate the farther you move from the central latitude line. Breaking the raster data into blocks with more than a couple pixels in height, creates an unpleasant striping pattern due to the differing latitude values and the inaccurate values near the upper and lower edges of the block. For these reasons, we determined that splitting the data into small rows with pixel sized height would yield the most accurate and most efficient results.

After the splitting process, we ended up with 36,991 different DEMs, less than the originally planned 37,186 due to the presence of "NO DATA" values above and below the DEM.

Step 2: Distributive Processing

Step 3: Distributed Processing

For solar radiation analysis, we submitted the split data along with a python script and a submit file to our scheduler, HTCondor. HTCondor distributes the jobs among the nodes of our GalaxyGIS cluster and each node processes its individual job. When each job completes its processing, it is sent back to the machine that submitted the jobs. Using this method, we can process 700+ jobs in the time that it would take a single computer to process 1 job.

Step 4: Mosaic to New Raster

Once all of the jobs return to the submit machine, we can start the merging process. To merge all of the pieces back together, we used the Mosaic to New Raster tool from ArcGIS. This tool allows us to easily merge all of the raster pieces into one complete solar radiation map.