Project Team: Devika Jain, Jeff Blossom, Ben Lewis, Xiaokang Fu, Jack Hayes, Haoyu Wang, Wendy Guan
In the rapidly evolving field of geospatial big data, the demand for robust infrastructures capable of handling big data while ensuring cost efficiency and performance has become paramount. The Geospatial Infrastructure Enhancement project, led by the Center for Geographic Analysis (CGA) at Harvard University, represents a significant leap in the realm of geospatial big data infrastructure enhancement. The goal of the CGA is not to solve a single research question, but to design solutions for researchers engaging in geospatial data analysis. By utilizing the New England Research Cloud (NERC), this initiative aims to overcome the inherent challenges of handling, analyzing, and visualizing vast amounts of geospatial data. The project is particularly timely for CGA’s Devika Jain, who is leading the effort.
“Funded by the Office of Vice Provost for Research, the goal of this project is to empower researchers by developing robust infrastructure to enable execution of Geospatial big data/data science projects. Leveraging NERC resources, we developed a series of products, datasets, and solutions to make geospatial analytics faster, cheaper and more effective for Harvard researchers.” says Jain, data science project manager at CGA, a center located within Harvard’s Institute for Qualitative Social Science.”
The project acts as a powerful example of the transformative potential of cloud computing in academic research, offering scalable, cost-effective, and efficient solutions for complex geospatial tasks. It also yields insights into best practices and strategies for developing resilient infrastructures tailored to the demands of geospatial big data and data science.
Geospatial data presents unique challenges to researchers due to its volume, velocity, and variety. Additionally, the complexity of analysis and visualization requires robust resource management and customized visualization techniques. Local computing resources are often insufficient for these needs, necessitating a shift towards more scalable and efficient cloud-based solutions.
NERC, a project of the MOC Alliance, offers cutting-edge cloud computing services to academic and non-profit researchers. It provides a common cloud framework that can be easily tailored for data-driven analysis and visualization, making it an ideal platform for the Geospatial Infrastructure Enhancement project. By leveraging NERC, the project benefits from high-performance computing, efficient data storage, and flexible resource management, all at a lower cost than comparable commercial cloud services.
The project adopts several key strategies for utilizing the NERC to enhance geospatial data infrastructure:
1. Cloud Transition: Moving geospatial processes from local computing resources to the cloud or clusters ensures better scalability and resource management. This transition allows for efficient access to data storage and computing resources in close proximity.
2. GPU-Based Solutions: For high-performance tasks, the project utilizes GPU-based solutions. These are crucial for handling large datasets and performing complex analysis and visualizations quickly and efficiently.
3. Containerization: Containerizing applications enable rapid deployment and shutdown, providing flexibility and reducing overhead costs. This approach is particularly useful for on-demand tools that are not needed continuously.
4. Scalability: Applications are scaled based on computing needs, ensuring optimal resource allocation. This scalability is facilitated by NERC’s elastic storage and computing capabilities.
The project also emphasizes the importance of education and accessibility. It provides training and publications to help researchers and students utilize NERC’s capabilities effectively. Workshops, user manuals, and open-source repositories ensure that the knowledge and tools developed through the project are widely accessible. Notably, the project collaborates with industry technology providers like Heavy.ai and Istari.ai, making their platforms available for academic use, further enhancing the geospatial infrastructure across Harvard.
The Geospatial Infrastructure Enhancement project on NERC highlights the power of cloud computing in advancing geospatial research. By addressing the challenges of geospatial data through innovative solutions and leveraging the capabilities of NERC, the project not only enhances efficiency but also fosters collaboration and education in the field of geospatial research. As the effort continues to evolve, it promises to deliver even more sophisticated tools and insights, paving the way for groundbreaking discoveries and applications in geospatial science.
For more information on the project and to explore the capabilities of NERC, visit https://nerc.mghpcc.org/project/geospatial-infrastructure-enhancement/.
