Strengthening the Link between GIS and Science

insider

An interview with Kevin Butler about the integration of ArcGIS and SciPy
Geography is the science of our world, and GIS is a foundational technology for helping us to better understand that science. To further strengthen the link between GIS and science, today at the Esri Ocean GIS Forum we’re pleased to announce the integration of ArcGIS with SciPy, a Python-based ecosystem of open-source software for mathematics, science, and engineering.
I recently caught up with Kevin Butler, a Product Engineer with the Geoprocessing and Analysis Team, to ask him a few questions about the integration between ArcGIS and SciPy.

Matt Artz: What is SciPy?
Kevin Butler: SciPy is a software library that helps scientists, engineers, and GIS analysts perform custom scientific and technical computing. It is an open-source library built using Python—an easy to learn, highly scalable, stable scripting language. Python is both a programming language, and a collection of modules (small sections of code which perform some useful operations). For example, there is a Python module called ‘math’ which performs trigonometric functions and angular conversions. Python ships with a predefined set of modules which meet most of the needs for general scripting. SciPy extends the basic functionality of Python by adding modules which perform functions useful to the scientific and engineering communities.
SciPy is not just one Python module but a collection of modules and is sometimes referred to as the SciPy Stack. The SciPy Stack consists of seven core packages:

NumPy for numerical computation using arrays
SciPy, a collection of numerical algorithms
Matplotlib for 2D and 3D plotting
Pandas for high-performance data structures
SymPy for symbolic mathematics and computer algebra
IPython providing an interactive interface for quickly testing scripts and
Nose which allows you to test your Python code.

NumPy and Matplotlib have shipped with the ArcGIS platform for the last several releases. Adding the remaining core packages will fully integrate the SciPy Stack into the ArcGIS platform.

Matt: Why is it being integrated with ArcGIS?
Kevin: SciPy is being integrated with the ArcGIS platform to make scientific and technical computing easier for our users. While the SciPy Stack is open-source software and can be freely downloaded, it can sometimes be difficult to integrate with ArcGIS. Why? The strength of SciPy lies in its integration of many software modules, and installing these modules can be time consuming and error prone. For example, Pandas and Matplotlib may depend on a particular version of NumPy. If the correct version of NumPy is not loaded on your computer, Pandas and Matplotlib may not function as expected. Getting the correct versions of all of the components of the SciPy Stack can be challenging. Since SciPy will be integrated into the ArcGIS platform, all of these interdependencies will be taken care of for you.
Integrating SciPy with ArcGIS will make developing scientific and technical geoprocessing tools and scripts easier and more efficient. In other words, you won’t have to ‘start from scratch’ or ‘reinvent the wheel’ each time you start to solve a scientific or technical problem. The SciPy Stacks represents tens of thousands of lines of useful code which has already been thoroughly tested and documented. You can leverage the work of hundreds of other programmers and simply reuse code from one of the SciPy modules.

A custom Python geoprocessing tool designed to detect edges in an image. This tool leverages SciPy image filtering functions.

Another important reason that SciPy is being integrated with ArcGIS is that it will ensure that any custom scientific or technical geoprocessing tools you create are sharable. One of the fundamental design goals of Esri’s geoprocessing framework is the ability to distribute your tools among the ArcGIS community of users. If you develop a custom geoprocessing tool which depends on SciPy, you can share that tool as a geoprocessing package or a geoprocessing service and not have to worry whether the person you’re sharing with has the correct versions of SciPy loaded on their computer.

Matt: Can you give me some examples of what people might do with this?
Kevin: The possibilities of SciPy being integrated with ArcGIS are really only limited by the imagination of the GIS community. Python, and by extension SciPy, is embedded in the ArcGIS platform in order to make automating repetitive geoprocessing task and extending the geoprocessing framework easier. For the scientific and engineering communities, extending the geoprocessing framework will be the more important part. The SciPy stack has so many rich functions that it is hard to image all of the creative spatial problems that GIS users with scientific or engineering domain expertise will be able to tackle—but here are a few that come to mind:

An atmospheric scientist may use the image filtering modules to delineate zones of horizontal transport of water vapor.
A transportation geographer may use the Markov chain modules to simulate traffic flow.
A geoscientist may use the symbolic mathematics routines to trace faults and model crustal movement.
An ocean scientist may use the calculus module to calculate ocean dynamics
A fisheries scientist or resource manager may use the linear algebra modules to set a harvest quota for a fish stock.

Matt: Are there future plans for SciPy and ArcGIS?
Kevin: The future plans for SciPy and ArcGIS depend on the scientific and engineering practitioners in the GIS community. SciPy is open-source software, meaning that it grows and improves by users contributing new and/or improved code or modules. Newer versions of the SciPy stack will be integrated and supported in future releases of ArcGIS. Look for the familiar “What new in ArcGIS 10.x” documents on the web for information regarding potential new versions of the SciPy Stack.

Matt: How can our customers get it?
Kevin: SciPy will be integrated with ArcGIS through a staged release. It will be available for ArcGIS Pro with the 10.3 release as an optional install. At the 10.3.1 release, SciPy will be automatically installed with ArcGIS for Desktop.
What is ArcGIS Pro and why is SciPy being integrated there first? ArcGIS Pro is the new, premier application for visualizing, editing, and performing analysis. It has a 64-bit architecture and leverages threading to keep the user interface responsive and to utilize additional CPU cores on the local machine (a very important feature when running computationally intensive scientific algorithms).