ArcGIS Blog

GeoAI

ArcGIS Pro

What's new for GeoAI in the ArcGIS Pro 3.5 Image Analyst extension

By Pavan Yadav

Take advantage of powerful GeoAI capabilities in the ArcGIS Pro 3.5 Image Analyst extension. With this release, you can now do the following:

  • Explain object classification results better
  • Assess pixel classification accuracy
  • Monitor the model training process using TensorBoard
  • Utilize a new object detection model type

Discover how our latest features will streamline your workflows and boost your productivity.

Explainable AI: Understanding object classification models with Grad-CAM

We’ve added support for Gradient-weighted Class Activation Mapping (Grad-CAM) for the object classification deep learning workflows. This technique calculates the gradient of the most dominant logit with respect to the latest activation map, highlighting the most influential pixels in the image classification. When Grad-CAM support is enabled in the model definition, each feature in the output feature class now includes an attachment showing these influential pixels. This allows users to gain a better understanding of how the model makes decisions.

Grad-CAM highlights areas influencing damage classification: yellow for highest, then red.
Grad-CAM explanations for damaged structure classifications are shown. Yellow pixels indicate the highest influence on the model's decision, followed by red.

Compute Accuracy For Pixel Classification

The new Compute Accuracy For Pixel Classification tool evaluates the accuracy of pixel classification results from the Classify Pixels Using Deep Learning tool. It computes a confusion matrix based on omission and commission errors, and the Intersection over Union (IoU) score by comparing the output of the Classify Pixels Using Deep Learning tool with ground truth data.

The left side shows a portion of the accuracy report, while the right displays the Compute Accuracy For Pixel Classification tool interface.
Part of the accuracy report on the left, and the interface of the Compute Accuracy For Pixel Classification tool on the right.

TensorBoard integration in the Train Deep Learning Tool and Training Wizard

The Train Deep Learning tool and the Training wizard on the Imagery tab now include TensorBoard integration. This integration enables you to monitor the model training process. Selecting the ‘Enable TensorBoard’ option displays a TensorBoard dashboard, offering a visual representation of training logs. TensorBoard is a toolkit for visualizing training progress, displaying logs as graphs, images, histograms, and other data. This feature supports various models such as CycleGAN, DeepLab, and more. For additional details and the complete list of supported models, refer to What is TensorBoard?

TensorBoard interface
Example of the TensorBoard interface and reports

New RTDetrV2 object detection model type

ArcGIS Pro 3.5 introduces the RTDetrV2 model architecture for object detection. RTDETRV2 is built upon the architecture of RT-DETR, which is an object detection model that stands for “Real-Time DEtection Transformer.” This model is designed to perform object detection tasks with a focus on achieving real-time performance while maintaining high accuracy. RTDetrV2 provides selective multi-scale feature extraction in the decoder, and an optional discrete sampling operator, which makes the model easier to deploy on various hardware by removing certain technical limitations without sacrificing detection accuracy.

Wrap-up

These updates for GeoAI in the ArcGIS Pro 3.5 Image Analyst extension now provide you with new and enhanced tools for better understanding and control of your deep learning workflows. We hope these features will contribute to more effective and informative image analysis.

Share this article

Subscribe
Notify of
0 Comments
Oldest
Newest
Inline Feedbacks
View all comments
Joabel Barbieri(@joabelb)
March 1, 2023 7:35 am

Experience Builder doesn’t provide everything that WAB has. It’s a shame seeing it being retired instead of improved upon!!

Darryl Klassen(@dklassenkamloops-ca-2)
March 1, 2023 2:51 pm

Well it is just a matter of time before we have to move on as WAB is built on an old JS version. I know that EXB is getting more and more features all the time, but people need more time to adopt those tools in their day to day workflows before they remove options for using existing tools. Just because EXB has a feature, doesn’t mean its bugs have been flushed out, which needs to happen before people move off of WAB. Hopefully ESRI pushes this a bit more.

David Coley(@dcoley_sarco-2)
March 3, 2023 7:59 am

Yes – while I continue to be excited about the opportunities and capabilities the ExB provides, we still need – at a minimum – an interaction option between/with several widgets and the map popup. We can’t just rely on the Feature Info and Text (with dynamic link) widgets to return layer information. For example, the Search widget is great for feature layer searches to return info – but I still have to configure a separate Feature Info widgets to return results for each layer I want to use as a layer search source. This is a fundamental use of Search… Read more »

Will Christmas(@william-christmas_dgis)
March 3, 2023 8:39 am

BOOOOOOOOO!!! Upgrade WAB!!! ExB has some great features, BUT… WAB is easier and quicker to get an app going than ExB.

Adrian Marsden(@amarsden)
December 4, 2023 2:12 am

Alas, for sites like ours, with NO GPUs moving to Experience Builder is simply not an option, and it will be several years until our estate has migrated from VMs back to physical PCs

Aaron Vaughn(@aaron-vaughn_cogf)
May 14, 2024 8:55 am

With ArcGIS Solutions, the WAB in those content items will need updated (example: Sewer Data Management Solution and the Sewer Editor content item is from Web App Builder). Is there any info on how those can best be updated to Experience Builder?