From Big Data to Actionable Business Insights: JAKALA's Success with ArcGIS GeoAnalytics Engine

JAKALA, a marketing technology (MarTech) company headquartered in Milan with offices worldwide, specializes in marketing and sales, with a focus on artificial intelligence. Founded in 2000 by Matteo de Brabant, the company's 3,000 employees strive to deliver unique and customized solutions to more than 900 clients in more than 30 countries by integrating strategy, technology, and creativity. 

By utilizing the power of location intelligence, JAKALA aims to help companies unlock their potential by uncovering new opportunities, empowering decision-making, and enabling sustainable growth. 

Gaining a comprehensive understanding of consumer movement demands a significant volume of data, and processing this data can be time-consuming. JAKALA needed a way to efficiently analyze large volumes of data so that the company could provide valuable and timely insights for its clients. 

To overcome this challenge, JAKALA leveraged ArcGIS GeoAnalytics Engine in the Azure Databricks environment to provide insights more efficiently for the company's clients. 

Analyzing consumer movement is a critical component to understanding how various geographic areas attract visitors to points of interest (POIs) across different market sectors. Examples of POIs include hotels, restaurants, theaters, museums, shopping malls, and tourist attractions. To make sense of the data, it is important to understand not only the types of visitors, but also the POIs visited, the frequency of visits, and overall movement patterns. And, most importantly, it is critical that this data is analyzed and reported on in a timely fashion to facilitate decision-making. JAKALA faced several challenges around scaling up its analytic workflows to deliver quality data and analytics on the timeline needed by the company's clients.

Data volume and analytic complexity present a challenge when performing big data analysis. JAKALA works with large-volume people movement data that the company sources from a vendor that amalgamates opt-in location tracking. The sourced data relies on more than 10 million devices, generating around 1 billion data points per day. For monthly analytics, this typically means 30–40 billion data points that need to be incorporated in the analyses. The size of the data made it so that many geospatial analysis tools could not handle the entire dataset at once; the JAKALA team had to subset the data into weekly segments to complete the workflow. While segmented data is useful in some instances, longer time periods of data need to be used for forecasting as well as achieving richer pattern analyses.

For big data projects, it is important to have systems that scale easily. JAKALA's large volume of data in the workflow could not all be analyzed in one place or with one tool. This led to the shuffling of data between different analytics tools and systems to combine the data and generate final products. For this workflow, manual data compilation and analysis were time-consuming and inefficient, hindering JAKALA's ability to deliver timely insights to its clients.

Shortening the time to delivering insights is critical to decision-making, and processing large datasets needs to be fast. Once one month's data was received, it would take almost another month to complete data cleaning and analysis, resulting in a significant lag between data collection and analysis delivery.

JAKALA needed a single environment that would facilitate working with large volumes of data and spatial and tabular inputs as well as allow easy interoperability across all the company's data warehouses and analytics environments. 

To find the right solution, JAKALA explored several of Esri's big data products, including ArcGIS GeoAnalytics Engine. GeoAnalytics Engine enables spatial analysis of big data by extending Apache Spark with ready-to-use SQL functions and analysis tools. During the evaluation of a large analysis case with several billion data points, GeoAnalytics Engine demonstrated an optimal combination of analytical capabilities, swift performance, and seamless integration with JAKALA's data stored in a Microsoft Azure data store. 

Fabio D'Ovidio, head of the location intelligence platform at JAKALA, says, "In testing one of our largest datasets in GeoAnalytics Engine, we saw great performance. This made us realize that if this analysis case worked, then all of the other analyses for our use cases would also work quickly in GeoAnalytics Engine."

JAKALA is a Databricks partner and utilizes a comprehensive integration of Databricks’ platform to leverage the scalability and performance provided by that platform. Databricks provides an Apache Spark-based cloud platform with a unified set of tools for building, managing, and working with data at scale. GeoAnalytics Engine can work with Databricks using Azure, Amazon Web Services (AWS), or the Google Cloud platform as part of an integrated spatial analytics environment connected with the Databricks lakehouse architecture. 

Stefano Angarano, manager of web development and mobile data analysis at JAKALA, says, "Since our infrastructure is all on Azure, we can take our data stored there and easily combine it on Databricks with Esri's GeoAnalytics Engine. It's easier and more efficient to combine these environments instead of having to move the data somewhere else for analytics."

Esri's ArcGIS GeoAnalytics Engine provided the perfect solution with a Spark-native library of over 160 spatial functions and tools that work seamlessly within a Databricks workflow. As a bonus, JAKALA is now able to save time and energy on maintenance because working with GeoAnalytics Engine in Databricks allows staff to focus on the analytics and not on managing the company's own cloud or server infrastructure.

The integration of GeoAnalytics Engine with Azure Databricks has resulted in a substantial improvement in analytics processing time for JAKALA. Additionally, GeoAnalytics Engine has proved to be a practical and efficient solution, streamlining the overall analysis process and improving automation, scalability, and maintenance efficiency.

JAKALA staff saw immense improvement in analytic processing time when handling the company's monthly data—a result of the computing power of Databricks and GeoAnalytics Engine. 

"With GeoAnalytics Engine, we are able to receive all the data and perform the necessary analysis within a couple of days, whereas without it, it would take us weeks. This makes it much easier to provide our clients the data that they need, at the speed needed to make timely decisions," says Angarano. 

The shortened analysis time not only facilitates timelier data delivery, but also the performance gained makes it possible to complete analysis on much larger datasets. For instance, staff can now examine annual patterns instead of weekly or monthly patterns to facilitate longer-range analysis and forecasting. This helps staff conduct more comprehensive and accurate analyses and provide valuable insights and forecasts to the company's clients based on a more detailed understanding of annual patterns and trends.

JAKALA also found that GeoAnalytics Engine helped streamline the overall process. Using GeoAnalytics Engine in the Databricks environment improved staff's ability to automate and scale the analysis. They were able to analyze large volumes of data in a more efficient manner and could adjust the scale of resources based on the complexity of the analysis. Maintenance of the system was also simplified as GeoAnalytics Engine is cloud based and requires minimal manual updates.

"Performing monthly data analysis is a recurring task for us, and each month we have a consistent set of operations to perform. Having an automated solution is crucial for us to streamline the analysis process and eliminate the need for manual analysis, ensuring efficiency and accuracy in our operations," says Angarano. 

Using GeoAnalytics Engine to drive the analysis workflow has allowed JAKALA to offer new services to its customers, such as additional movement analysis and the ability to bring in larger amounts and longer timespans of data for improved context and forecasting. In the future, JAKALA is expanding to new data sources such as vehicle-generated data to complement the existing data staff work with from personal devices. This will open new doors for richer analyses about human movement and patterns.

"We are excited about the network analysis capabilities offered by GeoAnalytics Engine. The ability to leverage network-based service areas and closest facilities opens up new possibilities for us to incorporate additional metrics and enhance our analysis," says Angarano.

To explore a real-world use case about JAKALA utilizing GeoAnalytics Engine to analyze consumer movement data and evaluate the retail landscape, please read this blog to learn how retail mobility analytics can drive success. The use case delves into how JAKALA analyzes billions of data points to better understand geographic patterns, target advertising campaigns, and improve marketing strategies for its clients.