How to work with text fields in Data Engineering

Text fields. They often hold some of the most important information in our data; whether a text field contains addresses that kick off our spatial adventures with a dataset, or they hold unique IDs that make joins (and bringing even more data) possible, they are certainly some of the most important data we work with as analysts and GIS professionals.

But text fields can also be some of the peskiest to work with: in some cases, they arrive in gloriously unstructured form, unleashing trains of thought from a survey form’s open text entry. In others, they look like unique IDs until a join acts unexpectedly and you realize the field has no right to be used in a join. We can typically work around these problems and get through the needed data wrangling, but we can all use a little help and make the experience more enjoyable.

In this blog and short video, we’ll introduce how Data Engineering in ArcGIS Pro helps you explore and prepare data with text fields. We’ll introduce some of the key things you can do, including:

• Finding, mapping, and charting text fields – Very little explores a field as efficiently as making a map and chart of its values.
• Efficiently checking their content – The Statistics Panel becomes a “data dashboard” that quickly summarizes information about all your fields in one place.
• Converting and encode them – When sneaky numeric and date data types sometimes show up as text fields, convert them. When regression analysis needs categories to become numbers, encode them.

It’s a lot to digest, so sip on some coffee, tea, water, or beverage of your choice and take a look at the following short video to see what’s possible.

These capabilities make Data Engineering an engaging way to explore text data. It becomes second nature to simply load an interesting new dataset, open the Data Engineering view, and get acquainted with text fields. As always, the Data Engineering view works best with used in conjunction with the Attribute Table, charts, and other existing aspects of ArcGIS Pro.

Stay tuned for our next post where you can learn more about working with date fields. Till the next time!

About the authors

Alberto Nieto

Alberto Nieto is a Product Engineer on Esri’s Spatial Statistics team. In his role, he helps research, build, and maintain spatial data science capabilities in ArcGIS and works closely with government agencies to learn about the problems our software can help solve. Alberto’s background includes fourteen years of experience, including previous roles as a GIS Developer at Capital One and NOAA's Climate Prediction Center, and as a GIS Analyst at the Census Bureau and the Alachua County Environmental Protection Department.

Connect:

Lynne Buie

Lynne Buie is a Product Engineer on the Spatial Statistics team. She helps build spatial statistics tools and Data Engineering capabilities for ArcGIS Pro. She's passionate about using GIS to help make the world a better place.

Connect:

Ankita Bakshi

Ankita Bakshi is a Product Owner and a Senior Product Engineer on the Spatial Statistics Team at Esri. With a background in environmental engineering and computer science, she is passionate about solving social, economic, and environmental problems with Spatial Analysis and Data Science. In her role, Ankita enjoys researching, finding solutions to build software, and loves creating video and written content to make the software tools more approachable and applicable to real world challenges. Outside of work Ankita enjoys going on hikes and dancing to Bollywood music.

Connect: