Sample data<\/h2>\n
For this exercise, we\u2019ll use a subset of monthly evapotranspiration<\/a> (ET) from ET-Amazon<\/a> for the state of Rond\u00f4nia in Brazil. This region has been subject to deforestation, observed as a decrease in ET. The ET-Amazon data set has reports ET (mm\/month) in a three-dimensional space: x<\/strong>, y<\/strong>, and t<\/strong>.<\/p>\n After you downloaded the ETAmazon_subset.zip<\/a> file, extract its contents into your local computer to the C:\\temp\\netcdf_blog<\/strong> directory.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":832801,"id":832801,"title":"NetCDF-directory","filename":"directory-1.png","filesize":18213,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis\/data-management\/creating-netcdf-files-for-analysis-and-visualization-in-arcgis\/directory-2","alt":"NetCDF directory","author":"9412","description":"","caption":"","name":"directory-2","status":"inherit","uploaded_to":799071,"date":"2020-04-22 16:09:34","modified":"2020-04-22 16:09:47","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":275,"height":424,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","medium-width":169,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","medium_large-width":275,"medium_large-height":424,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","large-width":275,"large-height":424,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","1536x1536-width":275,"1536x1536-height":424,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","2048x2048-width":275,"2048x2048-height":424,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","card_image-width":275,"card_image-height":424,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/directory-1.png","wide_image-width":275,"wide_image-height":424}},"image_position":"left-center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":" In python, load the required libraries using the import <\/strong>statement. Assign the directory of the extracted data to the data_path<\/strong> variable. Use the netCDF4.Dataset<\/strong> function to create an empty netCDF file. Remember to use the the *.nc<\/strong> extension in the name of the file and the \u2018w\u2019 <\/strong>option to be able to write to the netCDF file.<\/p>\n"},{"acf_fc_layout":"content","content":" Global attributes are key to describe the netCDF to other collaborators or users. Remember to review the CF convention for a complete list of required and suggested attributes.<\/p>\n"},{"acf_fc_layout":"content","content":" Create three dimensions: one for longitude (x), one for latitude (y) and one for time (t). In the sample data, we have values for 176 rows (y), 244 columns (x), and 3 months (t).<\/p>\n"},{"acf_fc_layout":"content","content":" Create five variables: one variable per dimension (longitude, latitude, and time), one variable for the mapping grid (crs), and one variable for the evapotranspiration values (ET). Notice that each variable have associated dimensions to them. The longitude, latitude, and time variables have only their own dimension associated to each one of them. The mapping grid (crs) has zero dimensions, and ET has three dimensions (in order: time, latitude, and longitude) for a total of 176 x 244 x 3 = 128,832 data values.<\/p>\n"},{"acf_fc_layout":"content","content":" Let\u2019s load the values for the variables of time, latitude, longitude, and evapotranspiration.<\/p>\n Use the datetime<\/strong> library to calculate the seconds since epoch (01\/01\/1970). Store the values in a list and store them in the times_values<\/strong> variable.<\/p>\n"},{"acf_fc_layout":"content","content":" Create two lists using np.arange<\/strong> for the latitudes and longitudes of the center points of the grid cells. The grid cells are spaced 0.0025<\/strong> degrees. Add or subtract half-cell size (i.e. 0.0025\/2) to the starting point to get the coordinates at the middle instead of the edge of the grid cells.<\/p>\n"},{"acf_fc_layout":"content","content":" Let\u2019s explore three different ways to load values into the ET variables: from a raster, a point shapefile, and a csv file. In short, we load the values using python libraries like arcpy, csv, or numpy and save them into the et_var<\/strong> variable array.<\/p>\n Use the read<\/strong> method of the arcpy.Raster<\/a> object\u00a0to load the values of the ETAmazon_201306.tif<\/strong> raster. Store the array into time third position (index=2)<\/p>\n"},{"acf_fc_layout":"content","content":" Note:<\/strong> As an alternative workflow, we could also use the arcpy.RasterToNumPyArray<\/a>\u00a0function to load the values of the\u00a0ETAmazon_201306.tif\u00a0raster.<\/p>\n Use a SearchCursor<\/a> to read the values for each point in the ETAmazon_200906.shp<\/strong> shapefile. Get the longitude and latitude indices (i, j) by identifying the closest longitude or latitude values, that is where the absolute difference is the lowest. Store the array into time second position (index=1).<\/p>\n"},{"acf_fc_layout":"content","content":" Open the ETAmazon_200506.csv<\/strong> file and read the values as dictionaries, using the headers as keys. Similarly as with the point shapefile, identify the indices (m, n) of the closest latitude and longitude values. Store the array into time first position (index=0).<\/p>\n"},{"acf_fc_layout":"content","content":" Finally, close the nc<\/strong> file.<\/p>\n"},{"acf_fc_layout":"content","content":" NetCDF is a multidimensional data format that can be used extensively in the ArcGIS platform, from 3D visualizations to multidimensional analysis of complex data sets. It is easy to create netCDF files using python, from existing data formats such as rasters, point shapefiles, or csv files.\u00a0 NetCDF\u2019s are self-describing, meaning software packages can directly read the data and determine format.\u00a0 The structure contains the variable names and essential metadata such as units.\u00a0 These capabilities also lend the netCDF format to be used widely for scientific data archival. Government agencies such as NASA and NOAA use netCDF format to deliver data to end users.<\/p>\n Join GeoNet<\/a> and ask a question to our community of experts.<\/p>\nLoad libraries and create empty netCDF file<\/h2>\n
import<\/span> netCDF4\r\nimport<\/span> os\r\nimport<\/span> datetime as<\/span> dt\r\nimport<\/span> numpy as np\r\nimport<\/span> arcpy\r\nimport<\/span> csv\r\n\r\n# Data path<\/span>\r\ndata_path = r'C:\\temp\\netcdf_blog'<\/span>\r\n\r\n# Create NetCDF File<\/span>\r\noutput_nc = os.path.join(data_path, 'ETAmazon_20200406.nc'<\/span>)\r\nnc = netCDF4.Dataset(output_nc, 'w'<\/span>)<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Global attributes<\/h2>\n
# Global attributes<\/span>\r\nnc.title = 'ET-Amazon'<\/span>\r\nnc.summary = ('Actual monthly evapotranspiration in Rondonia, Brazil '\r\n 'for June 2005, 2009, and 2013'<\/span>)\r\nnc.keywords = 'Evapotranspiration, Amazon, Water cycle'<\/span>\r\nnc.license = ('This work is licensed under a Creative Commons '\r\n 'Attribution 4.0 International License.'<\/span>)\r\nnc.references = ('Paca, V.H., Espinoza-Davalos, G.E., Hessels, T.M., '\r\n 'Moreira, D., Comair, G.F., Bastiaanssen, W. (2019). '\r\n 'The Spatial Variability of Actual Evapotranspiration '\r\n 'Across the Amazon River Basin Based on Remote Sensing '\r\n 'Models Validated with Flux-Towers. Ecological Processes. '\r\n '8(1), 6. https:\/\/doi.org\/10.1186\/s13717-019-0158-8'<\/span>)\r\nnc.source = 'https:\/\/www.hydroshare.org\/resource\/24792a48a6394dcba52da62fa324ae40\/'<\/span>\r\nnc.Conventions = 'CF-1.6'<\/span>\r\nnc.institution = 'Esri'<\/span>\r\nnc.history = '{0} creation of ET-Amazon netcdf file.'<\/span>.format(\r\n dt.datetime.now().strftime(\"%Y-%m-%d\"<\/span>)\r\n )<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Create dimensions<\/h2>\n
# Create dimensions<\/span>\r\nlat_dim = nc.createDimension('latitude'<\/span>, 176<\/span>)\r\nlon_dim = nc.createDimension('longitude'<\/span>, 244<\/span>)\r\ntim_dim = nc.createDimension('time'<\/span>, 3<\/span>)<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Create variables<\/h2>\n
# Create variables<\/span>\r\nlat_var = nc.createVariable('latitude'<\/span>, np.float64, ('latitude'<\/span>))\r\nlat_var.units = 'degrees_north'<\/span>\r\nlat_var.standard_name = 'latitude'<\/span>\r\nlat_var.axis = 'Y'<\/span>\r\n\r\nlon_var = nc.createVariable('longitude'<\/span>, np.float64, ('longitude'<\/span>))\r\nlon_var.units = 'degrees_east'<\/span>\r\nlon_var.standard_name = 'longitude'<\/span>\r\nlon_var.axis = 'X'<\/span>\r\n\r\ntime_var = nc.createVariable('time'<\/span>, np.int32, ('time'<\/span>))\r\ntime_var.standard_name = 'time'<\/span>\r\ntime_var.calendar = 'gregorian'<\/span>\r\ntime_var.time_step = 'Monthly'<\/span>\r\ntime_var.units = 'Seconds since 1970-01-01 00:00:00'<\/span>\r\ntime_var.axis = 'T'<\/span>\r\n\r\ncrs_var = nc.createVariable('crs'<\/span>, np.int8, ())\r\ncrs_var.standard_name = 'crs'<\/span>\r\ncrs_var.grid_mapping_name = 'latitude_longitude'<\/span>\r\ncrs_var.crs_wkt = (\"GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',\"\r\n \"SPHEROID['WGS_1984',6378137.0,298.257223563]],\"\r\n \"PRIMEM['Greenwich',0.0],\"\r\n \"UNIT['Degree',0.0174532925199433]]\"<\/span>)\r\n\r\net_var = nc.createVariable('ET'<\/span>, np.int16, ('time'<\/span>, 'latitude'<\/span>, 'longitude'<\/span>),\r\n fill_value<\/span>=9999<\/span>)\r\net_var.units = 'mm\/month'<\/span>\r\net_var.long_name = 'Actual Evapotranspiration'<\/span>\r\net_var.short_name = 'ET'<\/span>\r\net_var.grid_mapping = 'crs'<\/span><\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Load values<\/h2>\n
Time<\/h3>\n
# Load values: time<\/span>\r\ndate_200506 = int<\/span>((dt.datetime(2005<\/span>,6<\/span>,1<\/span>) - dt.datetime(1970<\/span>,1<\/span>,1<\/span>)).total_seconds())\r\ndate_200906 = int<\/span>((dt.datetime(2009<\/span>,6<\/span>,1<\/span>) - dt.datetime(1970<\/span>,1<\/span>,1<\/span>)).total_seconds())\r\ndate_201306 = int<\/span>((dt.datetime(2013<\/span>,6<\/span>,1<\/span>) - dt.datetime(1970<\/span>,1<\/span>,1<\/span>)).total_seconds())\r\ntime_values = [date_200506, date_200906, date_201306]\r\ntime_var[:] = time_values<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Latitude and Longitude<\/h3>\n
# Load values: latitude and longitude<\/span>\r\nlat_values = np.arange(-9.6491667<\/span> - 0.0025<\/span>\/2<\/span>, -10.0891667<\/span>, -0.0025<\/span>)\r\nlon_values = np.arange(-64.735<\/span> + 0.0025<\/span>\/2<\/span>, -64.125<\/span>, 0.0025<\/span>)\r\nlat_var[:] = lat_values\r\nlon_var[:] = lon_values<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Evapotranspiration<\/h3>\n
Load values from raster<\/h4>\n
# Load ET values from raster<\/span>\r\nras_201306 = os.path.join(data_path, 'ETAmazon_201306.tif'<\/span>)\r\nraslyr_201306 = arcpy.Raster(ras_201306)\r\narray_201306 = raslyr_201306.read()\r\net_var[2<\/span>, :, :] = array_201306[:, :, 0<\/span>]<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"# Load ET values from raster<\/span>\r\nras_201306 = os.path.join(data_path, 'ETAmazon_201306.tif'<\/span>)\r\narray_201306 = arcpy.RasterToNumPyArray(ras_201306)\r\net_var[2<\/span>, :, :] = array_201306<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Load values from point shapefile<\/h4>\n
# Load ET values from point shapefile<\/span>\r\nshp_200906 = os.path.join(data_path, 'ETAmazon_200906.shp'<\/span>)\r\nwith<\/span> arcpy.da.SearchCursor(shp_200906, ['SHAPE@X'<\/span>, 'SHAPE@Y'<\/span>, 'ET'<\/span>]) as<\/span> cursor:\r\n for<\/span> row in<\/span> cursor:\r\n i = (np.abs(lon_values - row[0<\/span>])).argmin()\r\n j = (np.abs(lat_values - row[1<\/span>])).argmin()\r\n et_var[1<\/span>, j, i] = row[2<\/span>]<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Load values from csv file<\/h4>\n
# Load ET values from csv<\/span>\r\nfile_200806 = os.path.join(data_path, 'ETAmazon_200506.csv'<\/span>)\r\nwith<\/span> open(file_200806, 'r') as<\/span> f:\r\n reader = csv.DictReader(f)\r\n for<\/span> line in<\/span> reader:\r\n m = (np.abs(lon_values - float<\/span>(line['longitude'<\/span>]))).argmin()\r\n n = (np.abs(lat_values - float<\/span>(line['latitude'<\/span>]))).argmin()\r\n et_var[0<\/span>, n, m] = line['ET'<\/span>]<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Close netCDF file<\/h2>\n
nc.close()\r\n<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Summary<\/h2>\n
More information?<\/h2>\n
References<\/h2>\n
"},{"ID":6701,"user_firstname":"Keith","user_lastname":"VanGraafeiland","nickname":"Keith VanGraafeiland","user_nicename":"kvangraafeiland","display_name":"Keith VanGraafeiland","user_email":"KVanGraafeiland@esri.com","user_url":"https:\/\/oceans-esrioceans.hub.arcgis.com\/","user_registered":"2018-03-02 00:18:44","user_description":"Keith VanGraafeiland is a Senior Principal GIS Engineer at Esri and Ocean Content Lead for the ArcGIS Living Atlas of the World. With over 18 years of experience, Keith specializes in marine and environmental GIS, remote sensing, and advanced data science. Keith\u2019s work bridges oceanography and data science, using machine learning and analytics to turn complex data into actionable insights for the GIS community.","user_avatar":"
"}],"related_articles":"","card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/NetCDF_BlogCard.jpg","wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/04\/NetCDF_BlogBanner-e1726511576981.jpg"},"yoast_head":"\n