numpy<\/code>, netCDF4<\/code>, and pandas<\/code>. The aforementioned K-layers exported from SKUA-GOCAD in this example are written out as one layer per ASCII text file. Therefore, all text files must first be collected which includes the need for the standard module os<\/code> as well.<\/p>\nfolder<\/code> is a variable which indicates the folder directory where the ASCII files are located. files<\/code> holds a list of all files in the directory with their full path appended before the file name.<\/p>\n"},{"acf_fc_layout":"content","content":"import<\/span> netCDF4\r\nimport<\/span> os\r\nimport<\/span> numpy as np\r\nfrom<\/span> netCDF4 import<\/span> Dataset\r\nimport<\/span> pandas as pd\r\n \r\n# Folder path<\/span>\r\nfolder = r'C:\\temp\\myData'<\/span>\r\n\r\n# Collect all text file paths<\/span>\r\nfiles = [os.path.join(folder,f) for<\/span> f in<\/span> os.listdir(folder) if<\/span> os.path.isfile(os.path.join(folder, f))]\r\n<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"The ASCII file is delimited by a single space, and a newline separates entries. Using the read_table<\/code> function from the pandas module, a dataframe is created to hold the first K-layer entry. The size<\/code> variable is used to verify that each K-layer have an equal number of entries, due to the requirement of having equally spaced gridded data.<\/p>\n"},{"acf_fc_layout":"content","content":"# Create initial dataframe<\/span>\r\ndfPoints = pd.read_table(files[0<\/span>], delimiter = ' ') \r\n\r\n# All files need consistent size<\/span>\r\nsize = dfPoints.size\r\n<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Now that all files are collected into a list, they must be processed using a for loop starting at the first index (second element) of the list, since the zeroth index (first element) was already added to the dataframe. A dataframe buffer is created each iteration of the loop. If the size of the buffer is not equal to the size<\/code> variable, we continue to the next file in the list. If it is equal, then it is appended to the original dataframe, dfPoints<\/code>. The number of entries in this dataframe will be equal to the total number of entries across all valid files.<\/p>\n"},{"acf_fc_layout":"content","content":"# Put all data into dataframe<\/span>\r\nfor<\/span> i in<\/span> range(1, len(files)):\r\n dfBuffer = pd.read_csv(files[i], delimiter = ' ')\r\n if(dfBuffer.size != size):<\/span>\r\n continue\r\n<\/span><\/span>dfPoints = dfPoints.append(dfBuffer, ignore_index = True<\/span>)\r\n<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Since the data is not guaranteed to be in any kind of order, the dataframe is sorted using the sort_values<\/code> function. This data is sorted first by the Z coordinate, then Y, and finally X. Voxel layers have requirements for dimension order, (X,Y,Z,T) or (T,Z,Y,X) with the T and Z dimension being removeable, but at least 1 of them is required to be present in the data.<\/p>\nThe domains of each dimension must also be extracted from the dataframe. Using the index location, or iloc<\/code> operator, the raw values are accessed. Using the numpy function unique<\/code> a list of unique values for each dimension is created and then sorted.<\/p>\n"},{"acf_fc_layout":"content","content":"# Sort the values by Z, then Y, then X<\/span>\r\ndfPoints = dfPoints.sort_values(by=['Z','Y','X'<\/span>])\r\n\r\n# Create domain for longitude, latitude, and Z<\/span>\r\nxDomain = np.sort(np.unique(dfPoints.iloc[:,0<\/span>].values))\r\nyDomain = np.sort(np.unique(dfPoints.iloc[:,1<\/span>].values))\r\nzDomain = np.sort(np.unique(dfPoints.iloc[:,2<\/span>].values))<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Now the data is processed and mostly ready to be added to a NetCDF file. A output file must first be created using the Dataset<\/code> constructor. Dimensions of the NetCDF file are set using the createDimension<\/code> function for X, Y, and Z. The length of each domain must also be specified.<\/p>\n"},{"acf_fc_layout":"content","content":"# Create NetCDF<\/span>\r\noutDataSet = Dataset('myOutput.nc'<\/span>, 'w'<\/span>, format = 'NETCDF4'<\/span>)\r\n\r\n# Create dimensions<\/span>\r\noutDataset.createDimension('z'<\/span>,len(zDomain))\r\noutDataset.createDimension('y'<\/span>,len(zDomain))\r\noutDataset.createDimension('x'<\/span>,len(zDomain))<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"The createVariable<\/code> function is used to create a variables in a NetCDF file. Each dimension must have a variable associated with it, which contains the values in their respective domains. Other variables, such as Lithology and Predicted KF in this case, have multiple dimensions associated with it. For example, ncLithology<\/code> has the Z,Y,X dimensions, therefore the number of entries for ncLithology<\/code> will be the length of each dimension multiplied together. fill_value<\/code> is a property of the variable which indicates what value was used to fill missing data.<\/p>\n"},{"acf_fc_layout":"content","content":"# Create variables<\/span>\r\nncZ = outDataset.createVariable('z'<\/span>, np.float32, 'z'<\/span>)\r\nncY = outDataset.createVariable('y'<\/span>, np.float32, 'y'<\/span>)\r\nncX = outDataset.createVariable('x'<\/span>, np.float32, 'x'<\/span>)\r\nncPredictedKF = outDataset.createVariable('Predicted_kf'<\/span>, np.float32, ('z'<\/span>, 'y'<\/span>, 'x'<\/span>), fill_value = -99999<\/span>)\r\nncLithology = outDataset.createVariable('Lithology'<\/span>, int, ('z'<\/span>, 'y'<\/span>, 'x'<\/span>), fill_value = -9999<\/span>)<\/code><\/pre>\n"},{"acf_fc_layout":"content","content":"Now that the variables for the NetCDF file are created, the values in the dataframe that were processed earlier must be assigned to the variables. Using the\u00a0[:]<\/code> operator, the values processed are assigned to the NetCDF variable values. For the variables which have three dimensions, the [:,:,:]<\/code> is the same operator, but implies that there are three dimensions to the array. Variables more than one dimension must have their dataframe values be reshaped to match the size of the multidimensional array which was allocated for the output. Numpy has a reshape<\/code> function which allows this to be accomplished. The second argument of reshape<\/code> is a tuple of values, and in this example the tuple consists of the length of each dimension, Z,Y,X.<\/p>\n"},{"acf_fc_layout":"content","content":"# Assign values<\/span>\r\nncX[:] = xDomain[:]\r\nncY[:] = yDomain[:]\r\nncZ[:] = zDomain[:]\r\nncPredictedKF[:,:,:] = np.reshape(\r\n dfPoints['Predicted_kf'<\/span>].values,\r\n zDomain.shape[0<\/span>], yDomain.shape[0<\/span>], xDomain.shape[0<\/span>]\r\n)\r\nncLithology[:,:,:] = np.reshape(\r\n dfPoints['Lithology'<\/span>].values,\r\n zDomain.shape[0<\/span>], yDomain.shape[0<\/span>], xDomain.shape[0<\/span>]\r\n)<\/code><\/pre>\n <\/p>\n"},{"acf_fc_layout":"content","content":"
"}],"related_articles":"","card_image":false,"wide_image":false},"yoast_head":"\n