ArcGIS Survey123

Integrating ArcGIS Survey123 with AWS Lambda

This is a follow-up to my previous blog post on integrating the ArcGIS API for Python and Survey123 with Azure Functions. In that post, I highlighted two separate solutions using Azure Functions. While it is not necessary, it would be worthwhile to give it read to understand some of the context behind the use of serverless functions in the ArcGIS platform.

The first solution in my original post outlined the process for setting up an Azure Function with the ArcGIS API for Python. This is certainly possible in AWS Lambda but there are some limitations, such as building a minimum dependencies package in a Linux environment due to strict size requirements imposed for Lambda. I will not be focusing on that integration here as it would take another blog post entirely to address the steps.

The intent of this post is to really walk you through that process of setting up a webhook using an API Gateway and Lambda in AWS, as well as some bonus security tips on setting up a custom Lambda authorizer for the API Gateway. The end goal of sending an email notification when a response is submitted in Survey123 is pretty much the same as the second solution in my previous post. You can find the code samples used throughout this post on GitHub.

Setting Up The Lambda Function

1. In the AWS console, look for Lambda under Services or search for it in the search bar. The console should bring you to the Functions page.
2. Click Create function. You will be presented with four different options to create your new function.

Filled in options for creating a Lambda Function

3. You should be brought to an overview page with the Code tab open by default once your function is successfully created. Most of our time will be spent on this and the Configuration tabs.

Overview page for a Lambda Function.

4. Double-click the lambda_function.py file in the Code source pane. You should see some sample code in your editor. Essentially, this function handler is where all the magic happens; in more technical terms, where the code executes when the function is invoked by some type of event (trigger). The handler takes in two arguments: the event and context objects. Our focus will be on the event object as it will contain important information from Survey123 that is used to do some extra validation on the request in my code to ensure its authenticity. I do want to point out that these validation checks are not entirely necessary (it is a bit superfluous for security-sake), but will provide you with an idea on how to traverse the request sent to the handler.

Return statements from the Python Lambda function.
Deploy text highlighted in red to save the Lambda function changes.

5. Let’s head over to the Configuration tab to make one more change.

The three environment variables used in the lambda function.

6. Our function is now completely setup and ready to be invoked by a trigger.

All the completed options for adding a trigger to a Lambda function.
The trigger link (after being added to the function) is highlighted in red.

Configuring the API Gateway

The API Gateway is one of the most critical components in this workflow, as it creates and exposes a REST endpoint to which the Survey123 request is sent. As of right now, this endpoint will not work since we have not configured Cross-origin resource sharing (CORS) for our API. This is entirely due to the fact that the HTTP request is non-simple, requiring a preflight request to be sent to the server for verification. If you selected the API Gateway link after setting up the trigger in the Lambda function, you should be brought to the Resources page for your API.

1. Select Actions and then Enable CORS

Enable Cors button is highlighted in red in the API Gateway configuration page.

2. The only default option we will change is the Access-Control-Allow-Origin. As of right now, it is set to a wildcard (*), allowing requests to come from any origin. Let’s change that to only allow those requests from the Survey123 website. Change the wildcard to https://survey123.arcgis.com. Those three headers should also look familiar to you, since they were included in our return statement in the Lambda function. If you were to remove those headers, our function would throw an error.

All of the completed options for enabling CORS on the API Gateway.

3. Click Enable CORS and replace existing CORS headers and Yes, replace existing values at the next prompt. If you do not see any errors during the configuration, that means CORS was enabled successfully.

4. Next, we need to deploy our changes to our default stage to make it usable in the webhook process. This process essentially saves any changes you have made to your resources. Typically, you would follow a development lifecycle and deploy changes from dev -> stag -> prod to ensure things run smoothly. In the Resources pane, select Actions and then Deploy API.

The deploy API action is highlighted in red within the configuration page.

5. In the pop-up box, you will be prompted to select a Deployment stage. You can either apply the changes to the default or create a new one.

The default pop-up when deploying an API Gateway to a stage.

6. Once the changes are deployed to the API Gateway, all we need to do now is to configure the webhook in our published survey.

The default stage editor shows the invoke URL and resource path (highlighted in red) for the API Gateway.
The payload URL is highlighted in red in the S123 configuration page for a webhook.

7. Upon successful configuration, you should receive an email notification.

The email response from a successful S123 webhook.
The selection for View logs in CloudWatch are highlighted in red on the Lambda configuration page.

Throwing a Lambda authorizer into the mix (Optional)

If you are like me and prefer a little more security implemented into your API Gateway, we can add a custom Lambda authorizer to control who invokes the Lambda function attached to our REST API by parsing out specific information. Essentially, this will provide an extra layer of security to authenticate access to the Lambda function and thereby limiting its invocation to valid requests only.

1. Navigate to the Lambda functions page in the AWS console and select Create function.

Completed inputs for creating the authorizer function in Lambda.
The JavaScript code used in the Lambda authorizer.

2. In the Configuration tab under Environment variables, select Edit and add the two variables used in the script.

The two environment variables used in the Lambda authorizer function.

3. In your API Gateway, go to Authorizers and select Create New Authorizer.

Completed configuration for creating an authorizer in the API Gateway.

4. The last step in this process is to apply the authorizer to a resource method in the API Gateway.

The process to add an authorizer to the ANY Method Request.

5. Retest your webhook through Survey123 to ensure it still works as expected. If you do not receive an email or get any errors in your browser’s console, you can check the CloudWatch logs of your authorizer for more information. I would highly recommend viewing the logs regardless as it will give you an idea on how the request is presented to the authorizer, which differs from the one sent to your actual Lambda function for the webhook.

Conclusion

This blog post walked you through an example on integrating Survey123 webhooks with AWS Lambda and the API Gateway with some additional security suggestions to lock things down further. Lastly, it should be noted that this workflow in AWS can also be applied to other components of the ArcGIS platform.

About the author

Taylor Teske

Taylor is a Technical Consultant in Professional Services out of Denver, CO, focusing on integrating the ArcGIS platform with other technologies and systems on-premise and in the cloud.

Connect:

Leave a Reply

Please Login to comment

Next Article

Monthly Linux Tip: Properly Setting File Handles and Processes Limits

Read this article