{"id":7122,"date":"2024-07-17T08:00:00","date_gmt":"2024-07-17T06:00:00","guid":{"rendered":"https:\/\/blog.besharp.it\/?p=7122"},"modified":"2024-07-17T09:48:52","modified_gmt":"2024-07-17T07:48:52","slug":"proactive-data-driven-error-management-using-aws-x-ray-and-cloudwatch","status":"publish","type":"post","link":"https:\/\/blog.besharp.it\/proactive-data-driven-error-management-using-aws-x-ray-and-cloudwatch\/","title":{"rendered":"Proactive Data-Driven Error Management Using AWS X-Ray and CloudWatch"},"content":{"rendered":"\n
“Imagination is more important than knowledge. Knowledge is limited. Imagination encircles the world”<\/em>. Albert Einstein <\/p>\n\n\n\n User experience is critical for a website’s success: if a user experiences errors while using a service, it will impact the conversion rate and income. <\/p>\n\n\n\n Correlating data to find the root cause of an issue is a time-consuming process: It takes a lot of work to search the enormous amount of logs to identify what the user was doing and its journey. Sometimes, it’s also impossible to identify small glitches that worsen the user journey through our website.<\/p>\n\n\n\n Let’s assume that once the user logs into a website and starts a transaction, it is assigned a unique ID (a shopping cart identification, an order ID, or a session cookie). <\/p>\n\n\n\n If an error occurs and the user contacts us to solve it, we can indeed identify the issue. However, we need to manually correlate data from the error to the identifier used by our application. Then, we might know what the user was doing and find the cause of the problem.<\/p>\n\n\n\n This kind of approach is reactive<\/em>. What if we could switch the paradigm to proactively leverage the data and tools we already have to see which transactions have issues? Can we reach out easily and quickly to help the user before he opens a ticket? <\/p>\n\n\n\n We have already seen monitoring solutions on AWS: AWS X-Ray<\/a>, Amazon CloudWatch<\/a>, and CloudWatch RUM<\/a>, as well as how to extract meaningful data for our business intelligence. However, correlating live data to respond quickly to events and errors requires a deeper integration.<\/p>\n\n\n\n In this article, we’ll see how we can use a less-known CloudWatch feature (CloudWatch Custom Widgets) to gain better and faster insights about our users without using additional services. We already have the data we need; we only have to unleash our imagination \ud83d\ude42 <\/p>\n\n\n\n First things first: there’s an X-Ray integration with RUM: it’s a simple checkbox, and it will start tracing and correlating errors with lambdas, API calls, and AWS Services; here is an example from an actual use case:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n As you can see, with serverless architectures, complexity grows, and tracing requires skills and patience. <\/p>\n\n\n\n X-Ray adds a tracing header to ease the correlation between user requests and every component. <\/p>\n\n\n\n While this can be useful for a developer, it is difficult to use for different teams, such as support and data analysis. It also doesn’t have information about the application domain knowledge required to decide how to modify the application behavior. Can we have a link between an application error and application data? <\/p>\n\n\n\n For sure! Meet CloudWatch Custom Widgets.<\/strong><\/p>\n\n\n\n Custom Widgets help search and visualize data conveniently by relying on Lambda Functions, adding a helpful computation layer that can automate the human error-prone job of correlating data available in different places. <\/p>\n\n\n\n Let’s see how we have a dashboard that visualizes transactions with errors, searching and extracting meaningful data. <\/p>\n\n\n\n We will assume that X-ray traces already contain the transaction ID for convenience. We now have to find a way to get all the errors and display details about the transactions, extracting other meaningful information from our data sources. <\/p>\n\n\n\n Lucky for us, we can search X-Ray traces for errors recorded by RUM that happened in user’s browsers with a syntax similar to this:<\/p>\n\n\n\n With this simple query, we can have all the traces. Finding the transaction ID is only a matter of searching the records returned by our query in a lambda function using your preferred programming language with the AWS SDK libraries (i.e., python and boto3). <\/p>\n\n\n\n The same lambda can search for other information in databases, so we can retrieve information about the user ID, email, or other details to help solve the issue.<\/p>\n\n\n\nThe basics: tracing errors<\/h2>\n\n\n\n
<\/figure>\n\n\n\n
<\/figure>\n\n\n\n
fault AND (service(id(type: \"client::RUM\" )))<\/code><\/pre>\n\n\n\n