{"id":7479,"date":"2024-12-04T08:00:00","date_gmt":"2024-12-04T07:00:00","guid":{"rendered":"https:\/\/blog.besharp.it\/?p=7479"},"modified":"2024-12-04T15:18:07","modified_gmt":"2024-12-04T14:18:07","slug":"extracting-data-from-sap-in-a-plug-and-play-way-with-amazon-appflow","status":"publish","type":"post","link":"https:\/\/blog.besharp.it\/extracting-data-from-sap-in-a-plug-and-play-way-with-amazon-appflow\/","title":{"rendered":"Extracting data from SAP in a plug-and-play way with Amazon AppFlow"},"content":{"rendered":"\n
In a world where data drives business innovation, the ability to collect, organize, and analyze it efficiently is crucial. A data lake is a centralized repository that ingests and stores large volumes of data in its original form. Thanks to its scalable architecture, a data lake can accommodate all types of data from any source \u2013 structured (relational database tables, csv files), semi-structured (XML files, webpages), and unstructured (images, audio files, tweets) \u2013 all without sacrificing fidelity.<\/p>\n\n\n\n
AWS offers a comprehensive ecosystem of services that enable you to create and manage data lakes, making them an ideal foundation for a wide range of analytic needs. Additionally, through integrations with third-party services, it\u2019s possible to further enhance a data lake, gaining a holistic and in-depth view of business data.<\/p>\n\n\n\n
In this article, we will explore a specific integration between Amazon AppFlow and SAP to prepare data for a data analysis platform.<\/p>\n\n\n\n
Let’s start defining the main actors in our scenario.<\/p>\n\n\n\n
Imagine being able to transfer data between your SaaS applications and AWS quickly, securely, and without implementing complex automation. That\u2019s the magic that Amazon AppFlow offers: a service that simplifies integrations and allows you to automate data flows with just a few clicks, taking care of all the important aspects of data flow integrations. It relieves you of the duties of managing complex infrastructure and writing code to integrate different data sources.<\/p>\n\n\n\n
A crucial aspect of data flow integration is the privacy of the connection channel used. Fortunately, AWS helps address this concern with its PrivateLink service. Amazon AppFlow natively integrates with AWS PrivateLink<\/strong>, enabling connections to data sources through a private connection without exposing data to the public internet. This ensures enhanced data security, especially when dealing with sensitive information.<\/p>\n\n\n\n Let’s analyze our AWS Organization’s setup focusing on the main aspects:<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n Please note that the SAP-owned account, as well as our accounts are all connected through Transit Gateway VPC Attachments. This scalable network setup ensures that all our current and future AWS accounts will be able to communicate between each other. Another aspect that you should consider is the proper configuration of the routing: following the best practice of least privilege guarantees that accounts have access only to the data and services they are explicitly permitted to use.<\/p>\n\n\n\n Once the SAP services are connected to the main account we need to find an effortless way to make the data available to our consumers. For this purpose, we can use an Endpoint Service powered by AWS PrivateLink<\/p>\n\n\n\n PrivateLink enables us to share a service hosted in AWS with other consumer accounts, it requires a Network Load Balancer (supports also Gateway Load Balancers), which receives requests from consumers and routes them to your service.<\/p>\n\n\n\n In order to configure a private Amazon AppFlow connection, we must first create and verify an Endpoint Service that will be the entry point to the SAP services.<\/p>\n\n\n\n To create the Endpoint Service, you need to consider various things:<\/p>\n\n\n\n Although these steps are more focused on SAP services, you can apply the same principles to any other external services that you want to integrate with your consumer applications in AWS.<\/p>\n\n\n\n Wait a few minutes for provisioning and validation to take effect and you should be ready to go!<\/p>\n\n\n\n Now that we have an Endpoint Service, we can move to the child AWS accounts and create a new Flow that has a SAP OData source.<\/p>\n\n\n\n From the Amazon AppFlow console select SAP OData as Connector type and insert the required parameters for the connection:<\/p>\n\n\n\nPrerequisites and AWS Organization Layout<\/h2>\n\n\n\n
\n
<\/figure>\n\n\n\n
Setup Endpoint Service<\/h3>\n\n\n\n
\n
Setup Amazon AppFlow Connection<\/h3>\n\n\n\n