Extracting data from SAP in a plug-and-play way with Amazon AppFlow
04 December 2024 - 6 min. read
Mehmed Dourmouch
DevOps Engineer
The infrastructure as Code (IaC) has been one of the most popular keywords in the cloud topic.
This made cloud infrastructure implementation much more reusable, versionable, and maintainable.
Several implementations and standards of this concept are recently born.
Some of those are Ops-oriented, which are usually based on declarative configuration files, like Terraform or AWS SAM.
Other tools are more Dev-oriented, using popular programming languages like Typescript, Java, or Python.
The official tool to develop your IaC in your AWS account is the Cloud Development Kit (CDK), made by AWS.
We have used it a lot, for different use cases, and today we are going to share our experiences to provide you with some useful tips. Let's dive deep!
AWS CDK is available for different programming languages, but we usually chose to use Typescript because it allows developers to use Javascript elasticity and Java type safety.
Those characteristics are very useful to speed up development, keeping type safety's advantages and enabling the usage of object-oriented design patterns.
A very cool thing about AWS CDK is the modularized structure.
Each AWS service it's represented as a node module, so it's a dependency of your node project.
This permits you to create a lightweight project without useless imports.
Usually, we design the CDK project structure by splitting the main stack into several nested stacks, using domain-driven criteria.
AWS CDK provides you with several abstraction layers to create AWS -resources.
We have identified 3 abstraction's macro-categories:
Typically can be identified from their name's root: "Cfn". Those elements give the possibility to customize your resources the same way you'd do on a CloudFormation Template.
(ie: CfnStateMachineProps)
These constructors are our favorites, they abstract and simplify some tricky resource configurations, without losing control of the created resources.
(ie: StateMachine)
These constructors can be very useful if you don't want to customize every resource's details.
They provide a very abstract configuration that creates several cloud resources.
We recommend reading them very carefully the documentation before implementing them to be sure that you need all resources that will be created.
(ie: FargateService)
AWS StepFunction it's a service that enables the creation of State machines on the cloud.
We found it very handy and well implemented into AWS CDK.
Each step of a state machine has one input and one output and can be of several types:
To create and make the state machine work there is a JSON-based language called Amazon States Language, it enables developers to create conditions and transitions that require manual interactions that connect steps to each other.
This language is a JSON object that contains these attributes:
This field is used as you use comments in a traditional programming language, it is used to describe State machine behaviors, is not required, so you can omit it if you want.
This field is not required and it is a number, it defines the maximum number of seconds an execution of the state machine can run. If it runs longer than the specified time, the execution fails with a state.
This attribute is not required, it is the version of the Amazon States Language and is used in the state machine (default is "1.0").
This field is mandatory and is a JSON object, used to describe the states that compose machine state.
Each key of that object is the step name.
Here is an example of it:
{
"State1" : {
},
"State2" : {
},
...
}
This attribute is required and defines the first state to invoke to start State Machine.
To create a Step Function using IaC technology with official AWS tools, you can use the CloudFormation service.
This is an example of a very simple flow implementation of a state machine.
As you can notice, a simple HelloWorld can be very verbose and tricky to configure with a standard Cloudformation template, because the JSON State language is not well integrated with the YAML template.
In one of our use cases, we needed to develop a flow for parallel operations and elaborate a report when all are completed.
With a standard template, this can be a little painful, so we tried to do it with CDK.
We found out that CDK other than providing a way to create and integrate cloud resources with your StepFunction easily also provides State Language abstractions.
Thanks to that writing a flow is more intuitive and maintainable.
Undoubtedly, AWS CDK is a very powerful tool. Anyway, it is important to keep in mind that it brings the need for a lot of programming knowledge. For this reason, it is necessary to master fundamental topics such as OOP (Object-oriented programming) before approaching CDK constructors.
This will allow you to create a scalable and maintainable code structure and to fully take advantage of IaC.
In this article, we have shared a real use case where AWS CDK can be very useful and some best practices that we usually implement in projects.
I hope you enjoyed this overview of AWS CDK and StepFunction. All questions are welcome!
See you again in 14 days for a new article on Proud2beCloud!