Amazon Bedrock Knowledge Bases allow you to connect your AI agents to existing data repositories, enabling them to access and utilize relevant information to enhance their responses.
Here are some key points about Knowledge Bases and Agents in Amazon Bedrock:<\/li>\n<\/ol>\n\n\n\n
- \n
- Knowledge Bases use a technique called Retrieval-Augmented Generation (RAG) to incorporate information from customer data sources into the responses generated by the foundation models powering the agents.
<\/li>\n\n\n\n - To connect a Knowledge Base to your data, you specify an Amazon S3 bucket containing the relevant documents or data files as the data source.
<\/li>\n\n\n\n - Knowledge Bases provide enriched contextual information to the agents, streamlining development by offering a fully managed RAG solution.
<\/li>\n\n\n\n - This abstraction accelerates time-to-market by minimizing the effort required to incorporate your data into the agent’s functionality and optimizes cost by negating the need for continuous model retraining to use private data.
<\/li>\n\n\n\n - Amazon Bedrock Agents can perform various tasks such as task orchestration, interactive data collection, task fulfillment, system integration, data querying, and source attribution, leveraging the connected Knowledge Bases.
<\/li>\n\n\n\n - The agents can process natural language inputs from users, preprocess the inputs, interact with Knowledge Bases and other action groups, orchestrate tasks, and provide post-processed responses.<\/li>\n<\/ul>\n\n\n\n
2. AWS API Gateway and Lambda<\/strong>: API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. Lambda is AWS’s serverless computing service that lets you run code without provisioning or managing servers. Together, they provide a scalable and cost-effective way to expose our AI agent to the world<\/p>\n\n\n\n
3. Pinecone: <\/strong>Pinecone <\/strong>is a vector database that allows for efficient storage and retrieval of high-dimensional vectors, making it perfect for semantic search applications.<\/p>\n\n\n\n
Designed specifically for storing, managing, and searching vector embeddings, Pinecone offers a powerful solution for developers and data scientists working on AI projects. <\/p>\n\n\n\n
At its core, Pinecone excels in managing vector embeddings, which are numerical representations of data commonly used in machine learning models. These embeddings can represent a wide variety of information, from text and images to audio and user behavior patterns. The service’s ability to efficiently store and query billions of these vectors sets it apart in the realm of data management for AI applications. <\/p>\n\n\n\n
One of Pinecone’s most notable features is its scalability. The service can effortlessly handle billions of vectors and process thousands of queries per second, making it an ideal choice for large-scale applications that demand high performance. This scalability is complemented by Pinecone’s low latency, with query times typically measured in milliseconds, enabling real-time applications that require instant responses. <\/p>\n\n\n\n
The heart of Pinecone’s functionality lies in its advanced similarity search capabilities. By leveraging sophisticated indexing techniques, including approximate nearest neighbor (ANN) search, Pinecone allows users to quickly find the most similar vectors to a given query vector, which is crucial for applications such as recommendation systems, semantic search, and image or audio similarity detection and obviously to prepare RAGs for LLMs. As a Cloud-native solution, Pinecone offers the advantages of a fully managed serverless service. This approach significantly reduces users’ operational overhead, allowing them to focus on developing their applications rather than managing infrastructure. The service provides a straightforward API that integrates seamlessly with various programming languages and frameworks, making it accessible to a wide range of developers. <\/p>\n\n\n\n
Pinecone’s versatility is further enhanced by its support for metadata filtering: users can attach additional information to their vectors and use this metadata to refine search results, which is very important in scenarios where contextual information plays a crucial role in determining relevance. <\/p>\n\n\n\n
The service also offers robust data management capabilities, including upsert operations that allow for easy updates and insertions of vector data.<\/p>\n\n\n\n
Its combination of speed, scalability, and advanced search capabilities makes it an invaluable tool for organizations looking to harness the full potential of their vector data.<\/p>\n\n\n\n
In our case, we’ll use it to store and query tokenized information about a customer of ours’s electrical services taken from the institutional website.<\/p>\n\n\n\n
4. Aurora Serverless PostgreSQL v2<\/strong>: Aurora Serverless is an on-demand, auto-scaling configuration for Amazon Aurora. We’ll use it with PostgreSQL compatibility to store and retrieve customer data efficiently and Aurora Data APIs for database interaction. The Aurora Data API is a secure HTTPS API that allows you to run SQL queries against an Amazon Aurora database without needing database drivers or managing connections. <\/p>\n\n\n\n
Here are some key reasons why you should consider using the Aurora Data API:<\/p>\n\n\n\n
- \n
- Simplify Application Development<\/strong>: The Data API provides an intuitive interface for interacting with your Aurora database, eliminating the need to manage complex database drivers and connection pooling. This can accelerate modern application development, especially for serverless applications like AWS Lambda.
<\/li>\n\n\n\n- Improved Scalability<\/strong>: The Data API automatically handles connection pooling and sharing between the API and the database. This helps your database applications scale more efficiently without the overhead of managing connections yourself.
<\/li>\n\n\n\n- Serverless Integration<\/strong>: The Data API enables seamless integration with serverless services like AWS Lambda, AWS AppSync, and AWS Cloud9, making it easier to build serverless applications that interact with your Aurora database.
<\/li>\n\n\n\n- Increased Availability<\/strong>: With the redesigned Data API for Aurora Serverless v2 and Aurora provisioned clusters, there is no rate limit imposed on requests, allowing for better scalability and availability.<\/li>\n<\/ol>\n\n\n\n
<\/p>\n\n\n
\n![\"\"](\"https:\/\/blog.besharp.it\/wp-content\/uploads\/2024\/09\/image1-1024x576.png\")
<\/figure><\/div>\n\n\n<\/p>\n\n\n\n
Setting Up the Environment <\/h3>\n\n\n\n
- \n
- AWS Account Setup:<\/strong> Ensure you have an AWS account with the necessary permissions to create and manage Bedrock, API Gateway, Lambda, and Aurora Serverless resources. <\/li>\n\n\n\n
- Pinecone Setup:<\/strong> Create a Pinecone account and set up a new index for storing the customer’s service information. <\/li>\n<\/ol>\n\n\n\n
To subscribe to Pinecone from AWS Marketplace, follow these steps:<\/p>\n\n\n\n
- \n
- Sign in to the AWS Management Console with your AWS account.<\/li>\n\n\n\n
- Navigate to the AWS Marketplace and search for “Pinecone”.<\/li>\n\n\n\n
- On the Pinecone product page, review the details and pricing information.<\/li>\n\n\n\n
- Choose “Continue to Subscribe” to start the subscription process.<\/li>\n\n\n\n
- Follow the prompts to review and accept the terms and conditions.<\/li>\n\n\n\n
- Configure any necessary settings or options for the subscription.<\/li>\n\n\n\n
- Review the final details and complete the subscription process.<\/li>\n<\/ol>\n\n\n\n
After subscribing, you can access and use Pinecone from within your AWS account: got to Marketplace > Manage subscriptions > Set Up Product (pinecone entry) > Set Up Account<\/p>\n\n\n\n
<\/p>\n\n\n
\n![\"create](\"https:\/\/blog.besharp.it\/wp-content\/uploads\/2024\/09\/image3-1024x551.png\")
<\/figure><\/div>\n\n\n<\/p>\n\n\n\n
3. Aurora Serverless PostgreSQL v2:<\/strong> Setup Create an Aurora Serverless PostgreSQL cluster in your AWS account. Configure the necessary tables for data. (See the guide<\/a>)<\/p>\n\n\n\n
4. Creating the Claude 3 Agent with AWS Bedrock: <\/strong>Accessing AWS Bedrock Navigate to the AWS Bedrock console and request access to the Claude 3 model: go to this link<\/a>; select relevant models and request access. This may take some time for approval. <\/p>\n\n\n\n
After model is approved (e.g. Antropic Sonnet 3) you’ll need to create an agent: visit this link<\/a> and create an agent with the name you prefer.<\/p>\n\n\n\n
5. Setup Bedrock Knowledge Base:<\/strong> visit this link<\/a> to create a KB, select pinecone when prompted. Use this guide<\/a> for reference. At some point, you\u2019ll need to create to Secrets Manager to store the Pinecone secret. After everything is set, you can populate the KB with data. To do so we just downloaded the text from the relevant Customer webpages but you can upload all the documents you think will be useful in the KB, supported formats are text and PDF. If you go for Opensearch Serverless (the AWS native vector DB) you can also connect internal enterprise data sources directly, such as Sharepoint, Confluence, Salesforce, and generic web crawlers.<\/p>\n\n\n\n
6. Defining the Agent’s Behavior:<\/strong> Using the following metaprompt, or similar, define the agent’s behavior. This includes specifying its role as a customer’s customer service agent, outlining its main tasks, and setting guidelines for interaction.<\/p>\n\n\n\n
Example metaprompt:<\/p>\n\n\n\n
You are a Customer customer service agent. Your main task is to help the customer by answering their questions about services. You have to:\n\n1. Provide information about these services: Interventi sul punto di fornitura, Illuminazione cimiteriale, Allaccio alla rete elettrica. You can find the information in your knowledge base.\n\n2. List all the customer's invoice. The customer has to provide numero utenza and tipo utenza (bollette energia oppure bolletta illuminazione cimiteriale) in the prompt.\n\n3. Get all the information of a single invoice. The customer has to provide numero bolletta in the prompt.\n\n4. Send mail with a single invoice attached. The customer has to provide numero bolletta and email address in the prompt.\n\nYou need to have all the required parameters to invoke the action group. Don't complete the missing parameters. Ask the customer to provide the missing one.\n\nIf the provided email is formally wrong you have to ask the customer to insert a valid email\n\nIf you have found invoices that match the user's criteria print them in a user friendly format. Always reply in italian.\n\nIf you can't find any invoice on the criteria provided by the user, use a polite tone to let him know that you were unable to find any invoices that met the user's search criteria. Ask them to try again and give them guidance on what criteria their missing to get results that best meet their criteria. You also need to be flexible. If it doesn't match their exact criteria, you can still state other invoices you have in the desired period.\n\nWhen you provide answers don't link any vector db source reference.\nDon't answer to other topic questions.\n<\/code><\/pre>\n\n\n\n7. Implementing the Agent Logic: <\/strong>Create a new Lambda function that will serve as the backend for our agent. This function will allow Claude to execute operations defined in the OpenAPI file you need to upload to the agent in the action group. A simple code example for the lambda is the following:<\/p>\n\n\n\n
import json\nimport boto3\n\n# Replace these with your actual ARNs\ndb_clust_arn = \"your_database_cluster_arn\"\ndb_secret_arn = \"your_database_secret_arn\"\n\nrds_data = boto3.client(\"rds-data\")\ns3_client = boto3.client(\"s3\")\n\n\ndef lambda_handler(event, context):\n\tprint(event)\n\n\tagent = event[\"agent\"]\n\taction_group = event[\"actionGroup\"]\n\n\t# Flatten parameters for easier access\n\tflattened_params = {param[\"name\"]: param[\"value\"] for param in event[\"parameters\"]}\n\n\tprint(flattened_params)\n\n\tparameters = event.get(\"parameters\", [])\n\tprint(parameters)\n\tinvoice_number = parameters[0].get(\"value\")\n\treceiver_email = parameters[1].get(\"value\")\n\n\t# Construct SQL query\n\tget_invoice_information_sql = f\"\"\"\n \tSELECT *\n \tFROM invoices\n \tWHERE account_number = '{flattened_params['account_number']}'\n\t\"\"\"\n\n\tif flattened_params.get(\"account_type\"):\n \tget_invoice_information_sql += (\n \tf\" AND account_type = '{flattened_params['account_type']}'\"\n \t)\n\n\tprint(get_invoice_information_sql)\n\n\t# Execute SQL query\n\tresponse = rds_data.execute_statement(\n \tresourceArn=db_clust_arn,\n \tsecretArn=db_secret_arn,\n \tdatabase=\"your_database_name\",\n \tsql=get_invoice_information_sql,\n\t)\n\n\tprint(response)\n\n\trecords = response.get(\"records\")\n\tprint(records)\n\n\t# Prepare response\n\tresponse_body = {\"TEXT\": {\"body\": f\"{records}\"}}\n\n\taction_response = {\n \t\"actionGroup\": action_group,\n \t\"functionResponse\": {\n \t\"responseState\": \"REPROMPT\",\n \t\"responseBody\": response_body,\n \t},\n\t}\n\n\tsession_attributes = event[\"sessionAttributes\"]\n\tprompt_session_attributes = event[\"promptSessionAttributes\"]\n\n\t# Construct final Lambda response\n\tlambda_response = {\n \t\"response\": action_response,\n \t\"messageVersion\": event[\"messageVersion\"],\n \t\"sessionAttributes\": session_attributes,\n \t\"promptSessionAttributes\": prompt_session_attributes,\n\t}\n\n\treturn lambda_response\n<\/code><\/pre>\n\n\n\nNow that we have the Lambda function we can go on and add the Api actions and register the KB in the agent we created before: go back to the agent dashboard and edit the agent by adding the knowledge base and adding a new action group.<\/p>\n\n\n\n
<\/p>\n\n\n
\n![\"\"](\"https:\/\/blog.besharp.it\/wp-content\/uploads\/2024\/09\/image4-970x1024.png\")
<\/figure><\/div>\n\n\n<\/p>\n\n\n\n
You can add a new action group by adding instructions and an openapi swagger YAML, here is our example:<\/p>\n\n\n\n
\nopenapi: 3.0.0\ninfo:\n title: Customer invoices service\n version: 1.0.0\n description: APIs for retrieving customer's invoices\npaths:\n \/list-invoices\/:\n\tget:\n \tsummary: recupera la lista di bollette dati numero utenza e tipo utenza\n \tdescription: recupera la lista di bollette, quindi numero bolletta, periodo e importo, dati numero utenza e tipo utenza\n \toperationId: listInvoices\n \tparameters:\n \t- name: numero_utenza\n \tin: path\n \tdescription: Numero di utenza dell'utente che fa la richesta, \u00e8 necessario che sia fornito dall'utente\n \trequired: true\n \tschema:\n \ttype: string\n \t- name: tipo_utenza\n \tin: path\n \tdescription: Tipo di utenza dell'utente che fa la richiesta, valori ammessi \"Energia Elettrica\" e \"Illuminazione Cimiteriale\"\n \trequired: false\n \tschema:\n \ttype: string\n \tenum:\n \t- Energia Elettrica\n \t- Illuminazione Cimiteriale\n \tresponses:\n \t\"200\":\n \tdescription: recupera la lista di bollette dati numero utenza e tipo utenza\n \tcontent:\n \tapplication\/json:\n \tschema:\n \ttype: array\n \titems:\n \ttype: object\n \tproperties:\n \tnumero_utenza:\n \ttype: string\n \tdescription: Numero di utenza dell'utente che fa la richesta\n \ttipo_utenza:\n \ttype: string\n \tdescription: Tipo di utenza dell'utente che fa la richiesta\n \tlink_pdf_bolletta:\n \ttype: string\n \tdescription: S3 URI del pdf della\/delle bolletta\/e richieste\n \/get-invoices\/:\n\tget:\n \tsummary: Recupera tutti i dettagli di una singola bolletta, dato il numero di bolletta\n \tdescription: Recupera tutti i dettagli di una singola bolletta, dato il numero di bolletta\n \toperationId: getInvoices\n \tparameters:\n \t- name: numero_bolletta\n \tin: path\n \tdescription: Numero della bolletta, \u00e8 necessario che sia fornito dall'utente\n \trequired: true\n \tschema:\n \ttype: string\n \tresponses:\n \t\"200\":\n \tdescription: Recupera tutti i dettagli di una singola bolletta, dato il numero di bolletta\n \tcontent:\n \tapplication\/json:\n \tschema:\n \ttype: array\n \titems:\n \ttype: object\n \tproperties:\n \tnumero_utenza:\n \ttype: string\n \tdescription: Numero di utenza dell'utente che fa la richesta\n \ttipo_utenza:\n \ttype: string\n \tdescription: Tipo di utenza dell'utente che fa la richiesta\n \tlink_pdf_bolletta:\n \ttype: string\n \tdescription: S3 URI del pdf della\/delle bolletta\/e richieste\n \/list-readings\/:\n\tget:\n \tsummary: Recupera tutti le letture per codice contratto\n \tdescription: Recupera tutte le letture per codice contratto\n \toperationId: listReadings\n \tparameters:\n \t- name: codice_contratto\n \tin: path\n \tdescription: codice contratto, \u00e8 necessario che sia fornito dall'utente\n \trequired: true\n \tschema:\n \ttype: string\n \tresponses:\n \t\"200\":\n \tdescription: Recupera tutti le letture tramite codice contratto\n \tcontent:\n \tapplication\/json:\n \tschema:\n \ttype: array\n \titems:\n \ttype: object\n \tproperties:\n \tcodice_contratto:\n \ttype: string\n \tdescription: Codice del contratto utenza\n \t \n \/send-mail\/:\n\tget:\n \tsummary: Inviare una mail con allegata la bolletta richiesta, dato il numero bolletta e l'indirizzo email del cliente\n \tdescription: Inviare una mail con allegata la bolletta richiesta, dato il numero bolletta e l'indirizzo email del cliente\n \toperationId: sendMail\n \tparameters:\n \t- name: numero_bolletta\n \tin: path\n \tdescription: Numero della bolletta, \u00e8 necessario che sia fornito dall'utente\n \trequired: true\n \tschema:\n \ttype: string\n \t- name: email\n \tin: path\n \tdescription: Inviare una mail con allegata la bolletta richiesta, dato il numero bolletta e l'indirizzo email del cliente\n \trequired: true\n \tschema:\n \ttype: string\n \tresponses:\n \t\"200\":\n \tdescription: Recupera tutti i dettagli di una singola bolletta, dato il numero di bolletta\n \tcontent:\n \tapplication\/json:\n \tschema:\n \ttype: array\n \titems:\n \ttype: object\n \tproperties:\n \tnumero_utenza:\n \ttype: string\n \tdescription: Numero di utenza dell'utente che fa la richesta\n \ttipo_utenza:\n \ttype: string\n \tdescription: Tipo di utenza dell'utente che fa la richiesta\n \tlink_pdf_bolletta:\n \ttype: string\n \tdescription: S3 URI del pdf della\/delle bolletta\/e richieste\n<\/code><\/pre>\n\n\n\nBelow you can find the configuration we set in the action group section:<\/p>\n\n\n\n
<\/p>\n\n\n
\n![\"\"](\"https:\/\/blog.besharp.it\/wp-content\/uploads\/2024\/09\/image2-1024x815.png\")
<\/figure><\/div>\n\n\n<\/p>\n\n\n\n
After this has been set you should be able to invoke the agent through the integrated testing environment, just remember to save and publish the agent before trying. Should something fails you can find the error in the integrated environment, a very common mistake is not setting the correct lambda resource policy:<\/p>\n\n\n\n
<\/p>\n\n\n
\n![\"\"](\"https:\/\/blog.besharp.it\/wp-content\/uploads\/2024\/09\/image5-1024x797.png\")
<\/figure><\/div>\n\n\n<\/p>\n\n\n\n
The resource based agentsInvokeFunction statement should be similar to this:<\/p>\n\n\n\n
{\n \"Version\": \"2012-10-17\",\n \"Id\": \"default\",\n \"Statement\": [\n\t{\n \t\"Sid\": \"agentsInvokeFunction\",\n \t\"Effect\": \"Allow\",\n \t\"Principal\": {\n \t\"Service\": \"bedrock.amazonaws.com\"\n \t},\n \t\"Action\": \"lambda:invokeFunction\",\n \t\"Resource\": \"arn:aws:lambda:us-east-1:471112860008:function:generic-invoice-request-pg1aj\"\n\t}\n ]\n}\n<\/code><\/pre>\n\n\n\n8. Setting Up Logging<\/strong>: to set up logging in AWS Bedrock:<\/p>\n\n\n\n
- \n
- Navigate to the AWS CloudWatch console. <\/li>\n\n\n\n
- Create a new log group specifically for Bedrock logs. <\/li>\n\n\n\n
- In the Bedrock console, go to the “Settings” section. <\/li>\n\n\n\n
- Enable logging and select the log group you created. <\/li>\n\n\n\n
- Choose the log types you want to capture (e.g., request logs, response logs). <\/li>\n\n\n\n
- Set up log retention policies to manage storage costs. <\/li>\n<\/ul>\n\n\n\n
Consider using CloudWatch Insights to analyze your Bedrock logs for patterns or issues. You should also enable logging for the Knowledge base.<\/p>\n\n\n\n
9. Exposing the Agent via API Gateway<\/strong>: in the lambda console, create a new lambda function in python. The lambda will use boto3 to invoke the Bedrock agent with the user prompt, here is our simple example:<\/p>\n\n\n\n
import boto3\nimport json\nimport random\nimport string\n\n# Replace these with your actual Agent IDs\nAGENT_ID = \"YOUR_AGENT_ID\"\nAGENT_ALIAS_ID = \"YOUR_AGENT_ALIAS_ID\"\n\nbedrock_agent_runtime = boto3.client(\"bedrock-agent-runtime\")\n\n\ndef lambda_handler(event, context):\n\tprint(event)\n\n\t# Parse the incoming event body\n\tbody = json.loads(event[\"body\"])\n\tprint(body)\n\n\tsession_id = body[\"sessionId\"]\n\tinput_text = body[\"message\"]\n\tclient_code = body[\"clientCode\"]\n\n\t# Invoke the Bedrock agent\n\tresponse = bedrock_agent_runtime.invoke_agent(\n \tenableTrace=True,\n \tagentId=AGENT_ID,\n \tagentAliasId=AGENT_ALIAS_ID,\n \tsessionId=session_id,\n \tinputText=input_text,\n\t)\n\n\tprint(response)\n\n\t# Process the response chunks\n\tresp_text = \"\"\n\tfor chunk in response[\"completion\"]:\n \tprint(chunk)\n \tif \"chunk\" in chunk:\n \tdecoded_chunk = chunk[\"chunk\"][\"bytes\"].decode()\n \tprint(decoded_chunk)\n \tresp_text += decoded_chunk\n\n\t# Prepare the response\n\tresponse = {\n \t\"statusCode\": 200,\n \t\"headers\": {\n \t\"Content-Type\": \"application\/json\",\n \t\"Access-Control-Allow-Origin\": \"*\",\n \t\"Access-Control-Allow-Credentials\": True,\n \t},\n \t\"body\": resp_text,\n\t}\n\n\treturn response\n<\/code><\/pre>\n\n\n\nIn the API Gateway console, create a new REST API. Set up a resource and POST method that will trigger your Lambda function then configure the integration between API Gateway and your Lambda function. Ensure that the necessary permissions are in place for API Gateway to invoke Lambda. Deploy your API to a stage (e.g., “prod”) and note down the invocation URL, this is what clients will use to interact with your agent. <\/p>\n\n\n\n
10. Testing and Refinement\u00a0<\/strong><\/p>\n\n\n\n
Use tools like Postman or curl to send requests to your API endpoint and verify that the agent is responding correctly and test various scenarios, including: <\/p>\n\n\n\n
- \n
- General inquiries about the Customer services <\/li>\n\n\n\n
- Requests for invoice listings <\/li>\n\n\n\n
- Requests for specific invoice details <\/li>\n<\/ul>\n\n\n\n
Refine your agent’s behavior by adjusting the metaprompt, fine-tuning the Claude 3 model parameters, or modifying your Lambda function logic. <\/p>\n\n\n\n
Considerations and Best Practices <\/h3>\n\n\n\n
- \n
- Set up CloudWatch logs and metrics to monitor your Lambda function and API Gateway. This will help you track usage, detect errors, and optimize performance. Use the Bedrock logging setup we discussed earlier to gain insights into model interactions. <\/li>\n<\/ol>\n\n\n\n
- \n
- Be aware of the pricing models for the services you’re using:\n
- Set up CloudWatch logs and metrics to monitor your Lambda function and API Gateway. This will help you track usage, detect errors, and optimize performance. Use the Bedrock logging setup we discussed earlier to gain insights into model interactions. <\/li>\n<\/ol>\n\n\n\n
- Pinecone Setup:<\/strong> Create a Pinecone account and set up a new index for storing the customer’s service information. <\/li>\n<\/ol>\n\n\n\n
- Improved Scalability<\/strong>: The Data API automatically handles connection pooling and sharing between the API and the database. This helps your database applications scale more efficiently without the overhead of managing connections yourself.
- Simplify Application Development<\/strong>: The Data API provides an intuitive interface for interacting with your Aurora database, eliminating the need to manage complex database drivers and connection pooling. This can accelerate modern application development, especially for serverless applications like AWS Lambda.