Description

I made this post is to summarize my learning about security set up Foundational Models of AWS Bedrock. I was living almost one month before where i started thinking in security in AI so now i am training for AIF-CO1 certification. I was surprise about guardrails so i decided to make an experiment to filter gender bias.

Well some phrases can be tricky like:

Emotional & Behavioral Stereotypes:

• “X are too emotional and dramatic.”
• “X are passive and submissive.”
• “X is too aggressive/bossy for a Y(she/he).”

Requirements

• AWS account.
• AWS User Bedrock.
• AWS policies for guardrails and Bedrock. link how to set up: https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference-examples.html

Steps

1. Choose a model and try it in the Open Playground. In my case i am using meta.llama3-8b-instruct-v1:0.

Amazon Bedrock > Model Catalog > Llama 3 8B Instruct

Then you will see the chat which you can interact

2. Then, create a script to test the model to check the correct access with the user.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

client = session.client("bedrock-runtime")

response = client.invoke_model(
    modelId=model_id,
    contentType="application/json",
    accept="application/json",
    body=json.dumps({
        "prompt": in_user,
        "max_gen_len": 200
    })
)

print(response["body"].read())

3. Add the guardrails in the model invocation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

bedrock_runtime = session.client(service_name='bedrock-runtime')
try:
    response = bedrock_runtime.converse(
        modelId=model_id,
        messages=conversation,
        guardrailConfig={
            'guardrailIdentifier': guardrail_id,
            'guardrailVersion': guardrail_version
        }
    )
    print(response['output']['message']['content'][0]['text'])
    print("Guardrail Action:", response['guardrailAction'])
except Exception as e:
    print(e)

note: i am using the function converse.

Conclusion

Take your time when implementing LLMs, as there can be pitfalls. It’s important to set them up securely to ensure others can use them safely.

Tip I wasted my tome to invoke my Guardrail becausde it has the default version 1 so i had to create a version of my guardrail.

IMPORTANT

I’m currently focused on my studies while actively looking for job opportunities to apply my skills and gain experience. 🥺

Offtopic

Lastly, if you have a dog, please remember to take good care of them.🐶