Part 1: Basics of Using Ansible Core to Request a Let’s Encrypt Certificate

Prerequisites

Ansible Core installed on your control node.
Ansible Vault enabled.
Cloudflare account with API token and DNS Zone ID for the zone your using to create the Certificate.
- you can use this endpoint in postman to retrieve your DNS Zone ID: https://api.cloudflare.com/client/v4/zones/
Basic knowledge of Ansible and YAML syntax.

For this tutorial I will be using Ubuntu 22.04.4 LTS. This can be installed on a standalone VM/server, run on windows via WSL or on a docker container.

Gitlab Repository

The Repository for this project is Here

Steps

1. Setting Up Ansible Vault

Ansible Vault allows you to encrypt sensitive data such as API keys. Follow these steps to set up Ansible Vault for storing your Cloudflare API key and DNS Zone ID.

1.1 Create the Vault File

Create a vaultvars.yml file to store your sensitive information:

ansible-vault create vaultvars.yml

You’ll be prompted to enter a password for encrypting the file. Use a strong password and remember it, as you’ll need it to edit or use the file.

1.2 Add Sensitive Variables

Copy add the following variables to the vaultvars.yml file:

# vaultvars.yml
API_key: your_cloudflare_api_key
zone_id: your_cloudflare_dns_zone_id

Replace your_cloudflare_api_key and your_cloudflare_dns_zone_id with your actual Cloudflare API key and DNS Zone ID.

1.3 save and Exit

The file will be opened in vi. to save and exit press esc. then type :wq! [enter]

The contents are now encrypted and will be used securely in your playbook.

2. Writing the Ansible Playbook

Create an Ansible playbook (request_cert.yml) that uses the encrypted variables and handles the Let’s Encrypt certificate request.

you can use your favorite editor I like to use nano

nano request_cert.yml

2.1 Define the Playbook

Paste the following code in to the newly Created file request_cert.yml:

2.2 Explanation

Here’s a detailed breakdown of each task in the Ansible playbook:

Playbook Structure

---
- name: Create Letsencrypt cert
  hosts: localhost
  gather_facts: false
  vars_prompt:
    - name: fqdn
      prompt: Enter domain name for the new Certificate
      private: false
  vars_files:
    - vaultvars.yml
  vars:
    files_loc: ~/cert-store
    le_private_key: "{{ files_loc }}/le-account.key"
    csr_private_key: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.key"
    email: your_email@example.com
    dns_zone: "{{ fqdn.split('.')[1] }}.{{ fqdn.split('.')[2] }}"
  tasks:
    ...

Purpose:
This is the basic Structure of the playbook

Key Parameters:

vars_prompt: Prompts the user to enter the FQDN for the new certificate.
vars_files: Specifies the vault file containing the encrypted variables.
vars: Defines default variables, such as file locations and email.
- files_loc: The location where the certs will be stored
- le_private_key: The location where the key will be stored
- csr_private_key: The location where the CSR private key will be stored.
- email: Email associated with the Let’s Encrypt account
- dns_zone: The FQDN without the subdomain. [example.com]
tasks: Executes various tasks, including directory creation, private key generation, CSR creation, ACME account setup, DNS challenge handling, and certificate retrieval.

Lets Breakdown Each Task

1. Create Directory for Certificate Files

- name: Create directory for certs directory
  file:
    path: "{{ files_loc }}/{{ fqdn }}"
    state: directory

Purpose:
Creates a directory at the specified path to store the certificate and key files for the domain.

Key Parameters:

path: The location of the directory to be created.
state: Ensures the directory exists (directory state).

Explanation:
Ensures that a directory is available to store the generated certificate and keys for the domain.

2. Generate Let’s Encrypt Account Private Key

- name: Generate Letsencrypt private key
  community.crypto.openssl_privatekey:
    path: "{{ le_private_key }}"
    state: present

Purpose:
Generates the private key for the Let’s Encrypt account if it does not already exist.

Key Parameters:

path: The location where the key will be stored.
state: Ensures the key exists (present state).

Explanation:
This private key is used to register and manage the Let’s Encrypt account.

3. Generate CSR Private Key

- name: Generate CSR private key
  community.crypto.openssl_privatekey:
    path: "{{ csr_private_key }}"
    state: present

Purpose:
Generates a private key for the Certificate Signing Request (CSR) if it does not already exist.

Key Parameters:

path: The location where the CSR private key will be stored.
state: Ensures the key exists (present state).

Explanation:
This key is used to create the CSR for the specific domain.

4. Ensure Let’s Encrypt Account Exists and Agrees to TOS

- name: Make sure account exists and has given contacts. We agree to TOS.
  community.crypto.acme_account:
    validate_certs: false
    account_key_src: "{{ le_private_key }}"
    state: present
    terms_agreed: yes
    acme_version: 2
    acme_directory: https://acme-v02.api.letsencrypt.org/directory
    contact:
      - "mailto:nomail@{{ dns_zone }}"

Purpose:
Registers the Let’s Encrypt account or ensures it exists, and agrees to the terms of service.

Key Parameters:

account_key_src: Path to the Let’s Encrypt account key.
state: Ensures the account is registered (present state).
terms_agreed: Automatically agrees to the Let’s Encrypt terms of service.
acme_directory: URL of the Let’s Encrypt ACME directory.
contact: Contact email for the Let’s Encrypt account.

Explanation:
This task sets up the Let’s Encrypt account and ensures that the user agrees to the terms and service required for certificate issuance.

5. Generate an OpenSSL Certificate Signing Request

- name: Generate an OpenSSL Certificate Signing Request
  community.crypto.openssl_csr:
    path: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.csr"
    privatekey_path: "{{ csr_private_key }}"
    common_name: "{{ fqdn }}"

Purpose:
Generates a CSR using the previously created CSR private key.

Key Parameters:

path: Path to save the generated CSR.
privatekey_path: Path to the CSR private key.
common_name: The domain name for which the certificate is requested.

Explanation:
The CSR is required by Let’s Encrypt to issue a certificate for the specified domain.

6. Create a Challenge Using an Account Key File

- name: Create a challenge using an account key file.
  community.crypto.acme_certificate:
    validate_certs: false
    account_key_src: "{{ le_private_key }}"
    account_email: "{{ email }}"
    src: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.csr"
    cert: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.crt"
    challenge: dns-01
    acme_directory: https://acme-v02.api.letsencrypt.org/directory
    remaining_days: 60
  register: com_challenge

Purpose:
Initiates the ACME process to create a DNS-01 challenge for Let’s Encrypt validation.

Key Parameters:

account_key_src: Path to the Let’s Encrypt account key.
account_email: Email associated with the Let’s Encrypt account.
src: Path to the CSR.
cert: Path to save the generated certificate.
challenge: Specifies the DNS-01 challenge type.
acme_directory: URL of the Let’s Encrypt ACME directory.
remaining_days: Number of days before expiration to consider renewal.

Explanation:
This task initiates the process of obtaining a certificate by generating a DNS challenge.

Output Registered As: com_challenge

7. Create a TXT Record on Cloudflare

- name: Create a TXT record on Cloudflare when the record file exists
  uri:
    url: "https://api.cloudflare.com/client/v4/zones/{{ zone_id }}/dns_records"
    method: POST
    body_format: json
    headers:
      X-Auth-Email: "{{ email }}"
      Authorization: "Bearer {{ API_key }}"
      Content-Type: "application/json"
    body:
      type: "TXT"
      name: "{{ com_challenge.challenge_data[fqdn]['dns-01'].record }}"
      content: "{{ com_challenge.challenge_data[fqdn]['dns-01'].resource_value }}"
      ttl: 1
    status_code: 200
  when: com_challenge.challenge_data[fqdn]['dns-01'].record is defined
  register: record

Purpose:
Creates a TXT DNS record in Cloudflare for the DNS-01 challenge validation.

Key Parameters:

url: API endpoint to create DNS records in Cloudflare.
method: HTTP method (POST).
headers: Authentication headers including the API key.
body: Payload containing the DNS record details.
status_code: Expected HTTP status code on success.

Explanation:
This task sets the DNS-01 challenge record in Cloudflare to validate domain ownership.

Condition: Only runs if com_challenge.challenge_data[fqdn]['dns-01'].record is defined.

Output Registered As: record

8. Extract and Store the Record ID

- name: Extract and store the record ID from the creation response
  set_fact:
    txt_record_id: "{{ record.json.result.id }}"
  when: record is defined and record.json.success

Purpose:
Extracts and stores the ID of the created TXT record from Cloudflare’s response.

Key Parameters:

txt_record_id: Variable to store the extracted record ID.

Explanation:
Stores the record ID for later use, particularly for removing the TXT record after validation.

Condition: Only runs if record is defined and the API call was successful.

9. Wait for TXT Entry to Appear

- name: Wait for TXT entry to appear
  community.dns.wait_for_txt:
    records:
      - name: "{{ com_challenge.challenge_data[fqdn]['dns-01'].record }}"
        values: "{{ com_challenge.challenge_data[fqdn]['dns-01'].resource_value }}"
        mode: equals
    always_ask_default_resolver: false
    timeout: 120

Purpose:
Waits for the TXT record to propagate and be accessible via DNS queries.

Key Parameters:

records: List of expected DNS records to appear.
timeout: Maximum time to wait for the records.

Explanation:
Ensures that the TXT record is properly propagated before proceeding with certificate validation.

10. Retrieve the Certificate After Validation

- name: Let the challenge be validated and retrieve the cert and intermediate certificate
  community.crypto.acme_certificate:
    validate_certs: false
    account_key_src: "{{ le_private_key }}"
    account_email: "{{ email }}"
    src: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.csr"
    cert: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.crt"
    fullchain: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}-fullchain.crt"
    chain: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}-intermediate.crt"
    challenge: dns-01
    remaining_days: 60
    data: "{{ com_challenge }}"
    acme_directory: https://acme-v02.api.letsencrypt.org/directory
    acme_version: 2
  when: com_challenge is changed
  register: lsresult
  until: "lsresult is not failed"

  retries: 12
  delay: 10

Purpose:
Validates the DNS-01 challenge and retrieves the certificate from Let’s Encrypt.

Key Parameters:

data: Challenge data to be validated.
fullchain: Path to save the full certificate chain.
chain: Path to save the intermediate certificate.
until, retries, delay: Controls for retry logic.

Explanation:
Finalizes the challenge and obtains the certificate, including the full chain and intermediate certificates.

Condition: Only runs if com_challenge has changed.

Output Registered As: lsresult

11. Remove TXT Record from Cloudflare

- name: Remove TXT record from Cloudflare after domain is verified
  uri:
    url: "https://api.cloudflare.com/client/v4/zones/{{ zone_id }}/dns_records/{{ txt_record_id }}"
    method: DELETE
    headers:
      X-Auth-Email: "{{ email }}"
      Authorization: "Bearer {{ API_key }}"
      Content-Type: "application/json"
    status_code: 200
  when: txt_record_id is defined and txt_record_id | length > 0

Purpose:
Deletes the TXT DNS record from Cloudflare after the domain has been successfully validated.

Key Parameters:

url: API endpoint to delete the DNS record in Cloudflare.
method: HTTP method (DELETE).
headers: Authentication headers including the API key.

Explanation:
Removes the temporary DNS record used for the DNS-01 challenge to clean up.

Condition: Only runs if txt_record_id is defined and has a valid length.

The completed Playbook should look like this:

---
- name: Create Letsencrypt cert
  hosts: localhost
  gather_facts: false
  vars_prompt:

    - name: fqdn
      prompt: Enter domain name for the new Certificate
      private: false

  vars_files:
    - vaultvars.yml
    
  vars:
    files_loc: ~/cert-store
    le_private_key: "{{ files_loc }}/le-account.key"
    csr_private_key: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.key"
    email: your_email@example.com
    dns_zone: "{{ fqdn.split('.')[1] }}.{{ fqdn.split('.')[2] }}"

  tasks:

    - name: Create directory for certs directory
      file:
        path: "{{ files_loc }}/{{ fqdn }}"
        state: directory

    - name: Generate Letsencrypt private key
      community.crypto.openssl_privatekey:
        path: "{{ le_private_key }}"
        state: present

    - name: Generate CSR private key
      community.crypto.openssl_privatekey:
        path: "{{ csr_private_key }}"
        state: present

    - name: Make sure account exists and has given contacts. We agree to TOS.
      community.crypto.acme_account:
        validate_certs: false
        account_key_src: "{{ le_private_key }}"
        state: present
        terms_agreed: yes
        acme_version: 2
        acme_directory: https://acme-v02.api.letsencrypt.org/directory
        contact:
          - "mailto:nomail@{{ dns_zone }}"

    - name: Generate an OpenSSL Certificate Signing Request
      community.crypto.openssl_csr:
        path: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.csr"
        privatekey_path: "{{ csr_private_key }}"
        common_name: "{{ fqdn }}"

    - name: Create a challenge using an account key file.
      community.crypto.acme_certificate:
        validate_certs: false
        account_key_src: "{{ le_private_key }}"
        account_email: "{{ email }}"
        src: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.csr"
        cert: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.crt"
        challenge: dns-01
        acme_directory: https://acme-v02.api.letsencrypt.org/directory
        remaining_days: 60
      register: com_challenge

    - name: Create a TXT record on Cloudflare when the record file exists
      uri:
        url: "https://api.cloudflare.com/client/v4/zones/{{ zone_id }}/dns_records"
        method: POST
        body_format: json
        headers:
          X-Auth-Email: "{{ email }}"
          Authorization: "Bearer {{ API_key }}"
          Content-Type: "application/json"
        body:
          type: "TXT"
          name: "{{ com_challenge.challenge_data[fqdn]['dns-01'].record }}"
          content: "{{ com_challenge.challenge_data[fqdn]['dns-01'].resource_value }}"
          ttl: 1
        status_code: 200
      when: com_challenge.challenge_data[fqdn]['dns-01'].record is defined
      register: record

    - name: Extract and store the record ID from the creation response
      set_fact:
        txt_record_id: "{{ record.json.result.id }}"
      when: record is defined and record.json.success

    - name: Wait for TXT entry to appear
      community.dns.wait_for_txt:
        records:
          - name: "{{ com_challenge.challenge_data[fqdn]['dns-01'].record }}"
            values: "{{ com_challenge.challenge_data[fqdn]['dns-01'].resource_value }}"
            mode: equals
        always_ask_default_resolver: false
        timeout: 120

    - name: Let the challenge be validated and retrieve the cert and intermediate certificate
      community.crypto.acme_certificate:
        validate_certs: false
        account_key_src: "{{ le_private_key }}"
        account_email: "{{ email }}"
        src: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.csr"
        cert: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}.crt"
        fullchain: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}-fullchain.crt"
        chain: "{{ files_loc }}/{{ fqdn }}/{{ fqdn }}-intermediate.crt"
        challenge: dns-01
        remaining_days: 60
        data: "{{ com_challenge }}"
        acme_directory: https://acme-v02.api.letsencrypt.org/directory
        acme_version: 2
      when: com_challenge is changed
      register: lsresult
      until: "lsresult is not failed"
      retries: 12
      delay: 10

    - name: Remove TXT record from Cloudflare after domain is verified
      uri:
        url: "https://api.cloudflare.com/client/v4/zones/{{ zone_id }}/dns_records/{{ txt_record_id }}"
        method: DELETE
        headers:
          X-Auth-Email: "{{ email }}"
          Authorization: "Bearer {{ API_key }}"
          Content-Type: "application/json"
        status_code: 200
      when: txt_record_id is defined and txt_record_id | length > 0

3. Running the Playbook

Execute the playbook to request a new Let’s Encrypt certificate.

ansible-playbook request_cert.yml --ask-vault-pass

You’ll be prompted to enter the vault password and the FQDN. Ansible will use the encrypted API key and DNS Zone ID to create the necessary DNS records and request the certificate.

4. Verification

Check the specified directory (~/cert-store/<fqdn>) to verify that the certificate and key files have been created.

Summary

Each task in this playbook is part of a sequence that ensures:

Directory and key creation.
Account registration and CSR generation.
DNS-01 challenge setup and validation via Cloudflare.
Certificate issuance.
Cleanup of temporary DNS records.

I hope this breakdown helps you understand the specific purpose and function of each task, providing a step-by-step approach to requesting a Let’s Encrypt certificate using Ansible and managing DNS challenges via Cloudflare.

Conclusion

In this tutorial, you learned how to use Ansible Core to request a Let’s Encrypt certificate, securely storing sensitive information using Ansible Vault. This is a foundational step towards more advanced certificate lifecycle management and automation in subsequent tutorials.

Next Steps

Part 2: Managing Certificates with AWX/Ansible Automation Platform with GitOps
Part 3: Scheduling Auto-Renewal workflows and automating the deployment of certificates to Firewalls and F5’s
Part 4: ServiceNow Integration
BONUS : Using Thousandeyes with Event Driven Ansible to renew Certs before they Expire.

Stay tuned for the next part of the series, where we will be cover Managing Certificates with AWX/Ansible Automation Platform.

Part 1: Basics of Using Ansible Core to Request a Let’s Encrypt Certificate#

Prerequisites#

Steps#

1. Setting Up Ansible Vault#

1.1 Create the Vault File#

1.2 Add Sensitive Variables#

1.3 save and Exit#

2. Writing the Ansible Playbook#

2.1 Define the Playbook#

2.2 Explanation#

Playbook Structure#

Lets Breakdown Each Task#

1. Create Directory for Certificate Files#

2. Generate Let’s Encrypt Account Private Key#

3. Generate CSR Private Key#

4. Ensure Let’s Encrypt Account Exists and Agrees to TOS#

5. Generate an OpenSSL Certificate Signing Request#

6. Create a Challenge Using an Account Key File#

7. Create a TXT Record on Cloudflare#

8. Extract and Store the Record ID#

9. Wait for TXT Entry to Appear#

10. Retrieve the Certificate After Validation#

11. Remove TXT Record from Cloudflare#

The completed Playbook should look like this:#

3. Running the Playbook#

4. Verification#

Summary#

Conclusion#

Next Steps#

Part 1: Basics of Using Ansible Core to Request a Let’s Encrypt Certificate

Prerequisites

Steps

1. Setting Up Ansible Vault

1.1 Create the Vault File

1.2 Add Sensitive Variables

1.3 save and Exit

2. Writing the Ansible Playbook

2.1 Define the Playbook

2.2 Explanation

Playbook Structure

Lets Breakdown Each Task

1. Create Directory for Certificate Files

2. Generate Let’s Encrypt Account Private Key

3. Generate CSR Private Key

4. Ensure Let’s Encrypt Account Exists and Agrees to TOS

5. Generate an OpenSSL Certificate Signing Request

6. Create a Challenge Using an Account Key File

7. Create a TXT Record on Cloudflare

8. Extract and Store the Record ID

9. Wait for TXT Entry to Appear

10. Retrieve the Certificate After Validation

11. Remove TXT Record from Cloudflare

The completed Playbook should look like this:

3. Running the Playbook

4. Verification

Summary

Conclusion

Next Steps