Automated Meta Lead Form processor

Last Updated on November 18, 2024

The missing link of Meta to SFMC integration

Traditional methods of collecting lead data from Facebook and Google forms often create data silos or bottlenecks, making it challenging to act on this high-value first-party data. Without a streamlined process, businesses face delays in engagement, inconsistent data quality, and compliance risks related to identity and consent management. These inefficiencies lead to missed opportunities to engage prospects promptly with the right message. The inability to connect this data seamlessly to Salesforce Marketing Cloud (SFMC) and external systems like Salesforce Sales Cloud or consent management platforms can result in poor customer experiences, reduced ROI on ad spend, and potential non-compliance with privacy regulations.

How can you solve this?

This Automated Meta and Google Lead Form Processor, which I have been use for multiple implementations, offers a robust solution to these challenges. By leveraging SSJS for validation and data transfer, it ensures the efficient flow of leads from Meta (Facebook) and Google lead forms into SFMC and beyond. Leads generated via Facebook and Google lead forms are automatically funneled into data extensions created within SFMC. The process incorporates validation steps to ensure that data extensions created as part of the native Meta integration in SFMC have custom fields added to support with the integration. Once validated, the leads are securely transferred to external systems of record, such as Salesforce Sales Cloud for CRM management or consent management platforms for regulatory compliance.

How does it benefit you?

This automation enables you to activate intent-based audiences by syncing validated leads with paid media channels, such as Facebook Custom Audiences and Google Customer Match, for retargeting and lookalike campaigns. At the same time, leads are entered into nurture journeys in SFMC, ensuring personalized and timely engagement. This integrated approach eliminates manual data transfers, streamlining the entire process from lead collection to activation.

By adopting this system, you can capture high-value first-party data seamlessly and activate it in near real-time to drive better marketing results. I’m writing “near real-time” as this script is executed in Automation Studio, and you will need to rely on the limitations of it’s scheduling. But still, the solution ensures high-quality data reaches marketing and sales teams quickly, allowing for faster and more relevant engagement with prospects.

This scalable and efficient approach transforms lead management, maximizing the potential of paid media campaigns and enabling businesses to build stronger, long-term customer relationships. By connecting Facebook and Google lead ad campaigns to SFMC through a secure and intuitive configuration flow, businesses can significantly improve their ROI and ensure a seamless customer journey.

Technical details

The solution is forming a base for pushing leads created by native Lead Form integration with SFMC, to a destination of your choice (in this implementation we are utilising Sales Cloud API). It consists of two files (complete files are available on the bottom of the page): validate_dataextensions.js and upload_leads.js
For each lead form integrated with SFMC, a data extension is automatically created, which will receive lead information from Meta. As this is technically a “dead end” we need to build and facilitate a process responsible for picking up these records and pushing them to a relevant system of record. We want to add two fields into these data extensions, to keep track on which records have been processed, and their respective IDs in Sales Cloud.

Step 1

This process is divided into two steps. Initial step (validate_dataextensions.js) performs validation and modification of Data Extensions (DEs) within a specified folder. It needs to be run more infrequently (once per day), as we don’t expect new data extensions to be created that often:

1. Core Library and Proxy Setup
   Platform.Load("Core", "1.1.1");
   var proxy = new Script.Util.WSProxy();
  • Loads the required core library for advanced scripting in Marketing Cloud.
  • Initializes a Web Services Proxy (WSProxy) object for interacting with platform objects like Data Extensions.
2. validateDE Function
   function validateDE(customerKey) {

This function checks if specific fields (createdincrm, crmcontactid, crmconsentid) exist in a Data Extension and adds them if they are missing.

  • Retrieve Existing Fields: var fieldlist = proxy.retrieve("DataExtensionField", cols, { Property: "DataExtension.CustomerKey", SimpleOperator: "equals", Value: customerKey }); Retrieves all fields of the Data Extension using its CustomerKey.
  • Check for createdincrm: if (fieldname.indexOf('createdincrm') > -1) { var isCRMready = true; } Scans through the fields to see if a field named createdincrm exists.
  • Add Missing Fields: var de = DataExtension.Init(customerKey); var add1 = de.Fields.Add(crmcontactid); var add2 = de.Fields.Add(crmconsentid); var add3 = de.Fields.Add(createdincrm); If the createdincrm field is missing, the function adds the three fields (crmcontactid, crmconsentid, and createdincrm) to the Data Extension.
  • Return Messages:
    The function returns a message indicating whether the fields were already present or added.
3. checkFolder Function
   function checkFolder(folderid) {

This function processes all Data Extensions within a specific folder (identified by folderid).

  • Retrieve Data Extensions in the Folder: var dataextensions = proxy.retrieve("DataExtension", ["Name", "CustomerKey"], { Property: "CategoryID", SimpleOperator: "equals", Value: folderid }); Retrieves a list of Data Extensions and their CustomerKey values from the specified folder.
  • Validate Each Data Extension:
    javascript validateDE(ckey);
    Loops through the Data Extensions and calls the validateDE function for each, ensuring the required fields are present.
4. Execution
   checkFolder(834477);

The script runs the checkFolder function for a folder with the ID 834477.

Purpose

This script ensures that all Data Extensions in the specified folder:

  1. Contain the required fields (createdincrm, crmcontactid, crmconsentid).
  2. Are modified dynamically if these fields are missing.

Step 2

Second step (upload_leads.js) automates the process of uploading leads from a specified Data Extension (DE) to Salesforce CRM. This shoudl preferably run more frequently, to reduce the latency between a record arriving from Meta and until it arrives in Sales Cloud:

Key Functional Components

1. Load Core Library and Initialize WSProxy
Platform.Load("Core", "1.1.1");
var proxy = new Script.Util.WSProxy();
  • Loads the core SSJS library for interacting with Marketing Cloud.
  • WSProxy is used to interact with platform objects like Data Extensions efficiently.
2. OAuth2 Authentication for Salesforce CRM
var loginurl = "https://login.salesforce.com/services/oauth2/token?grant_type=password&client_id=" + clientId + "&client_secret=" + clientSecret + "&username=" + username + "&password=" + password;
  • The script authenticates with Salesforce CRM using OAuth2 to retrieve an access token (access_token) for API requests.
  • The token is stored and used for subsequent API calls.
3. Helper Functions
  • formatDate(date) and getTimestamp(dt)
  • These functions format JavaScript Date objects into the required formats for API payloads and logging.
4. uploadtocrm(dataextensionkey) Function

This is the core function that uploads leads from a Data Extension to Salesforce CRM:

Retrieve Rows to Process:

  var Filter = {Property:"createdincrm", SimpleOperator: "isNull"};
  var Rows = leads.Rows.Retrieve(Filter);
  • Fetches rows where the createdincrm field is NULL, indicating they haven’t been uploaded yet.

Prepare Lead Payload:

  var leadpayload = {
    "firstName": Rows[i]["first_name"],
    "email": Rows[i]["email"], 
    "marketID": country,
    "marketLanguage": lang,
    "interests": []
  };
  • Extracts lead information (e.g., name, email, market details) from the Data Extension row.
  • Sends the payload to the /services/apexrest/consumer endpoint to create a lead in Salesforce CRM.

Handle Response:

  var createJSON = Platform.Function.ParseJSON(String(createlead.content));
  var contactid = createJSON.contactId;
  • Retrieves the contactId from the CRM response, which is required for consent creation.

Create Consent Record:

  var consentpayload = {
    "verifiedDt": getTimestamp(createdincrm),
    "verified": true,
    "type": "Email", 
    "marketId": country, 
    "contactId": contactid
  };
  • Sends a consent payload to the /services/apexrest/consent endpoint in Salesforce CRM.

Update Data Extension Row:

  var update = leads.Rows.Update({"createdincrm": getTimestamp(createdincrm), "crmcontactid": contactid, "crmconsentid": consentId}, ["Lead ID"], [leadid]);
  • Marks the lead as processed by updating createdincrm, crmcontactid, and crmconsentid in the Data Extension.
5. collectLeads(folderid) Function
var dataextensions = proxy.retrieve("DataExtension", ["Name", "CustomerKey"], {
  Property: "CategoryID",
  SimpleOperator: "equals",
  Value: folderid
});
  • Retrieves all Data Extensions in a specific folder identified by folderid.
  • For each Data Extension, it calls uploadtocrm to process leads.
6. Execution
collectLeads(834477);
  • Processes all Data Extensions in folder 834477 by calling uploadtocrm for each.

Purpose of the Script

  • Automates lead management by:
    • Uploading unprocessed leads from Marketing Cloud to Salesforce CRM.
    • Creating consent records for these leads.
    • Updating the Data Extension to mark processed rows.
    • Ensures consistency between Marketing Cloud Data Extensions and Salesforce CRM.

Files

validate_dataextensions.js

upload_leads.js

Scroll to Top