GPT Knows My Salesforce Opportunities

in AI on October 26, 2023

By Charles McGuinness

See how chats can be a lot more interesting!

To avoid burying the lede, let me start with a chat session showing the app we’ll discuss in this post. In this session, all the CRM data is being brought in live from a Salesforce org:¹

Put simply, you ask a question of GPT, it figures out what data it needs from Salesforce, gets it, and answers the question. What to know your top opportunities? It will find them. Want it to write an email to a contact on an opportunity? It will use the opportunity and contact data to craft a tailored message. Wondering if you’ve forgotten something? It will remind you of open tasks.

Did I teach it how to do these things? No, never mention in my prompt any of these use cases or how to do them. So how does it do this? Functions and glue.

To start things off, I’ll walk through the high level design (and challenges), and next week I’ll walk through the code.

ChatBot Overview

At a high level, there are four participants in this process:

Briefly, the ChatBot communicates directly with all the other participants (the user, GPT, and Salesforce) and intermediates the interactions between them (like the dotted line between GPT and Salesforce, which goes through the ChatBot in reality).

To make this a bit clearer, if you ask “What are my opportunities?” here’s what happens:

The ChatBot reads in your question and then sends it on to GPT. GPT analyzes the question and determines it needs to query opportunities from Salesforce. GPT itself cannot query Salesforce, so it returns a message to the ChatBot telling the ChatBot to fetch the data. The ChatBot then queries Salesforce for the Data, and sends the results back to GPT. At this point, using the data just fetched, GPT can now finish generating an answer to the original question. GPT returns the response to the ChatBot, which then passes the answer back to the user. Here’s a trace of executing that process with debugging turned on:

The Info messages walk us through the process:

1. Calling GPT
2. Getting a return with a SQL statement
3. Translating the SQL into SOQL
4. Getting results back from Salesforce
5. Passing the result set to GPT for a second pass to answer the questions
6. GPT responding with the final text for the user

While in the previous set of articles about functions we gave GPT access to narrowly defined functions, in this ChatBot our function just takes any ol’ query that GPT wants to send us. How does it know how to make SQL queries? That’s something that OpenAI added to it in the latest versions of their models. So we don’t have to do too much work² to get GPT to figure out what it needs to query to answer a user’s question.

Challenges

As straightforward as the design is, conceptually, there’s a few difficult problems that had to be dealt with in building this app:

1. SQL (the common language for relational databases) is not SOQL (Salesforce’s query language for its objects). The two are very close, but many common SQL statements are invalid SOQL statements. In most cases, the ChatBot can translate from the SQL that GPT generates into the SOQL that Salesforce can execute. In other cases we have to use our prompts to discourage GPT from generating SQL we cannot translate.
2. Because of some of the differences between SQL and SOQL, GPT is not always going to put everything in its SQL statement that it might need in corresponding SOQL, so we have to do a bit of enrichment of the query at times.
3. Even though the data from Salesforce is queried on a focused, per-query basis and not shoved at GPT a priori, GPT still has to know what the schema looks like to generate its queries, which we do with a fairly complicated system prompt. Because there’s just no way to shove the entire Salesforce schema into the prompt, we have to be very selective³ about what we tell GPT about the data.

Next week I’ll talk about how to overcome these challenges and create a Salesforce ChatBot that can answer questions and offer suggestions to a sales rep.

1. The video has the delays between asking a question and getting an answered removed, so, yes, it went slower than this. Also, that’s not me typing, that’s a canned list of questions it’s working through. But, importantly, the data is really coming live from Salesforce. ↩︎
2. Maybve not too much, but definitely much.
   ↩︎
3. Not quite a pun. ↩︎