The case for AI Embedded coding
Old habits die hard — and todays AI coding assistants — which I love BTW — are great, but not necessarily best at using the power of AI Assistants instead of code.
Let me share a recent example that got me thinking. In our app, we import company-wide Fireflies transcripts of conversations we have with startups every day. We need to store each transcript with the right startup. This is trickier than it seems because from the Fireflies API we only get a transcript, a summary, and a list of participants.
Examples:
Meeting 1: Participants: jl@valor.vc, john@acme.com
Meeting 2: Participants: jl@valor.vc, john.doe@gmail.com
Meeting 3: Participants: jl@valor.vc
Meeting 4: Participants: <empty>In example 2- John logged in with his personal gmail, not his business email. In Meeting 3, for unknown reasons we did not get a other participant email at all and in meeting 4 we received no participant info. In all the cased above we do have a short summary something like:
MeetingSummary : "Meeting with John Do at Acme corp about a possible investment"In my original code, I wrote code that used the emails of the participants to look up contacts in our database, and if nothing was found that way next I would try to find the company by using the website URL. It looked something like this:
contact = None
valor = transcript['user']['email']
participants = ''
for c in transcript['participants']:
if ',' in c:
continue
participants += c + " "
c = c.upper()
if 'VALOR.VC' in c:
# if we don't have the Valor contact yet save this one
if valor == None:
valor = c
elif AccountID == None:
# try to find the account by looking up email
contact = getContact({'Email': c})
if contact.get('ContactID', None) != None:
AccountID = contact.get('AccountID', None)
company = getCompany({'AccountID': AccountID})
elif not c.split('@')[1].upper() in ['GMAIL.COM', 'OUTLOOK.COM', "BELLSOUTH.NET", "YAHOO.COM", "AOL.COM", "HOTMAIL.COM", "ICLOUD.COM", "MSN.COM", "COMCAST.NET", "ATT.NET", "SBCGLOBAL.NET", "VERIZON.NET", "COX.NET"]:
company = getCompany({'Website': c.split('@')[1]})
if isinstance(company, list):
AccountID = company[0].get("AccountID", None)
if AccountID != None and valor != None:
# we found both the valor contact and we have the Account id
breakThere are many ways this can fail. Depending on the platform, the participants may not be available. The participant might use a different email than the one in our contact database. Or the person might not yet exist in our database. After using this for several months, we noticed we were missing too many transcripts automatically (25%), so I wanted to improve our matching rate.
The code turns out to be a great example of why an AI Assistant version can do much better. When provided with the same information (the list of participants as text, plus a short summary of the transcript) and the same lookup functions to find a contact or a company we only have to prompt it how we want it to look for the company id.
Here is the prompt I used to build a version that handles all the transcripts that could not be identified with the hardcoded version:
Your job is to find the account based on a meeting summary.
Your clues are the people in the meeting (participants) and the company name.
The spelling of the company might not be correct and the same is true for
participants.
You will ignore any participant that are with Valor ventures (email valor.vc)
Try finding the company based on company name.
If the company is not found using getCompany, use the non-Valor participants'
email to find a contact
If found, the account is the account that belongs to the contact.
If you can't find the person by email, try with the contact name (only).
If still not found, consider retrying the company find.
Sometimes a company name is spelled as two words internally and one word
in the summary, or vice versa.
Also worth trying to search using the company name as the website URL
(especially in case of .AI type of names).
Make sure that data received from the search (company, description, record type)
makes sense with the transcript summary. If in doubt, return None for Company__c.
You will return "Company__c" with the Salesforce ID of the ACCOUNT that was found.
(Never the contact ID!)
Your response will be in JSON: "Company__c": None if not found
or the Salesforce account ID.We’re using gpt-4o to output JSON. Here’s an example of the output:
{
"Company__c": "001Jw00000Uj4uDIAR"
}This only works, of course, because I already had functions for finding a contact and finding a company that I could just plug into this Assistant. Here’s the getCompany and getContact template for function calling:
{
"name": "getCompany",
"description": "Find a company in Valor's Salesforce, by company name or website. Company name is preferred, for website use only the domain name.",
"strict": false,
"parameters": {
"type": "object",
"description": "A company in the Valor Salesforce Database.",
"properties": {
"Name": {
"description": "The name of the company",
"type": "string"
},
"Website": {
"description": "Company Website",
"type": "string"
}
},
"required": [
"Name"
]
}
}{
"name": "getContact",
"description": "Find a contact in Salesforce using email address, or full name + company name. Email is preferred.",
"strict": false,
"parameters": {
"type": "object",
"properties": {
"FullName": {
"type": "string",
"description": "can be used to search by name"
},
"Email": {
"type": "string"
},
"Company": {
"type": "string",
"description": "Company Name"
}
},
"required": ["Email"]
}
}I think it is easy to see how the Assistant version is a ‘higher level’ code, ie like going from Assembler to C. The most amazing part is how much better the prompt version is at the doing the job, compared to the original ‘hardcoded’ Python version.
The code to execute the Assistant is minimal:
def get_account(request, id: str):
meeting = FirefliesMeeting(id=id)
if meeting != None and meeting.Summary__c != None:
# run Assistant to find account
task = assistantTask(assistantName="Vic Fireflies Meeting Account Finder",
tools=["chatbot.salesforce:getCompany", "chatbot.salesforce:getContact"],
metadata={"id": meeting.id,
"sobject": "Fireflies_Meetings__c"
},
completionCall='chatbot.screen:updateRunwayApplication')
task.prompt = "Find the salesforce account id for this meeting. " + (f"Participants: {meeting.Participants__c}" if meeting.Participants__c != None else "") + " " + (f"Summary: {meeting.Summary__c}" if meeting.Summary__c != None else "")
task.createRun()
return {'company_id': meeting.company_id, 'fireflies_id': meeting.fireflies_id}
else:
return {'status': 'not found or empty summary'}(See my previous article for running Assistant prompts on Django/Celery.)
Because we’re using the strict JSON output, the Assistant will always return the JSON that I expect and I afterwards I simply update the record with the account id provided.
This shift reminds me of how moving from Assembly to C made programming more accessible, allowing us to express higher-level concepts without managing every instruction. The leap to interpreted and runtime-compiled languages further abstracted away memory management, letting us focus on solving problems instead of implementing details like memory management.
Now what I call AI embedded code is pushing that abstraction even further. Instead of only writing precise ‘if then’ type code, we’re blending prompted parts of the code. It’s funny how we might finally be delivering on the ‘low code/no code’ promise!
The same arguments against AI embedded code can be made as those against using any higher-level code compared to compiled or assembly languages: it requires more computational “cycles” and scales differently. So, it’s true that this might not yet be ideal for processing millions of scripts. Additionally, the cost is around 2,000 tokens per transcript, which comes to roughly half a cent. But it is way easier to read (and update!) and the Python code works only for 75% or so of the cases where a human would be able to find the account) while the AI Assistant version handles 97% of the cases with just that fairly simple prompt.
The revelation here for me was not so much the solution — I already had quite a few similar prompts in my codebase — but the fact that I did not create the Assistant based solution the first time around! We are clearly creatures of habit and looking at the code version now I have a hard understand why I ‘hardcoded’ it in the first place.
I need to keep AI Embedded coding top of mind. (And funnily enough the AI Assisted code is NOT good for that because it will dutifully churn out fresh ‘old fashioned’ code)