Targeted recommendations with LLMs
How to approach personalized e-commerce
Context
AI and machine learning methods have been present in e-commerce and product recommendations for years. There are numerous implemented and proven solutions aimed at helping potential buyers find interesting products and increasing sales.
Nevertheless, the relatively recent advent of Large Language Models (LLMs) opens new opportunities to leverage GenAI to make recommendations more personalized and targeted.
Previous methods can be seen as a shop assistant who tells customers, "Other people (similar to you) usually buy this." In contrast, LLMs can be advisors who first understand your situation and needs (what problem you have, why you are here) and then make recommendations.
In fact, LLMs allow the combination of search, filtering, and recommendations (typical elements of an online store or marketplace) into a single co-working mechanism, achieving a synergistic effect. In the future, I will explore this aspect in a dedicated blog post.
Introduction
Recently, I have been exploring different methods to approach personalized recommendations with LLMs, and in this post, I would like to present a couple of ideas and share observations after implementing them as proof-of-concepts. I hope that you will find them useful. Also, if you are interested in the topic or a business cooperation, feel free to reach out to me.
Four Different Approaches
The PoCs focused on recommending electronic equipment from a base of thousands of items. All of the presented approaches can be extended to cover more product categories and support a larger inventory. However, some of them can be more suitable for it.
Below, you will find 4 paths with their strong and weak sides - taking into account recommendation quality, possibility to scale, potential cost of usage and maintenance.
Prompt Chaining and Pre-filtering
Initially, I thought that this approach would be the weakest. Surprisingly, it worked quite well. It has two main steps. First, transforming a user need to a set of requirements. Second, based on the requirements, finding matching products.
Defining the requirements should lead to a structure similar to how the data is stored by asking LLM to return output in the right JSON format. On top of that, we can enforce enum type values for some fields to make the filtering easier.
During the requirements phase, LLM reveals its potential by being able to extract a lot of details thanks to having built-in vast general knowledge. For example, when someone looks for a laptop and mentions just its occupation, the LLM can figure out which software will be used and what specs it needs, etc.
If we want to support many different product categories, the prompt chain could look like this:
Define the searched product category.
Generate a set of requirements matching user needs and available data schema for the given category.
The next phase is the pre-filtering, which does not use LLM and is possible thanks to matching the requirements with the data schema. It can be done at the database or code level. The goal of the pre-filtering is to narrow down the list of potential recommendations to roughly 50-100 products. In case of having no products fulfilling all of the requirements, fuzzy search methods can be used.
The last step involves again LLM, which is prompted to pick top recommendations based on the previous prompts and pre-filtered products (which are included in the final prompt).
Pros
High quality of the recommendations, especially when products belong to the same or similar categories and their attributes are crucial to finding a good match.
If the pre-filtering is well designed, the cost per recommendation is low.
Cons
Mismatches between requirements returned by the LLM and the data schema may break the pre-filtering. Thus, it may require more development or maintenance, especially when there are many very different product types.
Vector Database
Similar to the previous method, first, we translate the user's needs into requirements using the LLM. Then, we use vector similarity search to look for matching products. In the vector database, products are stored as encoded JSONs, so the product vectors contain information about available attributes.
As an extension, we could encode some of the product parameters as metadata in the vector DB so we could leverage pre-filtering (as in the prior approach) to improve recommendation quality.
Pros
Simple development and logic flow, causing reduced maintenance.
May work well in scenarios where there is a high variety between product types and buyer choice is more subjective.
Cons
Poor quality of recommendations when there are many similarities between products and specific details are crucial.
Required vector DB as an additional infrastructure.
RAG with OpenAI Assistant API
OpenAI has opened the Assistant API as a beta, providing a very interesting functionality - RAG - which seems to be very handy in the product recommendation scenario.
In this approach, we follow a similar path as in the first method. We convert the user’s needs into product requirements with LLM and then use a document retrieval tool to find the matching products.
In theory, it should work like a charm. In practice, it does not go as expected. The Assistant API is not as consistent as the standard ChatGPT completions in terms of keeping the requested answer format, resulting in worse recommendations and lower stability.
Pros
Simple development and logic flow, causing reduced maintenance.
Low costs of searching for products by LLM (even without any additional pre-filtering).
No need to host your own vector database.
Cons
Difficult to integrate with other components due to the instability in answers returned by the LLM and not keeping the requested format.
Common hallucinations - returning non-existing products as recommendations.
Mixing prompt chaining and filtering with Assistant API
Analyzing the pros and cons of the methods presented so far, we can notice that the weak points of Prompt Chaining and Pre-filtering can actually be addressed with the strengths of the RAG with OpenAI Assistant API. Both methods are complementary to each other, and we can try to distill the best from each of them.
First, we use standard ChatGPT completion to translate the user’s needs into product requirements. The standard completion keeps the requested format.
Second, we use the Assistant API to find products matching the requirements. By using OpenAI’s document retrieval tool, we keep the costs low and avoid the difficult-to-maintain pre-filtering. Moreover, to reduce hallucinations, we prompt LLM to return only a very minimal subset of product information (including ID), and we redefine our data schema to be more unique.
Lastly, based on the recommended product IDs by the Assistant API, we retrieve product data from a regular DB - ensuring that the products exist and their information is coherent and consistent.
Pros
High quality of the recommendations.
Low cost per recommendation.
Still, simple development and logic flow, causing reduced maintenance.
Minimized hallucinations.
Easy integration with other components.
Suitable for all types of inventories.
Cons
Rarely, some hallucinations may still occur, but treating them as “software exceptions” should make them transparent for the end-user who would still receive recommendations.
Final Thoughts and Next Steps
Observations:
OpenAI’s Assistant API is still in a beta phase, so over time it may overcome its limitations.
OpenAI’s document retrieval tool seems to work better than a regular vector database (like Chroma DB).
Addressing hallucinations requires a couple of tricks, but it is possible.
To materialize the targeted and personalized recommendations, we also need to rethink UI/UX and buyer feedback when the recommendations are presented. Also, all these buyer interactions, connected with AI, can be a rich source of analytical data if addressed in the right way.