While many eCommerce search queries today are just one to two words, younger people tend to search differently. They often use complete sentences and look for contextual results that match their intent. This shift requires a different approach to search that goes beyond keywords to understand the meaning of shopper’s requests.
New Search Algorithms
Rather than matching keywords, new eCommerce search engines base their algorithms on Vector Search, which uses neural networks aided by natural language processing to analyze a query. Vector search uses distances in the embedding space to represent similarity. Finding related data becomes searching for the nearest neighbors of your query.
Vector embeddings are stored in multi-dimensional vectors and numerically represent data and related context. AI models that generate embeddings are trained on millions of examples to return more relevant and accurate results.
Vector embedding maps the words from the search to a corresponding vector to detect synonyms, intent, and ranking, and it clusters concepts to deliver more complete results. For example, the search query “fall wedding guest dresses for black tie event” would return relevant results for long dresses, dark colors, and sleeve options, even if the items weren’t all tagged with the exact keywords.
The idea at the core of a vector search engine is that similar data and documents have matching vectors. Thus, when you use vector embeddings and index both queries and documents with them, you find similar documents as the closest neighbors of your search query.
Vector search powers semantic or similarity search. Since the meaning and context are captured in the embedding, we can understand what users mean with vector search without requiring a precise keyword match. And, it works with both textual data and images. It can handle longer search queries and reduce the return of “no results” compared to keyword search alone. That technique makes it easier for eCommerce sites to match buyer intent, personalize the shopping experience, and answer questions.
The vector models are also great for product recommendations as they learn to recognize similar documents and their vectors in the embedding space. For example, an application may recommend products that other shoppers who purchased the same item also liked.
Other metrics can be combined with Vector Distances to achieve multiple goals. For example, product recommendations can be ranked by revenue potential and satisfaction scores.
When natural language processing (NLP) is combined with processing documents with text embeddings, AI models can deliver full-text answers to customers’ questions. That approach spares users from examining lengthy manuals and empowers merchant teams to provide answers faster.
A question-answering transformer model can learn from the documents from the knowledge base and FAQ to return the best answer.
Generative AI technologies like ChatGPT will seismically impact eCommerce search because they solve the critical problem of accurately matching user search intent with the right product results.
Presently, eCommerce search is primarily keyword-based. For example, if I search “What computer should I get?” on Amazon, the top three results are a book by Ramit Sethy, “I Will Teach You to be Rich”, a Lenovo laptop, followed by another book called “Shit I Can’t Remember”.
ChatGPT will fundamentally change the approach to search because its natural language processing system can understand user intent thanks to an unprecedented predictive model trained on nearly a trillion words.
Here’s is the proof: When I type “What computer should I get?” in ChatGPT, I get a detailed bullet-point response with top considerations for choosing a computer, such as intended use, operating system, portability, budget, and technical characteristics like memory and display size.
ChatGPT understands what I’m trying to find.
