We spoke with search industry professionals and innovators about persistent challenges, trending opportunities, and the technologies people and businesses are using to stay relevant in competitive search results.
A trend driving massive advancements in search technology is the shift from keywords to data that better represents the meaning of the query and what is known about it.
Keyword research has been driving content discovery since 1230 AD. It was then that the French cardinal and biblical commentator, Cardinal Hugh de St Cher, completed the first known index in history.
Vector search marks a major shift from this traditional method of finding information to a future in which all of the complex data that makes up modern content assets can be leveraged.
So what do you need to know about it right now?
We reached out to Edo freedomthe former head of Amazon’s artificial intelligence lab and now CEO of Pinecone, for an introduction to vector research and why you might want to have related technologies on your radar.
We asked Liberty:
- How will vector search redefine traditional keyword research?
- How would you explain vector research to a 5-year-old child?
- What are some of the challenges you’ve encountered when using ML algorithms for Amazon Web Services (AWS) customers and how have you overcome them?
- What is Pinecone and what is it used for?
- What advice or tips do you have for SEO beginners who are just entering the world of ML and AI?
Let’s start with this: why is natural language processing (NLP) so important to the future of SEO, and how can marketers prepare for what’s next?
We burned the ships of keyword research
Freedom of Edo: “Just as SEOs have mastered the PageRank algorithm, they now need to know NLP to be successful and beat the competition.
Unlike PageRank, however, the field of NLP is growing rapidly and has thousands of contributors.
This will take more effort than following Matt Cutts (from Google) on Twitter and tracking SERP changes.
Fortunately, although NLP is a more complicated subject, it is not shrouded in mystery like PageRank is.
Much of the work in NLP is done in the open, with free and plentiful research materials, open source software, and NLP online course.
One thing is clear about NLP: it’s here to stay.
It’s far from perfect, but it’s improving fast, and big tech companies have burned the ships of keyword research and there’s no turning back.
Vector search allows us to search the way we speak
How will vector search redefine traditional keyword research?
Freedom of Edo: “Vector research does not redefine the search by keyword; it replaces it all-canvas.
Instead of working with keywords – and their synonyms and misspellings – vector search works with vector embeddings.
It is a piece of data that represents the meaning of the search phrase as well as other known information about the query or the user.
(To a human, vector integration is unrecognizable and looks like a long array of numbers.)
This representation of the search phrase and user is then used to sort through massive collections of embeds that represent other content and user preferences to find the most relevant result.
From the user’s point of view, this means that he can search as he speaks.
They no longer need to learn the quirks and syntax of search engines.
From an SEO perspective, this means they can really focus on themes and topics without worrying about specific keywords.
How would you explain vector research to a 5 year old?
Freedom of Edo: “Our article explaining basics of vector research approximate.
The ELI5 version, as I’ve practiced it in my own family, is this: If I say “Italian food,” you might think of pizza or pasta.
You learned that these things are related because you remember eating pizza at an Italian restaurant or learning that pasta is popular in Italy.
But a computer has never learned that. So, the phrase “Italian food” means exactly that and doesn’t contain any information that it’s related to pasta or pizza.
So when I ask a computer to search for “Italian restaurant”, it may omit pizzerias.
Machine learning is a way to help computers understand the meaning of what we say or type.
And vector search is a way for these computers to search through everything they know, based on meaning, not exact words.
So now if I ask the computer to recommend an Italian place, it might suggest your favorite pizzeria just like you would.
Organizations can finally focus on creating and curating content for humans.
Several thousand scientists and engineers work tirelessly to make ML and NLP look like the human mind.
Do you really want to go against this? The winning strategy for SEO is to optimize for the human mind.
Overcoming Machine Learning Challenges
What are some of the challenges you’ve encountered when using ML algorithms for Amazon Web Services (AWS) customers and how have you overcome them?
Freedom of Edo: “I can’t speak to specific AWS projects or challenges. I can say more broadly, from my experience I have seen that ML algorithms are no longer the bottlenecks.
Granted, they’re far from perfect and there’s a lot of work to be done, but that work is being done at breakneck speed.
The next challenge is to run these algorithms at the scale needed to support consumer products and enterprise applications.
The representations I mentioned earlier, vector embeddings, are computationally expensive to navigate.
An index of only 1 million elements (vector embeddings) already requires specialized software as well as careful tuning; an index of 100 million articles requires specialized software and infrastructure; an index of 1 billion items or more requires you to be Google or Amazon.
(By the way, that’s why I started Pinecone: to make it easy for engineering teams to add vector search to their apps.)”
What is the pine cone?
What is Pinecone and what is it used for?
Freedom of Edo: Today, Pinecone makes it easy for engineers to create fast, fresh, filtered vector research in their applications.
It gives engineering teams the research infrastructure needed to run vector research at scale, all bundled in a managed service with a simple API.
(We removed version numbers because releases come quickly and because as a managed service, users always get the latest version and don’t have to worry about updates.)
Working with algorithms is extremely fun and absolutely worth the challenges.
With vector search, we are at the intersection of state-of-the-art algorithms, database architectures and serverless applications.
And we see our customers applying this technology to products that are revolutionizing consumer and enterprise applications such as semantic search, recommender systems, computer security, wearable devices, computer vision, and more.
First steps in ML and AI
What advice or tips do you have for SEO beginners who are just entering the world of ML and AI?
Freedom of Edo: “Don’t feel intimidated. Even the most brilliant researchers in this field “get it”.
Learning more about AI/ML beyond the surface articles will make you a better SEO professional, and there are plenty of free resources that will help you do just that.
For those interested in careers in this field, we are currently hiring in all teams: engineering, research, customer success, sales, marketing and operations.
Featured Image: Courtesy of Pinecone