(AI) Natural Language Processing / ML

NLP / ML - Query Understanding

My Role
Lead Product Manager
Timeline
Fall

Problem

Strategy Challenge - What is the future of search in a world of recommendations?

Execution Challenge - How do increase the relevance of jobs for a given query?

Situation

Strategy

  • The team existed as a silo from other teams and had historical collaboration challenges
  • The outcome of the teams historical efforts were not tracked; therefore unable to be attributed to the teams' value
  • Historical focus was on keyword search not intent-driven experiences

Execution

  • Relevance can be improved through optimizing head vs. tail queries with different strategies and the same expected lift
  • Team had many microservices powering various parts of the search experience and using different data sources and strategies (i.e related search logic being popularity and recency based rather than ontology-driven)
  • Ontological relationships within our taxonomy did not exist

Contributions

1. Proved value of team and our assets through tracing, blackout tests on services, and discovery conversations
2. Evangelized historical outcomes and charted out possible future applications of assets to prove future value
3. Defined "intent" for the product portfolio and built long-term product portfolio strategy focused on discovery and an ontology-driven search experiences powered by a unified knowledge-graph
4. Supported and analyzed relevance experiments ran by our taxonomy team

Outcome

  • Aligned partner PM teams
  • Receive buy-in from leadership on strategy
  • Led to a promotion and +20% pay increase
  • X% lift in job relevance score

Methodology

Manual labeling, Bert Models, Taxonomies, Ontologies, Decision trees, Nearest neighbor