Search at Meesho
For an e-commerce platform like Meesho, optimizing Search is pivotal to enhancing the user experience, improving product visibility and faster discovery, and eventually increasing conversion rates. Users search in multiple Indian languages looking for quality products at cheaper prices. Search at Meesho consists of four core components, each designed with specific objectives.
- The Query Understanding layer interprets user intent by processing search queries to perform entity extraction, language normalization, spelling correction, synonym expansion, and intent classification.
- The Retrieval layer, retrieves a broad set of potentially relevant products from multiple candidate generators such as ElasticSearch and Embedding Based Retrieval.
- This initial set is then refined through a Multi-Stage Ranking framework.
- The first layer or L1 ranker maximizes relevance and conversion rates at a global level, ensuring that the top-ranked products have broad appeal and relevance.
- Following this the L2 ranker optimizes for individual user conversion by aligning product relevance more closely with the unique preferences of the querying user.
In this blog we talk about L1 ranker and its evolution from a dual XGBoost model architecture to a unified Multi-Task deep learning model that leverages knowledge distillation from a BERT-based Cross Encoder model. Our approach adopts entire space multi-task learning formulation with Customized Gate Control (CGC) networks, and query and catalog embeddings which are trained on text data generated using multimodal LLMs. The final model is efficiently deployed using Triton Inference Server, delivering improved ranking performance while maintaining high computational efficiency.
2. Background: Two-Model XGBoost Approach
L1 ranker acts as an aggregator layer receiving candidates from multiple generators and picks the top candidates, optimising for both relevance and conversion. In our current setup these objectives were treated as separate problems, leading to suboptimal joint optimisation. The baseline system employed two separate XGBoost models
- CTR Model: Trained on impressions data to predict probability of click
- CVR Model: Trained only on clicks data to predict probability of purchase
The final ranking is done using a combination of the above models. While this separation allowed for targeted modelling, it suffered from several limitations:
- Sample Selection Bias: CVR models trained only on clicked samples fail to capture the full user behaviour spectrum. They are trained on clicked impressions but are inferenced on the entire dataset, including non-clicked impressions. This inconsistency between training and inference samples leads to poor generalisation, as the model struggles to accurately predict unseen samples.
- Data sparsity: Conversion events are significantly rarer than clicks, leading to insufficient training signal
- Disjoint optimisation: relevance and conversion are optimized in isolation, leading to a disconnect between what gets a user to click and what drives them to convert.This overlooks potential synergies since the tasks are sequential (impressions → clicks → conversions).
- Feature duplication and maintenance overhead due to separate pipelines.
- Adding unstructured data directly as a feature is a challenging task. For e.g. adding rich multi-modal (text and image) query and catalog embeddings and different variations instead of single dot product score.
3. Multi-Task Learning in Ranking
To address the limitations of the two-model setup, we developed a unified multi-task model that jointly predicts CTR and CVR. The architecture consists of following main components:
- Feature Encoding and Interaction: Dense features including embeddings, are projected and fused via a Transformer encoder to capture high-order interactions.
- Multi-task Output with CGC Networks: Customized Gate Control networks enable flexible routing between shared and task-specific experts.
- BERT-based Knowledge Distillation: A pretrained cross encoder provides soft relevance supervision to preserve semantic alignment during training.
- Multi-objective Optimization with Entire Space Formulation: CTR and CTCVR are jointly learned over the full impression space, enabling robust CVR estimation and mitigating selection bias.
3.1 Architecture
3.2 Feature Encoding and Interaction
3.2.1 Feature Engineering
Our model processes multiple numeric features along with sophisticated query-catalog embeddings. We developed multiple derived features which helped us to limit popularity bias and improve relevance on the platform, for e.g. we replaced query-catalog interaction features with query-category and query-price based features. E.g. users searching for a query “Saree” typically clicks or purchases in a specific price range compared to query “Panchi Saree”, which is usually higher priced. These features act as proxy relevance signals boosting filtering of irrelevant products and discovery of long tail products which lacked historical data.
Each scalar feature is initially normalised and mapped to a high-dimensional embedding vector using linear layers followed by activation functions which are then mixed by the self-attention modules.
3.2.2 Query and Catalog Embeddings
One of the key improvements in the deep learning based approach over XGBoost was the direct use of embeddings rather than single dot product score to capture relevance between query and catalog. These embeddings were trained during the retrieval stage where we first leveraged LLMs to extract visual attributes from images to get text in natural language form and generate rich summaries of multimodal content. This was then fine-tuned using a BERT encoder model with shared weights between query and catalog using contrastive setting.
We added multiple embeddings capturing different aspects of query-catalog relationships
- Dot Product Similarity: captures overall semantic similarity, how well they align in the embedding space.
- Element-wise Difference: The subtraction operation captures the semantic distance or gap between what the user is searching for and what's being offered. This tells the model not just "how similar" but "in what ways" the query and product differ.
- Hadamard Product: Unlike the dot product which gives an overall similarity score, the Hadamard product preserves dimensional information about where query and product align. This allows the model to learn that certain types of semantic alignment are more important for different tasks.
3.2.3 Transformer Encoder Block for Feature Interaction
The processed features are fed into a transformer encoder to capture high-order interactions between features, where the [CLS] token is prepended and used as a global aggregator. The encoders modeled flexible pairwise interactions across all input features using self-attention. Transformers naturally handle a mix of dense and embedding features by treating each as a token, where multiple attention heads learn cross-modal interactions simultaneously. They also support attention masking, allowing the model to ignore missing tokens during training, which reduces noise and improves generalization, which was useful for our use case.
3.3 Customized Gate Control (CGC) Network: Balancing Shared and Specialized Learning
To support multi-task learning where tasks like CTR and CVR have overlapping but distinct optimization objectives, we evaluated several expert-sharing architectures: Shared Bottom, Multi-gate Mixture-of-Experts (MMoE), and Customized Gate Control (CGC). Shared bottom architectures suffer from task interference where conflicting gradients from different tasks can hurt overall performance. MMoE addresses this by introducing task-specific gates that select relevant experts, but all tasks share the same pool of experts, limiting task-specific specialization. Customized Gate Control (CGC) extends MMoE by introducing both shared experts (for common patterns) and task-specific experts (for specialized behaviors), with customized gates controlling access to both pools. The key advantage is that CGC can learn when to leverage shared representations versus when to rely on task-specific experts, reducing negative transfer while maintaining positive knowledge sharing.
3.4 Knowledge Distillation
From the candidate generator layer, L1 receives a mix of relevant and irrelevant candidates where certain products (often popular products) tend to have high CTR and CVR rates regardless of their relevance to the specific query. In order to ensure the model balances both relevance and conversion objectives we trained a specialised teacher model focused purely on relevance assessment, then transferred this knowledge to our multi-task student model.
The teacher model is a high-capacity BERT based cross-encoder designed for providing soft target scores that encode fine-grained semantic matching between user queries and e-commerce catalog listings. To address the inconsistency and noise in raw catalog data, we leveraged LLM to generate semantically rich summaries by distilling structured and multimodal product information—including titles, descriptions, and image captions—into clean, human-readable text. These summaries, combined with user queries, are jointly encoded using a transformer using focal loss with full cross-attention to capture deep inter-token relationships. For training we curated a balanced dataset of query–catalog pairs using historical user interaction data (add-to-carts, clicks, purchases) as positive signals and generated two types of negatives per query: random negatives, sampled from unrelated catalogs, and hard negatives, selected from the same subcategory as the positives but with low embedding similarity based on prior relevance models.
4. Model Training
4.1 Loss Function
To address the data sparsity and selection bias in conversion modelling, we adopt the ESMM framework. Instead of modelling conversion (CVR) only on clicked data, it jointly models CTR and CTCVR (click-through × conversion) over the entire impression space. Given the impression x, the model predicts:
- Post-view click-through rate, pCTR = p(Click = 1 | x)
- Click-through conversion rate, pCTCVR = pCTR * p(Order = 1 | Click=1, x)
pCTCVR denotes the conditional probability of the product being purchased given that it has been viewed, which depicts complete user behaviour impression → click → purchase.
The final loss is a linear combination of the losses of the individual tasks:
Total loss = w1 * L_CTR + w2 * L_CTCVR + w1 * L_KD
where,
- L_CTR = Binary cross-entropy on CTR task.
- L_CTCVR = Binary cross-entropy on CTCVR task.
- L_KD = KL divergence with teacher model
- w_k = Task specific weights
4.2 Other Key Features
- Training data: We sampled our data ensuring query diversity across head, torso and tail, removing outlier sessions and restricting frequently occurring query-catalog pairs. The data was collected over a larger time frame which allowed better generalisation.
- Position bias: can mislead the model by attributing higher relevance to top-ranked products purely due to their placement, not their actual quality. To mitigate this, we introduce a position side tower whose output is added to the main tower’s logits during training. At inference time, we drop the position tower output, ensuring the final ranking reflects true relevance rather than positional influence.
- Distributed training with DDP: We trained the model using PyTorch DDP with mixed precision using iterable DataLoader for large-scale efficiency, optimized with AdamW and a cosine scheduler with warm-up.
5. Offline and Online Results
Our evaluation framework includes both offline metrics and online A/B testing. In offline setup we evaluate two key areas:
- Ranking metrics like Precision@k, nDCG@k, MRR for clicks and orders - helps to measure whether the model surfaces higher-quality, more engaging products at the top of the list.
- Feed distribution metrics - helps to analyze the diversity and fairness of the top-K recommendations—evaluating average product rating, catalog diversity (unique product count), price, quality (bad rating count) and bias toward specific candidate generators (e.g., ElasticSearch vs. EBR).
In offline tests using historical session data, our model showed a 1.66% uplift in click precision@10 and 1.02% in order precision@10 over the baseline XGBoost model. Compared to a variant without knowledge distillation, we observed an improvement of 13% in click precision, affirming its value in preserving semantic relevance. In A/B testing model delivered significant improvement in CTR and click MRR over baseline.
We qualitatively analyzed the output for a few queries by comparing the final ranked feeds of the two models: XGBoost (top) vs. Multi-task model (bottom).
query = “boot heel” (Women and Men Footwear)
query = “rucksack bag for women” (Travel Accessories)
query = “earring bird” (Women Jewellery)
query = “bottle speaker”
query = “polo tshirt black” (Men Fashion)
query = “carrom bod” (spelling mistake)
query = “lavender skirt long” (Women Western Wear)
Conclusion
This work presents a unified deep learning framework that aligns semantic relevance with conversion objectives through multi-task learning, transformer-based feature interaction, and guided supervision via knowledge distillation. By overcoming the constraints of traditional dual-model setups, we build a more flexible and signal-rich L1 ranker. The integration of CGC for expert disentanglement, attention masking for handling sparsity, and efficient deployment via Triton enables better model performance and production-ready scalability.
