Search Analytics

You are a search analytics engineer. The user wants to track search queries, monitor click-through rates, and use that data to tune search relevance.

What to check first

• Verify your search engine has a logging mechanism enabled (Elasticsearch, Algolia, or custom solution)
• Check that your frontend captures click events on search results with data-* attributes or event listeners
• Confirm you have write access to an analytics database or time-series storage (MongoDB, ClickHouse, or cloud analytics service)

Steps

1. Add a query logger on the search endpoint that records the exact query string, timestamp, user ID, and search context (filters applied, sort order)
2. Attach a unique search_session_id to each search request so you can correlate clicks back to specific queries
3. Instrument result click tracking on the frontend by listening to click events on result links and POST-ing the clicked result ID, position, and search_session_id to an analytics endpoint
4. Store query and click events in separate collections/tables: queries table (query, timestamp, user_id, result_count, session_id) and clicks table (session_id, result_id, position, clicked_at, dwell_time)
5. Calculate click-through rate (CTR) per query by grouping clicks by query string and dividing successful clicks by total impressions
6. Identify zero-result queries and low-CTR queries (position 1 result gets <5% clicks) as relevance problems
7. Export aggregated metrics (top queries, CTR by position, query volume trends) to a dashboard for weekly review
8. Use CTR and position data to retrain relevance models or adjust ranking weights for underperforming query types

Code

// Search query logger (backend)
const express = require('express');
const { MongoClient } = require('mongodb');

const client = new MongoClient(process.env.MONGODB_URI);
const db = client.db('search_analytics');
const queriesCollection = db.collection('search_queries');
const clicksCollection = db.collection('search_clicks');

app.post('/api/search', async (req, res) => {
  const { q, filters = {}, userId } = req.body;
  const sessionId = crypto.randomUUID();
  
  const startTime = Date.now();
  const results = await performSearch(q, filters);
  const duration = Date.now() - startTime;
  
  // Log query event
  await queriesCollection.insertOne({
    query: q,
    sessionId,
    userId,
    filters,
    resultCount: results.length,
    duration,
    timestamp: new Date(),
  });
  
  // Return results with session ID embedded
  res.json({
    results: results.map((r, i) => ({ ...r, position: i + 1 })),
    sessionId,
  });
});

// Frontend click tracking (client-side)
document.addEventListener('click', async (e) => {

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Treating this skill as a one-shot solution — most workflows need iteration and verification
• Skipping the verification steps — you don't know it worked until you measure
• Applying this skill without understanding the underlying problem — read the related docs first

When NOT to Use This Skill

• When a simpler manual approach would take less than 10 minutes
• On critical production systems without testing in staging first
• When you don't have permission or authorization to make these changes

How to Verify It Worked

• Run the verification steps documented above
• Compare the output against your expected baseline
• Check logs for any warnings or errors — silent failures are the worst kind

Production Considerations

• Test in staging before deploying to production
• Have a rollback plan — every change should be reversible
• Monitor the affected systems for at least 24 hours after the change