Distributed Tracing

You are a distributed systems engineer. The user wants to set up distributed tracing across microservices to track requests and identify bottlenecks.

What to check first

• Run docker ps to verify a tracing backend (Jaeger, Zipkin) is running on the expected port
• Check npm list opentelemetry-api opentelemetry-sdk-node to see if OpenTelemetry is installed
• Confirm your service exports traces via curl http://localhost:16686/api/traces (Jaeger endpoint)

Steps

1. Install OpenTelemetry core packages: npm install @opentelemetry/api @opentelemetry/sdk-node @opentelemetry/auto @opentelemetry/sdk-trace-node @opentelemetry/exporter-trace-otlp-http
2. Set the OTEL_EXPORTER_OTLP_ENDPOINT environment variable to point to your collector: export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
3. Import and initialize the tracing SDK at the very top of your entry file (before other imports) using NodeSDK with span processors
4. Install instrumentation packages for your specific frameworks: npm install @opentelemetry/instrumentation-http @opentelemetry/instrumentation-express (or appropriate framework)
5. Register instrumentations when creating the NodeSDK instance via the instrumentations array
6. Add baggage propagators to trace context across service boundaries using CompositePropagator with W3CTraceContextPropagator and JaegerBaggagePropagator
7. Configure the OTLP exporter with correct headers and concurrencyLimit for your traffic volume
8. Test by making HTTP requests and checking the Jaeger UI at http://localhost:16686 for span traces

Code

// tracing.js - Import this FIRST in your entry file (index.js or app.js)
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');
const { BatchSpanProcessor } = require('@opentelemetry/sdk-trace-node');
const { W3CTraceContextPropagator, JaegerBaggagePropagator } = require('@opentelemetry/core');
const { CompositePropagator } = require('@opentelemetry/core');
const { registerInstrumentations } = require('@opentelemetry/instrumentation');

const otlpExporter = new OTLPTraceExporter({
  url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT || 'http://localhost:4318

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Treating this skill as a one-shot solution — most workflows need iteration and verification
• Skipping the verification steps — you don't know it worked until you measure
• Applying this skill without understanding the underlying problem — read the related docs first

When NOT to Use This Skill

• When a simpler manual approach would take less than 10 minutes
• On critical production systems without testing in staging first
• When you don't have permission or authorization to make these changes

How to Verify It Worked

• Run the verification steps documented above
• Compare the output against your expected baseline
• Check logs for any warnings or errors — silent failures are the worst kind

Production Considerations

• Test in staging before deploying to production
• Have a rollback plan — every change should be reversible
• Monitor the affected systems for at least 24 hours after the change