End-to-end session tracing with payloads
Trace a complete user journey across a browser frontend, a Vercel AI Node.js server, and a Java backend into one correlated Session with HTTP request and response payloads in Softprobe Explorer.
The Travel OTA demo (softprobe/demo-ota) keeps its application source uninstrumented on main. Its compose.blackbox.yml overlay attaches Softprobe browser recording, Vercel AI telemetry, and OpenTelemetry eBPF Instrumentation (OBI) at startup so you can run the full-stack demo on macOS or Linux in one command.
Architecture and instrumentation seams
A traveler chats in the browser web app (travel-web), which calls the Vercel AI Node.js server (travel-agent), which calls the Java backend (travel-ota) and an external airline NDC API:
All three tiers export telemetry using the same Ingestion Key (spk_...) and join into a single Session through gen_ai.conversation.id, session.id, and the W3C traceparent header. Each tier uses a distinct OpenTelemetry service.name (travel-web, travel-agent, travel-ota) so Explorer's Graph and Span Detail separate every service and outbound peer target.
Run the demo with Docker Compose
You need Docker with Compose, a Softprobe Explorer workspace, and a Google Gemini API key. Use only synthetic travel details in this demo.
1. Create an Agent in Explorer
In Explorer, open Agents and click + Connect agent. Name the Agent travel-ota-demo, choose Vercel AI, set the environment to demo, and copy its Ingestion Key (spk_...).
2. Clone demo-ota and start the Blackbox overlay
git clone --depth 1 https://github.com/softprobe/demo-ota.git
cd demo-ota
export INGESTION_KEY="spk_REPLACE_WITH_INGESTION_KEY"
export GOOGLE_GENERATIVE_AI_API_KEY="YOUR_GEMINI_API_KEY"
docker compose -f compose.blackbox.yml up --buildFor staging Explorer, also export:
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="https://explorer-staging.softprobe.ai/api/thelake/v1/traces"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment.name=staging"
export SOFTPROBE_ENVIRONMENT="staging"3. Run a journey and inspect the Session Graph
Open http://127.0.0.1:4310/agent, select your configured Gemini model, and complete a synthetic search, booking, and ticket-issuance flow:
- Search for a flight:text
Find a one-way economy flight from LHR to CDG for next Tuesday. - Select an offer (or reply
Prepare offer 1 for Alex Demo), then click Confirm booking when the review card appears. - Click Pay ... and issue ticket with the prefilled demo card.
In Softprobe Explorer, open Sessions, filter for travel-ota-demo, and open the newest Session. Switch to Graph and click nodes to inspect Span Detail across all three tiers:
- Web app (
travel-web). RRWeb DOM replay and browserPOSTclient spans withurl.templateroute patterns and redacted JSON payloads. - Vercel AI server (
travel-agent).invoke_agentroot spans,chatgeneration spans, andexecute_toolspans (travel_search,travel_prepare,travel_book,travel_order) linked bygen_ai.conversation.id. - Java backend (
travel-ota). InboundPOST /api/flights/search,/api/flights/book,/api/flights/orders/query, and/api/flights/payandissueserver spans with[REDACTED]card details, plus outboundPOST /ndc/v21.3/*client spans to the external NDC airline host.
Press Ctrl+C or run docker compose -f compose.blackbox.yml down when finished.
How each tier is instrumented
The blackbox/ directory in demo-ota wires all three tiers without modifying the uninstrumented application source:
Browser (@softprobe/web-record)
blackbox/entrypoint.sh loads @softprobe/web-record/dist/browser.js via <script data-softprobe-options='...' defer> in index.html and allows the OTLP origin in Content-Security-Policy:
- Sets
serviceName: "travel-web"and exports OTLP traces and rrweb DOM recordings (maskAllInputs: true). - Configures
httpBodyCapture.routeswith wildcard*segments to trace businessPOSTcalls while skipping 1-secondGET /agent/api/sessions/:idpolling. - Captures JSON bodies up to 64 KiB and replaces
cardNumber,cvv,cvc,password, andsecretwith[REDACTED]. - Adopts
ses_*session IDs automatically from/agent/api/sessionsresponses,/sessions/:idURLs, andsessionStorage.
See @softprobe/web-record and WebRecordInitOptions for every option.
Vercel AI server (@softprobe/vercel-ai-sdk)
blackbox/init-vercel-ai.mjs runs via NODE_OPTIONS="--import /opt/blackbox/init-vercel-ai.mjs" before src/start.mjs starts:
import { initVercelAiTelemetry } from "@softprobe/vercel-ai-sdk";
initVercelAiTelemetry({
serviceName: process.env.SOFTPROBE_AI_SERVICE_NAME || "travel-agent",
});initVercelAiTelemetry extracts W3C traceparent and ses_* session IDs from incoming browser requests, records agent, generation, and tool spans, and forwards traceparent on outgoing fetch, http, and https calls to the Java backend.
Java backend (OpenTelemetry OBI)
blackbox/Dockerfile.obi installs official OpenTelemetry OBI v1.0.0 in a sidecar container sharing the app's PID and network namespace. blackbox/obi.yaml selects the Java travel-ota process (OTEL_SERVICE_NAME=travel-ota), propagates W3C headers, redacts $.paymentDetails.cardNumber and $.paymentDetails.cvv on /api/flights/payandissue, and extracts JSON bodies for /api/flights/* and /ndc/v21.3/*.
See the OBI distributed traces guide and Config v2 reference for Linux kernel and container deployment details.
Use your own app
To apply this pattern to your own stack, assign a distinct OpenTelemetry service.name to each tier:
- Web app. For bundled apps (Vite, Next.js, Webpack), install
@softprobe/web-recordand callRecordSdk.init({ ingestionKey, serviceName: 'my-web' }). For plain HTML pages without a bundler, add<script src="https://unpkg.com/@softprobe/web-record/dist/browser.js" data-softprobe-options='{"ingestionKey":"spk_...","serviceName":"my-web"}' defer></script>to<head>and allow your OTLP endpoint origin inconnect-src. UsehttpBodyCaptureto scope routes or addredactJsonPaths. - AI agent server. Install
@softprobe/vercel-ai-sdkand@softprobe/tracing, then callinitVercelAiTelemetryat server startup. For Python, OpenAI, LangChain, OpenCode, or standard OpenTelemetry GenAI (gen_ai.*), see the Python SDK (softprobe-py) and JavaScript / TypeScript SDK (softprobe-js) references. - Backend services. Set
OTEL_SERVICE_NAMEon each backend process or Kubernetes workload and run OBI with your process selector (cmd_args_globoropen_ports) andurl_path_patternsallowlist.
API references
- JavaScript / TypeScript SDK (
softprobe-js) for@softprobe/web-record,@softprobe/vercel-ai-sdk, and@softprobe/tracing - Python SDK (
softprobe-py) forSoftprobeClient,observe_openai,softprobe.langchain, andsoftprobe.propagation - Vercel AI SDK guide
- OBI Config v2 reference