---
url: /en/platform/getting-started/quick-start.md
---
# Quick Start Guide

::: tip Automate with sp CLI
For Java record-and-replay testing (agent, policies, replay), see [Testing getting started](/en/testing/getting-started). To automate with `sp`, see [CLI quickstart](/en/testing/getting-started).
:::

::: info Important
The Quick Start demo environment already has the SESSIFY (`@softprobe/sessify`) pre-installed and enabled, so you don't need to install it again.
If you want to integrate the SDK into your own frontend app, see [SESSIFY Integration](/en/platform/sessify).
:::

Get started with SP-Istio Agent in minutes using a local Kubernetes cluster with Kind.

::: info Time Estimate

* Setup: 10-15 minutes
* Demo exploration: 5-10 minutes
  :::

## Prerequisites

* **Operating System**: macOS (or Linux with Docker)
* **Required Tools**:
  * [Docker Desktop](https://www.docker.com/products/docker-desktop)
  * [Kind](https://kind.sigs.k8s.io/) - `brew install kind`
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/) - `brew install kubectl`
  * [Istio CLI](https://istio.io/latest/docs/setup/getting-started/#download) - `brew install istioctl`

### Install All Tools at Once

```bash
brew install kind kubectl istioctl
```

## Step 1: Set up Kind Cluster with Istio

Create a Kind cluster and install Istio with OpenTelemetry Operator:

```bash
curl -L https://raw.githubusercontent.com/softprobe/softprobe/main/scripts/cluster-setup.sh | sh
```

::: tip
This script will automatically:

* Create a local Kubernetes cluster using Kind
* Install Istio service mesh
* Install OpenTelemetry Operator for telemetry collection
  :::

## Step 2: Install the Travel Demo

Deploy the demo application with SP-Istio Agent:

::: info Important
Use the `minimal.yaml` file you downloaded from the [Account Setup](/en/platform/getting-started/account-setup) guide.
:::

```bash
# Install Softprobe Istio WASM Plugin (using the minimal.yaml downloaded from Account Setup)
kubectl apply -f minimal.yaml

# Install demo app
kubectl apply -f https://raw.githubusercontent.com/softprobe/softprobe/main/examples/travel/apps.yaml

# Expose the demo
sleep 10 && kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80
```

## Step 3: Try the Demo

1. Open [`http://localhost:8080/`](http://localhost:8080/) in your browser
2. Select a **pair** of cities

5) Process a payment with fake details

::: tip
Generate at least 5-10 bookings to see meaningful data in the dashboard.
:::

## Step 4: View Results in Softprobe Dashboard

After generating some traffic:

1. Go to [Softprobe Dashboard](https://dashboard.softprobe.ai)
2. Navigate to **Travel View** in the left navigation menu
3. Explore the captured requests and business-level traces

### Demo Video

## Cleanup

When you're done with the demo, clean up the Kind cluster:

```bash
kind delete cluster --name sp-demo-cluster
```

::: tip Congratulations!
You've successfully set up Softprobe and seen it in action! Next steps:

* [Production Deployment](/en/platform/deployment/installation) - Deploy to your production cluster
* [Configuration Guide](/en/platform/configuration/config) - Customize collection rules
* [Advanced Concepts](/en/platform/advanced-guides/concepts) - Learn how Softprobe works
  :::

---

---
url: /en/platform/getting-started/account-setup.md
---
# Account Setup & Public Key Management

Set up your Softprobe account and manage public keys for secure authentication.

::: info Prerequisites

* A valid email address
* Access to the Softprobe Dashboard
  :::

## Overview

This guide walks you through:

* Creating a Softprobe account
* Setting up your tenant group
* Generating and managing public keys
* Downloading configuration files

### Step 1: Create Your Account

1. Visit [Softprobe Dashboard](https://dashboard.softprobe.ai)
2. Click **"Sign Up"** to create a new account
3. Fill in your details:
   * **Email address** (will be your login username)
   * **Password** (minimum 8 characters)
4. Verify your email address by clicking the link sent to your inbox

### Step 2: Access Settings and Create Tenant Groups

After email verification and login, you can access the Settings page to manage tenant groups:

1. Navigate to the **[Settings](https://dashboard.softprobe.ai/settings)** page
2. **Create your first tenant group**
   * Click the "Create Group" button
   * **Tenant Group Name**: Choose a unique name for your organization
     * This will be used to organize your services and data
     * Example: `my-company-prod`, `acme-corp`, `team-alpha`
   * **Description** (optional): Add a brief description of your group
3. Click **"Create Tenant Group"**

::: tip
Choose your tenant group name carefully as it cannot be changed later. Use a name that clearly identifies your organization or team.
:::

### Step 3: Generate Public Key

Once your tenant group is created:

1. Navigate to **"Public Keys"** in the dashboard sidebar
2. Click **"Generate New Public Key"**
3. Provide the following information:
   * **Key Name**: A descriptive name (e.g., `production-cluster`, `dev-environment`)
   * **Environment**: Select the appropriate environment type
4. Click **"Generate Key"**

:::caution Important
Softprobe uses **public key authentication** instead of traditional API keys. This means:

* Your configuration contains only a public identifier, not a secret
* No sensitive credentials are stored in your Kubernetes configuration
* Public keys can be easily rotated without service disruption
* Uses asymmetric cryptography for authentication
  :::

## 📁 Configuration File

When you generate a public key, a `minimal.yaml` file is automatically downloaded. This file contains:

* Your public key identifier (not a secret)
* Pre-configured endpoints
* Default collection rules
* All necessary Kubernetes resources

### File Structure

The downloaded `minimal.yaml` includes:

```yaml
# WasmPlugin configuration with your public key
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: sp-istio-agent
spec:
  pluginConfig:
    public_key: "your-public-key-identifier"
    # ... other secure configurations
```

After completing account setup:

1. **For Quick Testing**: Follow the [Quick Start Guide](/en/platform/getting-started/quick-start)
2. **For Production**: Follow the [Production Installation Guide](/en/platform/deployment/installation)
3. **For Custom Configuration**: Review the [Configuration Reference](/en/platform/configuration/config)

## ❓ Troubleshooting

### Common Issues

**Can't access the dashboard?**

* Check your internet connection
* Verify the URL: `https://dashboard.softprobe.ai`
* Try clearing your browser cache

**Public key generation failed?**

* Ensure your tenant group is properly set up
* Check that you have the necessary permissions
* Try refreshing the page and generating again

***

## FAQ

> Common questions have been moved to a separate page:
>
> * [Getting Started FAQ](/en/platform/support/faq)

---

---
url: /en/platform/deployment/installation.md
---
# Server-side Agent (Istio WASM) & SESSIFY Installation

Deploy SP‑Istio Agent to your Istio service mesh, and integrate the SESSIFY for client-side enrichment.

::: tip Looking for user install?
Softprobe server install (Helm sp-backend), client install (`sp setup`, Spcode Service), `sp code`, `sp doctor`, and `sp upgrade` are documented under [Testing installation](/en/testing/installation/). This Platform page is for Istio mesh / SESSIFY deployment.
:::

## Prerequisites

Before installing SP-Istio Agent in production, ensure you have:

* A running Kubernetes cluster
* Istio installed and configured
* kubectl access with appropriate permissions
* Network connectivity to Softprobe endpoints

::: info Using GKE Autopilot?
If your cluster runs on GKE Autopilot, be aware of these common installation/permission constraints (summary from the full guide):

* NET\_ADMIN capability is disabled by default, which can break istio-init/iptables steps
* You cannot modify the CNI ConfigMap in the kube-system namespace (managed namespace restrictions)
* Some system namespaces are managed/protected and certain resources cannot be changed

Quick fixes:

* Enable workload policies when creating or updating the cluster: `--workload-policies=allow-net-admin`
* Disable the Istio CNI component during installation: `--set components.cni.enabled=false`

Read the full step-by-step guide, verification, and troubleshooting:
[GKE Autopilot Istio Installation Guide →](./GKE-Autopilot-Istio-Installation-Guide)
:::

## Install SESSIFY (Client-Side Enrichment)

Add the Softprobe SESSIFY to your frontend to create session-scoped context and capture route changes. This provides full-context visibility without modifying server-side code.

### Install package

```bash
npm install @softprobe/sessify
```

### Initialize at Your App Entry Point

```jsx
// app/layout.tsx or your application's entry file
'use client'
import { useEffect } from 'react';
import { initSessify } from '@softprobe/sessify';

export default function RootLayout({ children }) {
  useEffect(() => {
    // Initialize the session management library
    // This is the necessary first step to use @softprobe/sessify
    initSessify({});
  }, []);

  return (
    <html lang="en">
      <body>
        {children}
      </body>
    </html>
  );
}
```

See the full [Sessify guide](/en/platform/sessify) for framework-specific examples (React/Vue/Next.js) and advanced usage.

## Install Server-side Agent (Istio WasmPlugin)

Install SP‑Istio Agent using your personalized `minimal.yaml` file downloaded during [Account Setup](/en/platform/getting-started/account-setup). It contains your public key identifier and pre-configured settings.

```bash
# Use the minimal.yaml file downloaded from the Softprobe Dashboard
kubectl apply -f minimal.yaml
```

::: tip Namespace
If your `minimal.yaml` does not set `metadata.namespace`, specify a namespace explicitly:

```bash
kubectl apply -n <YOUR_NAMESPACE> -f minimal.yaml
```

Or set it in the YAML:

```yaml
metadata:
  namespace: <YOUR_NAMESPACE>
```

Choose the namespace that matches where Istio is installed and where you manage mesh‑wide resources in your cluster.
:::

This deploys the WasmPlugin globally across your Istio service mesh.

## Verify Installation

Check that the WasmPlugin has been created successfully:

```bash
kubectl get wasmplugin -A
```

You should see the SP-Istio Agent plugin listed.

## Restart Workloads

After applying the WasmPlugin/EnvoyFilter, restart affected workloads to load the updated sidecar configuration:

```bash
# Restart all deployments in a namespace (replace <namespace>)
kubectl rollout restart deployment -n <namespace>

# Or restart a single deployment
kubectl rollout restart deployment <name> -n <namespace>
```

## View Context View in Dashboard

If you enabled sidecar injection on a namespace just now, restarting ensures pods are recreated with the updated sidecar and configuration.

::: tip Next: View Context View in Dashboard
After deploying SP‑Istio Agent and initializing the SESSIFY, generate some traffic in your app, then:

1. Open your Softprobe Dashboard → Context View
2. Select the time range and environment (env) matching your deployment
3. Filter by serviceName if needed; search by userId/sessionId/request\_body\_hash to locate sessions
4. Click a session to inspect the end‑to‑end graph, spans, client metrics, and interaction events

You do not need to change server‑side code to get full‑context visibility.
:::

## Configuration

Softprobe’s default configuration captures HTTP traffic for all services in the mesh. To customize the capture scope, service identification, and advanced options, please refer to the full Configuration Guide: [Configuration Guide](/en/platform/configuration/config). You can start from the minimal example and gradually extend `collectionRules`, service discovery, and external communication settings based on your needs.

### Scoped Deployment

To deploy the agent to specific namespaces or workloads only, you can create a scoped WasmPlugin configuration. See the [Configuration Guide](/en/platform/configuration/config) for detailed configuration options.

---

---
url: /en/platform/deployment/GKE-Autopilot-Istio-Installation-Guide.md
---
# Complete Guide to Installing Istio on GKE Autopilot

## Overview

This guide provides detailed instructions on how to successfully install and configure Istio service mesh on Google Kubernetes Engine (GKE) Autopilot clusters. GKE Autopilot is Google's managed Kubernetes service that offers hardened defaults and a simplified management experience.

## Prerequisites

### System Requirements

* **GKE Cluster Version**: 1.27 or higher
* **gcloud CLI**: Installed and configured
* **kubectl**: Installed and configured
* **istioctl**: Istio command-line tool

### Permission Requirements

* Administrator access to the GKE cluster
* Permission to modify cluster configurations

## Core Issues and Solutions

### Problem Background

The main challenges when installing Istio on GKE Autopilot are:

1. **NET\_ADMIN Permission Restrictions**: Autopilot disables `NET_ADMIN` Linux capability by default as part of hardened defaults
2. **CNI Component Limitations**: Cannot modify ConfigMaps in the `kube-system` namespace
3. **Managed Namespace Restrictions**: Certain system namespaces are managed and protected by Google

### Key Solution

**Enabling NET\_ADMIN capability** is the key to solving Istio installation issues!

## Detailed Installation Steps

### Step 1: Check Cluster Version

```bash
# Check Kubernetes version
kubectl version --short

# Check cluster information
kubectl get nodes -o wide
```

Ensure the cluster version is 1.27 or higher.

### Step 2: Configure gcloud Project

```bash
# Set the correct project ID
gcloud config set project YOUR_PROJECT_ID

# Verify configuration
gcloud config list
```

### Step 3: Enable NET\_ADMIN Capability for Cluster

#### New Cluster Creation (Recommended)

```bash
gcloud container clusters create-auto istio-cluster \
    --region=us-central1 \
    --workload-policies=allow-net-admin \
    --cluster-version=1.27.2-gke.1200
```

#### Existing Cluster Update

```bash
gcloud container clusters update CLUSTER_NAME \
    --region=REGION \
    --workload-policies=allow-net-admin
```

**Important Note**: The `--workload-policies=allow-net-admin` parameter is crucial for successful Istio installation!

### Step 4: Install Istio

#### 4.1 Download Istio

```bash
# Download latest version
curl -L https://istio.io/downloadIstio | sh -

# Or download specific version
export ISTIO_VERSION=1.27.1
curl -L https://istio.io/downloadIstio | TARGET_ARCH=$(uname -m) sh -

# Add to PATH
cd istio-*
export PATH=$PWD/bin:$PATH
```

#### 4.2 Install Istio Control Plane

```bash
# Use default profile with CNI component disabled
istioctl install --set profile=default --set components.cni.enabled=false -y
```

**Key Configuration Explanation**:

* `--set profile=default`: Uses production-recommended configuration
* `--set components.cni.enabled=false`: Disables CNI component to avoid kube-system permission issues

### Step 5: Verify Installation

#### 5.1 Check Istio Component Status

```bash
# Check Istio system components
kubectl get pods -n istio-system

# Expected output:
# NAME                                    READY   STATUS    RESTARTS   AGE
# istio-ingressgateway-xxx               1/1     Running   0          2m
# istiod-xxx                             1/1     Running   0          2m
```

#### 5.2 Verify CRD Installation

```bash
# Check Istio CRDs
kubectl get crd | grep istio

# Should see the following CRDs:
# - wasmplugins.extensions.istio.io
# - serviceentries.networking.istio.io
# - destinationrules.networking.istio.io
# - envoyfilters.networking.istio.io
# - etc...
```

### Step 6: Configure Namespaces

#### 6.1 Enable Sidecar Injection

```bash
# Enable automatic sidecar injection for target namespace
kubectl label namespace YOUR_NAMESPACE istio-injection=enabled

# Verify label
kubectl describe namespace YOUR_NAMESPACE
```

#### 6.2 Apply Istio Configuration

```bash
# Apply your Istio resource configuration
kubectl apply -f your-istio-config.yaml
```

## Common Issues and Solutions

### Issue 1: NET\_ADMIN Permission Denied

**Error Message**:

```
linux capability 'NET_ADMIN' on container 'istio-init' not allowed
```

**Solution**:
Ensure `--workload-policies=allow-net-admin` is enabled

### Issue 2: CNI Installation Failure

**Error Message**:

```
failed to update resource with server-side apply for obj ConfigMap/kube-system/istio-cni-config
```

**Solution**:
Use `--set components.cni.enabled=false` to disable CNI component

### Issue 3: Project Permission Issues

**Error Message**:

```
Kubernetes Engine API has not been used in project
```

**Solution**:
Ensure gcloud configuration points to the correct project ID

## Best Practices

### Performance Optimization

1. **Resource Limits**: Set appropriate resource limits for sidecar containers
2. **Monitoring**: Deploy Istio monitoring components (Prometheus, Grafana, Jaeger)
3. **Log Management**: Configure appropriate log levels and rotation policies

### Maintenance Recommendations

1. **Regular Updates**: Keep Istio versions up to date
2. **Configuration Backup**: Regularly backup Istio configurations
3. **Test Environment**: Validate in test environment before production

## Deployment Verification

### Deploy Test Application

```bash
# Deploy sample application to verify Istio functionality
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.27/samples/bookinfo/platform/kube/bookinfo.yaml

# Check sidecar injection
kubectl get pods -o="custom-columns=NAME:.metadata.name,CONTAINERS:.spec.containers[*].name"
```

### Test Traffic Management

```bash
# Create Gateway and VirtualService
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.27/samples/bookinfo/networking/bookinfo-gateway.yaml

# Get Ingress Gateway address
kubectl get svc istio-ingressgateway -n istio-system
```

## Summary

Key points for successfully installing Istio on GKE Autopilot:

1. ✅ **Enable NET\_ADMIN capability**: This is the most important step
2. ✅ **Use correct configuration**: Disable CNI component to avoid permission issues
3. ✅ **Verify installation**: Ensure all components are running properly
4. ✅ **Configure namespaces**: Enable sidecar injection

By following this guide, you should be able to successfully deploy and run Istio service mesh on GKE Autopilot clusters.

## References

* [Istio Official Documentation](https://istio.io/latest/docs/)
* [GKE Autopilot Documentation](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview)
* [GKE Autopilot Hardened Defaults (Security)](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-security)

*This guide is based on actual deployment experience and is applicable to Istio 1.27+ and GKE 1.27+ versions.*

## Application Instrumentation on Autopilot (without Operator)

In some GKE Autopilot environments, installing cluster-wide operators (like OpenTelemetry Operator) can be constrained by security policies, private cluster firewall rules, or webhook requirements. If you can’t (or prefer not to) use the Operator, you can manually attach language-specific agents to your applications and export telemetry to an OTLP endpoint (Collector or Softprobe ingestion endpoint).

### General setup

* Choose an OTLP endpoint (Collector service or external ingestion URL)
* Set service metadata via environment variables
* Ensure egress from workloads to the OTLP endpoint (HTTP or gRPC)
* Prefer non-root containers and define resource requests/limits to comply with Autopilot

Common environment variables (adapt to your endpoint):

```bash
# Example (HTTP OTLP)
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.example.com"      # base URL, the SDK will append /v1/traces /v1/metrics
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"                 # or "grpc"
export OTEL_SERVICE_NAME="your-service"
export OTEL_RESOURCE_ATTRIBUTES="service.namespace=production,service.version=1.0.0"
# Optional headers (e.g., auth token)
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_TOKEN"
```

You may set these variables directly in your Kubernetes Deployment under `env:`.

***

### Java (JVM)

Attach the OpenTelemetry Java agent by adding `-javaagent` and environment variables:

```dockerfile
# Add the agent to the image (recommendation: bake into your app image)
ADD opentelemetry-javaagent.jar /otel/javaagent.jar
```

```yaml
# Deployment snippet
spec:
  template:
    spec:
      containers:
        - name: app
          image: your-registry/your-java-app:latest
          env:
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "https://otel.example.com"
            - name: OTEL_EXPORTER_OTLP_PROTOCOL
              value: "http/protobuf"
            - name: OTEL_SERVICE_NAME
              value: "your-service"
            - name: OTEL_RESOURCE_ATTRIBUTES
              value: "service.namespace=production,service.version=1.0.0"
            - name: OTEL_EXPORTER_OTLP_HEADERS
              value: "Authorization=Bearer YOUR_TOKEN"
            - name: JAVA_TOOL_OPTIONS
              value: "-javaagent:/otel/javaagent.jar"
          # or use command/args if you manage the JVM startup explicitly
```

If you control the startup script, you can also add: `-javaagent:/otel/javaagent.jar` to the JVM arguments.

***

### Node.js

Use the Node SDK and auto-instrumentations:

```bash
npm install @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node @opentelemetry/exporter-trace-otlp-http
```

Create a bootstrap file (e.g., `otel.js`):

```js
// otel.js
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');

const exporter = new OTLPTraceExporter({
  // The OTLP exporter appends /v1/traces automatically for HTTP
  url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
  headers: process.env.OTEL_EXPORTER_OTLP_HEADERS
    ? Object.fromEntries(process.env.OTEL_EXPORTER_OTLP_HEADERS.split(',').map(h => h.split('=')))
    : undefined,
});

const sdk = new NodeSDK({
  traceExporter: exporter,
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start();
```

Start your app with the bootstrap required:

```bash
# Option 1: require bootstrap
node -r ./otel.js app.js
# Option 2: via NODE_OPTIONS
export NODE_OPTIONS="--require ./otel.js" && node app.js
```

Set env variables in your Deployment as shown in the General setup section.

***

### Python

Use the Python distro and the CLI instrumentation:

```bash
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap --action=install
```

Run the application with instrumentation:

```bash
# Set env vars (as in General setup) then
opentelemetry-instrument python app.py
```

Alternatively, configure the SDK in code and use OTLP exporters.

***

### .NET

For .NET, you can use SDK-based instrumentation or auto-instrumentation (native profiler). SDK-based is simpler to adopt:

```csharp
// Program.cs (example)
using OpenTelemetry;
using OpenTelemetry.Trace;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry().WithTracing(tracerProviderBuilder =>
{
    tracerProviderBuilder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri(Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_ENDPOINT") ?? "https://otel.example.com");
            // For HTTP/protobuf, ensure protocol matches; set headers if needed
        });
});

var app = builder.Build();
app.MapGet("/", () => "Hello World!");
app.Run();
```

If you need auto-instrumentation, mount the auto-instrumentation files and set the profiler env vars (`CORECLR_ENABLE_PROFILING`, `CORECLR_PROFILER`, `CORECLR_PROFILER_PATH`, and relevant `OTEL_*` variables) in the Deployment.

***

### Go

Go commonly uses SDK-based instrumentation in code:

```go
// Example outline
import (
  "go.opentelemetry.io/otel"
  "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
  "go.opentelemetry.io/otel/sdk/resource"
  "go.opentelemetry.io/otel/sdk/trace"
)

func initTracer() (*trace.TracerProvider, error) {
  exporter, err := otlptracehttp.New(context.Background(), otlptracehttp.WithEndpoint(os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT")))
  if err != nil { return nil, err }

  tp := trace.NewTracerProvider(
    trace.WithBatcher(exporter),
    trace.WithResource(resource.Default()),
  )
  otel.SetTracerProvider(tp)
  return tp, nil
}
```

For eBPF-based HTTP telemetry in Go environments without code changes, consider a separate DaemonSet like Beyla. In Autopilot, ensure it complies with non-privileged policies.

***

### Troubleshooting on Autopilot

* Verify egress to the OTLP endpoint and TLS/cert requirements
* Define resource requests/limits for all containers
* Avoid privileged flags and root-only file paths
* Prefer baking agents into images instead of initContainers if your policy restricts them
* Check logs on both application and the OTLP backend/Collector to confirm export success

### Disable exporting (collect-only / no-export mode)

Sometimes you may want to enable instrumentation but temporarily disable exporting (e.g., for smoke tests in Autopilot). You can turn off exporters while keeping instrumentation active.

* Cross-language (environment variables):
  ```bash
  export OTEL_TRACES_EXPORTER=none
  export OTEL_METRICS_EXPORTER=none
  export OTEL_LOGS_EXPORTER=none
  ```
  This disables all exporters. The SDK will still create spans/metrics/logs according to instrumentation, but they will not be sent to any backend.

* Java (OpenTelemetry Java Agent):
  ```bash
  JAVA_TOOL_OPTIONS="-javaagent:/otel/opentelemetry-javaagent.jar \
    -Dotel.traces.exporter=none \
    -Dotel.metrics.exporter=none \
    -Dotel.logs.exporter=none \
    -Dotel.resource.attributes=service.name=sp-storage \
    -Dotel.instrumentation.http.server.capture-request-headers=tracestate \
    -Dotel.instrumentation.http.server.capture-response-headers=tracestate"
  ```
  Or add these system properties directly to your JVM start command:
  ```bash
  java -javaagent:/otel/opentelemetry-javaagent.jar \
    -Dotel.traces.exporter=none \
    -Dotel.metrics.exporter=none \
    -Dotel.logs.exporter=none \
    -Dotel.resource.attributes=service.name=sp-storage \
    -Dotel.instrumentation.http.server.capture-request-headers=tracestate \
    -Dotel.instrumentation.http.server.capture-response-headers=tracestate \
    -jar app.jar
  ```

Note:

* Disabling exporters reduces external traffic and is useful for validation; however, instrumentation overhead still exists because spans/metrics/logs are created. For production, restore the desired exporters (e.g., set OTEL\_TRACES\_EXPORTER=otlp).
* Ensure resource attributes (service.name, namespace, version) remain configured so you can easily switch exporting back on later without needing to change application manifests.

---

---
url: /en/platform/production/dashboard-user-guide.md
---
# Softprobe Dashboard User Guide

Welcome to Softprobe Dashboard. This guide helps you quickly understand the interface, manage tenants and members, and operate key features consistently.

## Overview

## 🚀 Quick Start

### 1. Registration & Login

* Visit the homepage and click “Sign Up” (top right)
* Enter email and password, complete email verification
* Log in to start using the system

::: tip
Account setup and public key management are covered in the [Account Setup Guide](/en/platform/getting-started/account-setup).
:::

### 2. Interface Overview

After logging in, you will see:

* **Left Navigation**: Quick access to modules
* **Top Bar**: User info and tenant switching
* **Main Area**: Operational interface for current module

## 📊 Core Features

### 1. Dashboard Homepage

Real-time monitoring of core system metrics

#### Data Overview Cards

* **Database Usage**: Current storage usage (GB), usage percentage, color indicators
* **Data Entry Statistics**: Total records, filter by time range, historical comparison
* **Session Statistics**: Active sessions, total sessions in selected period, trend charts
* **Performance Metrics**: Average response time, P95 metrics, system health indicators

::: tip
For end-to-end user journey correlation, install the [SESSIFY](/en/platform/sessify) on your frontend and propagate sessionId through your service mesh.
:::

#### Operations

* Data loads once on page entry (no auto-refresh)
* To update: switch time range, switch tenant, or refresh page

### 2. Tenant Management

Resource isolation and management in multi-tenant environments

#### Tenant Switching

* Use the tenant selector in the top bar (search and filter supported)
* Click target tenant to switch
* Create new tenant via “+” button and fill basic info

#### Tenant Settings

* Basic info: name, description, icon
* Member management: add/remove, roles and permissions
* Public keys: create/manage keys and validity
* Danger zone: delete tenant, export backup, clear data

### 3. Team Member Management

Permission control and collaborative management

#### Adding Members

* Go to Tenant Settings → Member Management
* Click “Add Member”, enter email, choose role
* Roles: **Administrator**, **Editor**, **Viewer**

#### Role Permissions

* **Administrator**: Full control, manage members/settings, Public keys, dangerous ops
* **Editor**: Operate data and monitoring configs, cannot manage members/settings
* **Viewer**: Read-only access, suitable for report viewers

::: tip Quick reminder
If you need higher quotas or custom retention, contact support@softprobe.ai with your tenant id and expected workload.
:::

See the [FAQ](/en/platform/support/faq) for common questions, including data storage and security, mobile access, and export options.

---

---
url: /en/platform/configuration/config.md
---
# Configuration Guide

Learn how to configure SP-Istio Agent for your specific use cases.

::: info Before You Begin

* Complete [Account Setup](/en/platform/getting-started/account-setup) to get your public key
* Have your `minimal.yaml` configuration file ready
* Basic understanding of YAML and regular expressions
  :::

## Overview

This guide covers:

* Understanding the configuration structure
* Setting up collection rules
* Configuring service discovery
* Advanced configuration options

## Configuration Structure

The SP-Istio Agent configuration is defined in a `minimal.yaml` file that contains three main components:

### 1. WasmPlugin Resource

The core component that loads the SP-Istio Agent into Istio's Envoy proxies:

```yaml
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: sp-istio-agent
  namespace: istio-system
spec:
  url: oci://docker.io/softprobe/softprobe:latest
  pluginConfig:
    # Your configuration goes here
```

### 2. EnvoyFilter Resource

Configures Envoy proxy settings for optimal performance:

```yaml
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: sp-istio-agent-config
  namespace: istio-system
spec:
  configPatches:
    # Envoy-specific configurations
```

### 3. Network Resources

Defines network policies and service discovery settings.

## Core Configuration Options

### Basic Settings

| Field            | Type   | Description                     | Default                  |
| ---------------- | ------ | ------------------------------- | ------------------------ |
| `public_key`     | string | Your Softprobe public key       | Required                 |
| `sp_backend_url` | string | Softprobe collection endpoint   | `https://o.softprobe.ai` |
| `service_name`   | string | Override service name detection | Auto-detected            |

### Example Basic Configuration

```yaml
pluginConfig:
  public_key: "your-public-key-here"
  sp_backend_url: "https://o.softprobe.ai"
  traffic_direction: "server"
  collectionRules:
    http:
      server:
        - path: ".*"
```

::: tip
Always start with the basic configuration and gradually add collection rules based on your specific needs.
:::

## 🎯 Collection Rules

Collection rules define what traffic to capture and analyze. They support both `SERVER` (inbound) and `CLIENT` (outbound) modes through the `traffic_direction` setting.

### Rule Structure

**For SERVER mode (inbound traffic):**

```yaml
pluginConfig:
  traffic_direction: "server"
  collectionRules:
    http:
      server:
        - path: ".*" # Regex pattern for URL paths
```

**For CLIENT mode (outbound traffic):**

```yaml
pluginConfig:
  traffic_direction: "client"
  collectionRules:
    http:
      client:
        - host: ".*" # Regex pattern for hostnames
          paths: [".*"] # List of regex patterns for paths
```

### SERVER Mode Rules

Capture inbound requests to your services:

```yaml
pluginConfig:
  traffic_direction: "server"
  collectionRules:
    http:
      server:
        # Capture all API endpoints
        - path: "/api/.*"
        # Capture specific endpoints
        - path: "/health"
        - path: "/metrics"
        - path: "/api/v1/users"
```

### CLIENT Mode Rules

Capture outbound requests from your services:

```yaml
pluginConfig:
  traffic_direction: "client"
  collectionRules:
    http:
      client:
        # Capture all outbound requests to any host
        - host: ".*"
          paths: [".*"]
        # Capture specific API calls to external services
        - host: ".*\\.googleapis\\.com"
          paths: ["/maps/api/.*", "/drive/v3/files"]
        # Capture requests to specific service
        - host: "my-api\\.com"
          paths: ["/api/.*"]
```

***

## Collection Rules Explained (`collectionRules`)

`collectionRules` is the most central and flexible configuration item in the SP-Istio Agent. It allows you to precisely define which HTTP traffic to collect, thus avoiding unnecessary data reporting, saving costs, and improving efficiency.

### Using Regular Expressions

In all collection rules, the values of the `path`, `host`, and `paths` fields are **regular expressions (Regex)**. This provides you with powerful matching capabilities.

* To match everything, use `.*`.
* For an exact match, write the string directly, for example, `"/api/users"`.
* To match a specific pattern, use regex syntax, for example, `"/api/v[0-9]+/items"` can match `/api/v1/items` and `/api/v2/items`.

**Note:** If an invalid regular expression is provided, the system automatically falls back to **exact string matching**.

## Service Discovery: Automatically Identifying Your Services

In a complex microservices environment, accurately identifying each service is crucial. The SP-Istio Agent achieves this through an `EnvoyFilter` named `inject-app-name-header`.

This `EnvoyFilter` executes a small piece of Lua script before processing each request. The sole purpose of this script is to **find the name of the current service** and add it to a special HTTP header, `x-sp-service-name`.

**How does it work?**

The script sequentially attempts to get the service name from several common Kubernetes environment variables, such as `OTEL_SERVICE_NAME`, `APP_NAME`, `SERVICE_NAME`, or by parsing it from `POD_NAME`. This multi-source approach greatly improves the accuracy of automatic service identification, and **you usually do not need any additional configuration**.

***

## External Communication: Connecting to the Softprobe Backend

To enable the SP-Istio Agent to send data to `o.softprobe.ai`, we need to explicitly authorize this external communication in Istio. This is done through two resources: `ServiceEntry` and `DestinationRule`.

* **`ServiceEntry`**: Adds `o.softprobe.ai` to Istio's service registry, making it a legitimate external service.
* **`DestinationRule`**: Configures TLS encryption for traffic to `o.softprobe.ai`, ensuring secure data transmission.

In short, these two resources together open a secure channel for the SP-Istio Agent to the Softprobe backend.

We hope this detailed guide helps you better understand and use the SP-Istio Agent. If you have any questions, please feel free to contact us!

## Who should use this guide

This guide is intended for:

* Platform/SRE engineers managing Kubernetes + Istio deployments
* Dev teams needing fine-grained collection rules
* Anyone customizing Softprobe to match business flows

## Next Steps

* Deploy or update your cluster with the [Installation Guide](/en/platform/deployment/installation)
* Verify data in the [Softprobe Dashboard](https://dashboard.softprobe.ai)
* Add client-side visibility via the [SESSIFY](/en/platform/sessify)

***

## Related Topics

* [Quick Start Guide](/en/platform/getting-started/quick-start)
* [Core Concepts](/en/platform/advanced-guides/concepts)
* [FAQ / Troubleshooting](/en/platform/support/faq)

---

---
url: /en/platform/advanced-guides/concepts.md
---
# Understanding Softprobe Concepts

Learn how Softprobe works, its architecture, and core principles for effective observability.

## 📋 Overview

Softprobe is a comprehensive observability platform designed for modern cloud-native applications. It provides:

* **Full-stack tracing** across services and user interactions
* **Performance monitoring** with rich context and correlations
* **Authentication-centered architecture** with public key authentication
* **Kubernetes-native integration** for seamless deployment

## 🏗️ How Softprobe Works

### Architecture Overview

Softprobe operates through a distributed architecture:

### Key Components

#### 1. SP-Istio Agent (Service Mesh Layer)

* **WASM Plugin**: Runs within Istio sidecars to capture service-to-service traffic
* **Envoy Filter**: Injects service identity headers and manages traffic capture
* **Zero Code Changes**: Works transparently with your existing applications

#### 2. SESSIFY (Frontend Layer)

* **Automatic Instrumentation**: Captures user interactions, performance metrics, and errors
* **Session Management**: Generates a per-tab sessionId and reuses it across navigations within the same tab; all frontend telemetry carries the sessionId for end-to-end correlation with backend traces and logs
* **Lightweight**: Minimal performance impact on your web applications

#### 3. Softprobe Backend

* **Collectors**: Receive and process observability data from all sources
* **Processing Engine**: Correlates, enriches, and stores telemetry data
* **Dashboard**: Visualizes insights and provides analytics capabilities

## 🔄 Data Flow

### Request Journey Through Your System

1. **User Interaction**: User interacts with your web application
2. **Frontend Capture**: SESSIFY captures performance metrics and user actions
3. **Backend Request**: Application makes service-to-service calls
4. **Mesh Capture**: SP-Istio Agent captures traffic within the service mesh
5. **Data Enrichment**: Service identity and context are added
6. **Transmission & Authentication**: Data is sent to the Softprobe backend and authenticated using public key signatures
7. **Correlation & Analysis**: Backend correlates frontend and backend data
8. **Visualization**: Insights are displayed in the dashboard

### Example: User Login Flow

```
1. User clicks "Login" button
   → SESSIFY captures click event

2. Frontend sends login request to backend
   → SESSIFY traces the API call

3. Backend service processes login
   → SP-Istio Agent captures service traffic

4. Backend calls user database
   → SP-Istio Agent captures database call

5. All data correlated by session ID
   → Softprobe shows complete user journey
```

## 🔐 Authentication & Data Transmission

### Public Key Authentication

Softprobe uses asymmetric cryptography for authentication:

* **No Secrets in Configuration**: Your public key is not sensitive
* **Cryptographic Proofs**: Requests are signed and verified
* **Easy Rotation**: Public keys can be changed without downtime
* **Audit Trail**: Authentication events are logged

### Data Handling & Transport

* **TLS Encryption**: Data is encrypted in transit
* **No Sensitive Storage**: SP-Istio Agent doesn't store sensitive data
* **Least Privilege**: Minimal permissions required for operation
* **Policy-Compatible**: Designed to work within common policies

## 🎯 Core Concepts

### Distributed Tracing

Softprobe provides end-to-end tracing across:

* **Frontend Applications**: User interactions and browser performance
* **Backend Services**: Service-to-service communication
* **External Dependencies**: Database calls, external API requests
* **Infrastructure**: Kubernetes pod and node context

### Service Mesh Integration

Leveraging Istio's capabilities:

* **Automatic Sidecar Injection**: No code changes required
* **Traffic Capture**: Both inbound and outbound traffic
* **Service Identity**: Automatic service discovery and naming
* **Zero-Trust Policies**: Works within service mesh policies

### Observability Data Model

Softprobe captures and correlates:

* **Spans**: Individual operations within a trace
* **Traces**: Complete request journeys across services
* **Metrics**: Performance measurements and resource usage
* **Logs**: Application and infrastructure logs
* **Events**: User interactions and business events

## ⚡ Performance Considerations

### Efficient Data Collection

* **Sampling**: Configurable sampling rates to manage data volume
* **Body Capture Limits**: Control payload sizes to minimize overhead
* **Local Processing**: Lightweight processing within the agent
* **Batched Transmission**: Efficient network utilization

### Resource Optimization

* **Memory Efficient**: WASM plugins have minimal memory footprint
* **CPU Lightweight**: Optimized processing algorithms
* **Network Efficient**: Compression and batching of telemetry data
* **Scalable Architecture**: Handles high-volume production workloads

## 🚀 Getting Started Concepts

### Development vs Production

* **Development**: Quick testing with local clusters and sample data
* **Staging**: Validate configuration with production-like environments
* **Production**: Full-scale deployment with optimized settings

### Environment Strategy

* **Separate Public Keys**: Use different keys for each environment
* **Targeted Deployment**: Deploy to specific namespaces as needed
* **Gradual Rollout**: Start with monitoring, then add tracing and capture

## 📊 What to Expect

### Immediate Benefits

* **Service Dependency Mapping**: Automatic discovery of service relationships
* **Performance Baselines**: Establish normal performance patterns
* **Error Detection**: Identify failing requests and services
* **User Experience Insights**: Understand real user behavior

### Advanced Capabilities

* **Root Cause Analysis**: Drill down into performance issues
* **Capacity Planning**: Identify resource bottlenecks
* **Business Metrics**: Correlate technical data with business outcomes
* **Proactive Monitoring**: Alert on anomalies before users are affected

## 🔧 Integration Patterns

### Microservices Architecture

Ideal for distributed systems with:

* Multiple interacting services
* Complex dependency graphs
* Cross-service troubleshooting needs
* Performance optimization requirements

### Monolithic Applications

Also valuable for single applications with:

* Internal component tracing
* External dependency monitoring
* User experience correlation
* Performance optimization opportunities

### Hybrid Environments

Works across diverse infrastructure:

* Kubernetes clusters
* Virtual machines
* Cloud services
* On-premises infrastructure

## 🎯 Next Steps

Now that you understand Softprobe's concepts:

1. **Set Up Your Account**: [Account Setup & Public Key Management](/en/platform/getting-started/account-setup)
2. **Try Quick Start**: [Quick Start Guide](/en/platform/getting-started/quick-start) for local testing
3. **Deploy to Production**: [Production Installation Guide](/en/platform/deployment/installation)
4. **Configure Advanced**: [Configuration Guide](/en/platform/configuration/config) for custom rules

## ❓ Frequently Asked Questions

### Q: Does Softprobe affect application performance?

**A**: The impact is minimal. SP-Istio Agent uses efficient WASM plugins and the SESSIFY is lightweight. Sampling can be configured to balance insight and overhead.

### Q: How is my data transmitted and authenticated?

**A**: All data is encrypted in transit (TLS) and authenticated via public key signatures. No sensitive credentials are stored in your configuration.

### Q: Can I use Softprobe with existing monitoring tools?

**A**: Yes. Softprobe complements existing tools by providing deeper distributed tracing and user experience correlation.

### Q: How quickly can I see results after installation?

**A**: Immediately. Data starts flowing to the dashboard as soon as the agent is deployed and your applications receive traffic.

***

## 📚 Additional Resources

* [Istio Documentation](https://istio.io/latest/docs/)
* [OpenTelemetry Concepts](https://opentelemetry.io/docs/concepts/)
* [Service Mesh Best Practices](https://istio.io/latest/docs/ops/best-practices/)
* [Observability Fundamentals](https://opentelemetry.io/docs/concepts/observability-primer/)

## 🆘 Support

Need help understanding concepts?

* **Documentation**: Browse our comprehensive guides
* **Community**: Join discussions with other users
* **Support**: Contact our team for technical assistance

***

*Last updated: January 2024*
*Version: v1.0*

---

---
url: /en/platform/advanced-guides/agent-architecture.md
---
# SP-Istio Agent Architecture

This page explains the server-side collection component of Softprobe — the SP‑Istio Agent — including how it is injected, what data it captures, and how it transmits telemetry securely and efficiently.

## Overview

* Deployment: Injected into Istio's Envoy sidecar via WasmPlugin
* Implementation: Rust + WebAssembly (Wasm)
* Capture scope: HTTP request/response, context headers, status codes, latency, and custom tags
* Transmission: Asynchronous batched delivery to the Softprobe backend with retry and queueing

## How It Works

1. Request enters Envoy → the Wasm filter intercepts and parses key fields (method, path, status code, headers, etc.)
2. Collection rules (collectionRules) decide whether to capture the event
3. The event is built and appended to an internal buffer/queue
4. Batches are sent asynchronously to the backend (e.g., o.softprobe.ai), carrying your `public_key` for authentication
5. The backend stores and indexes data; the Dashboard provides session views and cross‑service call trees

## Configuration Highlights

* `traffic_direction`: choose `server` (inbound) or `client` (outbound)
* `collectionRules`: use regex for `host`/`path` to precisely control capture scope
* Service discovery: an EnvoyFilter (Lua) injects `x-sp-service-name` automatically
* External communication: use ServiceEntry + DestinationRule to authorize secure egress to the Softprobe backend

## Performance & Reliability

* Low overhead: Wasm runs inside Envoy with no language/framework intrusion
* Asynchronous delivery: minimizes impact on business requests; supports backpressure and retry
* Resource control: rate limiting and queue sizing to avoid peak pressure

## Security

* Transport encryption: TLS, with egress restricted to approved backend domains
* Authentication: a public key validates tenant identity
* Privacy & compliance: field masking/redaction and configurable capture scope

## Working with OTLP

* Use side‑by‑side: the Agent provides zero‑intrusion HTTP capture; OTLP receives data from SDKs/Collectors
* The backend unifies indexing and correlation, enabling session and trace views across data sources

## Next Steps

* Read the configuration guide and deploy the Agent in a test environment
* Validate data and observability in the Dashboard
* Expand collection rules and tagging progressively based on business scenarios

---

---
url: /en/platform/billing/pricing.md
---
# Plans & Pricing

This page helps you understand Softprobe plan types, quotas, and billing models. Naming is unified: “Pro” in the Dashboard corresponds to “Business” in backend systems — they are the same plan.

## Plan Overview

Softprobe offers three plan families designed for different stages and scale:

* Free — fixed quota, best for evaluation
* Business — fixed quota, suitable for growing teams
* Enterprise — usage‑based billing, scales with demand; for Enterprise support, please contact support@softprobe.ai

## Quotas & Limits

Quotas vary by plan and are designed to balance performance, cost, and reliability.

* Free
  * Storage quota appropriate for evaluation
  * Limited seats and API rate limits
  * Project and Public key limits
  * Over‑quota behavior: blocked

* Business (fixed quota)
  * Larger storage quota for mid‑sized teams
  * More seats and higher API rate limits
  * More projects and Public keys
  * Over‑quota behavior: guided upgrade or additional charges (depending on account settings)

* Enterprise (usage‑based)
  * No fixed quotas — you pay for what you use
  * Common usage dimensions include storage, API calls, and monthly active users

For your exact quotas and entitlements, refer to the Billing section of your Dashboard.

## Billing Models

* Free: $0 per month
* Business (fixed quota): fixed monthly fee
* Enterprise (usage‑based): billed by actual usage (e.g., storage, API calls, active users)

Billing cycles are typically monthly. Charges and invoices are generated based on your plan and actual usage.

## Upgrades & Consistent Naming

* Free → Business (fixed quota): supported; can take effect immediately or at the next billing cycle
* Business (fixed quota) → Enterprise (usage‑based): supported; ideal for fast‑growing or variable workloads

If you need higher quotas or custom terms, contact support@softprobe.ai.

---

---
url: /en/platform/billing/subscriptions.md
---
# Billing & Subscriptions

This page explains the subscription lifecycle, billing cycles and usage, cancellation and resumption, and how plan changes affect your account.

## Subscription Lifecycle

Common states:

* ACTIVE — the subscription is active; billed by cycle or usage
* SUSPENDED — the subscription is paused; service is restricted or stopped
* CANCELLED — the subscription is cancelled; service stops after the current period (or immediately, depending on settings)

Typical transitions:

* Payment failure or insufficient balance triggers SUSPENDED
* User‑initiated cancellation, or “cancel at period end”
* Risk controls may degrade or stop service

## Billing Cycle & Usage

* Fixed plans: a fixed monthly fee; not affected by usage within the cycle
* Usage‑based: usage is aggregated monthly and billed at the end of the cycle
* Usage records and billing history are available for reconciliation and reporting

## Cancellation & Resumption

* Cancellation: choose “cancel immediately” or “cancel at the end of the period”
* Resumption: pay outstanding balances or complete payment to restore ACTIVE state
* Data retention: data is typically retained during suspension or cancellation; may be cleaned up after grace periods (see Terms)

## Plan Changes (Upgrade / Downgrade / Switch Billing Models)

* Free → Pro (Business): upgrade supported; can take effect immediately or at the next billing cycle
* Pro (Business) → Pay‑as‑you‑go (Enterprise): switch supported; you may configure usage caps and alerts
* Downgrade policy: if you downgrade from a higher‑quota plan, over‑quota data may be restricted or require migration
* Proration: mid‑cycle plan changes may trigger proration

## Risk Controls & Spending Limits

* Balance threshold alerts (e.g., reminder at $10, urgent at $5)
* Service degradation strategies: RESTRICTED (strict limits), SUSPENDED (stop service)
* Daily/monthly spending caps and automatic stop‑service protections to prevent unexpected costs

## Stripe Integration & Price Mapping

* Each plan maps to Stripe products and prices (fixed and metered)
* Subscriptions are billed based on these mappings
* Price changes take effect in the next billing period or are prorated based on settings

## Common Actions

* View current subscription and billing cycle: Dashboard → Subscription panel
* Change plan: Dashboard → Subscription panel (Pro/Business, Pay‑as‑you‑go)
* View usage and cost estimates: Dashboard → Usage & Cost
* Cancel or pause: Dashboard → Subscription panel (supports cancel at period end)

## Support

* Enterprise customization: support@softprobe.ai
* Technical questions: see API documentation and FAQ (Support section)

---

---
url: /en/platform/sessify.md
---
# SESSIFY Integration

**Lightweight • Zero-dependency • W3C-compliant distributed tracing**

SESSIFY is Softprobe's session lifecycle management and distributed tracing SDK, specifically designed for frontend application session management and request tracing. It automatically handles session creation, validation, keep-alive, and expiration logic, while injecting W3C-standard trace context into HTTP requests to achieve end-to-end session correlation.

Key benefits include automated session management, distributed tracing across microservices, security with Web Crypto API, and zero dependencies for minimal bundle size.

::: info
The Quick Start environment already has the SESSIFY (`@softprobe/sessify`) pre-installed and enabled. This document is for integrating the SDK into your own frontend applications (React/Vue/Next.js, etc.).
:::

## Prerequisites

* A modern web application (React, Vue, Next.js, or plain JavaScript)
* Node.js 16+ and a bundler (e.g., Vite, Webpack)

## Installation

```bash
# npm
npm install @softprobe/sessify

# yarn
yarn add @softprobe/sessify

# pnpm
pnpm add @softprobe/sessify
```

## Initialization

```ts
// React / Next.js Example
import { useEffect } from 'react';
import { initSessify } from '@softprobe/sessify';

useEffect(() => {
  // Initialize with default settings
  initSessify({
  });
}, []);

// Vanilla JS Example
import { initSessify } from '@softprobe/sessify';

initSessify({
});
```

## Key Features

## Core Usage

Once initialized, you can manage sessions anywhere in your application.

```javascript
import { getSessionId, startSession, endSession, isSessionActive } from '@softprobe/sessify';

// 1. Get Current Session
// If expired or missing, this automatically creates a new one.
const sessionId = getSessionId();
console.log('Active Session:', sessionId);

// 2. Check Status
if (isSessionActive()) {
  console.log('User is currently active');
}

// 3. Force Refresh (e.g., on Login)
// Invalidates the old session and starts a fresh one immediately.
const newSessionId = startSession();

// 4. Logout / Cleanup
// Clears storage and invalidates the session.
endSession();
```

## Configuration

The initSessify function accepts a configuration object to tailor behavior to your needs.

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| customTraceState | object | {} | Custom key-value pairs to include in tracestate headers. If provided |
| sessionStorageType | session/local | 'session' | Controls persistence. 'local' survives browser restarts; 'session' clears when the tab closes. |

### Custom Trace State Example

Enhance your request tracing by adding context like environment or version:

```javascript
initSessify({
  sessionStorageType: 'local',
  customTraceState: {
    'x-sp-env': 'production',
    'x-sp-ver': '1.0.0',
    'x-sp-tier': 'premium'
  }
});
```

Resulting Header: tracestate: x-sp-session-id=...,x-sp-env=production,x-sp-ver=1.0.0...

## Technical Details

### Session Lifecycle Strategy

* **Creation**: Uses a base36 timestamp combined with a Web Crypto API random string for a ~16 character unique ID.

* **Validation**: Automatically checks for timeout on every access. If the timeout is exceeded, the old session is discarded and a new one is seamlessly generated.

### HTTP Interception

@softprobe/sessify monkey-patches the global fetch API (safely) to:

* Check if a session is active.
* Inject the tracestate header conforming to W3C standards.
* Append your siteName or customTraceState and the current session\_id.

---

---
url: /en/platform/support/faq.md
---
## FAQ

### Q1: What if I lose the minimal.yaml configuration file?

**A:** The `minimal.yaml` file is only downloaded once when you create a public key. If you lose it:

1. Delete the current public key from your dashboard
2. Create a new public key (this will generate a new `minimal.yaml` file)
3. Save the new configuration file immediately

### Q2: Is my public key a secret?

**A:** No, your public key is not a secret. It's designed to be publicly visible and can be safely stored in version control. Integrity and authenticity are guaranteed by asymmetric cryptography: telemetry is signed (or authenticated) using mechanisms tied to your tenant and verified server-side against known public keys. No sensitive private keys are stored in your Kubernetes configuration.

### Q3: Can a public key be reused across environments?

**A:** Yes, a public key can be used in multiple environments, but we recommend:

* Use separate public keys for production and non-production environments
* This allows for better access control and auditing
* Different environments may have different rate limits and permissions

### Q4: How do I manage multiple projects?

**A:** We recommend creating different tenant groups for different projects:

1. Create a separate tenant group for each project
2. Generate separate public keys for each project
3. Invite relevant team members to the corresponding tenant groups

### Q5: Are there usage limits for public keys?

**A:** Usage limits may include:

* Request rate limits based on your subscription plan
* Data storage quotas
* Feature permission restrictions
* Check your subscription plan for specific limits

### Q6: What if there is no data after configuration?

**A:** Please check:

1. If the `minimal.yaml` file was applied correctly with `kubectl apply -f minimal.yaml`
2. If the Istio service mesh is properly installed in your cluster
3. If the network connection is normal
4. If the application pods have been restarted after applying the configuration
5. If there are any error messages in the application logs

***

## Related Topics

* [Quick Start](/en/platform/getting-started/quick-start)
* [Installation Guide](/en/platform/deployment/installation)
* [Configuration Reference](/en/platform/configuration/config)
* [Core Concepts](/en/platform/advanced-guides/concepts)

## Key Features at a Glance

* Dashboard: Modern responsive UI, real-time analytics, heatmaps, and user session recordings; built with React + Vite + TailwindCSS.
* OTEL Backend: Ingests traces/logs/metrics; BigQuery storage with indexed request\_body\_hash for fast lookups and correlation.
* Auth Service: Multi-tenant authentication/authorization; Public key lifecycle, JWT utilities, role-based access (OWNER/ADMIN/MEMBER).

::: info Why this matters
These components work together to deliver end-to-end observability with session correlation, efficient data lookup, and secure multi-tenant controls.
:::

## Precautions & Best Practices

* Environment separation: Use distinct tenants/public keys for prod vs. non-prod; start with conservative sampling and ramp up.
* Public key hygiene: Full keys are shown only at creation; store securely; rotate periodically; never commit secrets.
* Rate limits: Each Public key has hourly limits; design clients with retries/backoff and monitor usage counters.
* Data sensitivity: Avoid sending PII in attributes; rely on signing/verification and TLS; prefer hashed request bodies for correlation.
* Performance: BigQuery indexes grow with data; combine filters with request\_body\_hash; consider time partitions and cache if needed.
* Deployment: Use CI/CD (e.g., ArgoCD) with image tags; ensure dashboard release and backend schema changes are coordinated.

::: tip Best practice
Create separate tenant groups and public keys per project/environment to ease auditing and access control.
:::

::: warning Caution
Do not paste full Public keys into issue trackers or chat tools. Use masked previews in UI and rotate compromised keys immediately.
:::

## Plans & Quotas

* Storage quota per tenant: Configurable (e.g., storageQuotaGb in tenant settings). Usage should be monitored and alerted.
* API rate limits: Per-key hourly caps (rateLimitPerHour). Plan capacity around expected ingestion/query workloads.
* Multi-tenant boundaries: Resources (datasets/buckets) are scoped per tenant and environment for isolation.
* Enterprise options: Higher quotas, SLOs, and custom retention are available. Contact support@softprobe.ai for details.

---

---
url: /en/testing.md
---

# Softprobe Testing

**Record real traffic. Replay with automatic mocks. Compare without writing test cases.**

::: tip Ready to start?
🚀 If you are new to Softprobe, go straight to our **[Getting Started](/en/testing/getting-started)** guide! Try it out using a pre-built JAR in 5 minutes, and then onboard your own application.
:::

Softprobe Testing is record-and-replay regression for **Java** services. Attach the Softprobe Java agent as a `-javaagent`, capture inbound API traffic and outbound dependency calls in the background, then replay recorded cases in a test environment while dependencies are mocked from stored data and results are compared automatically.

::: info Product areas on this site
| Area | You use it when... |
|------|---------------------|
| **[Platform](/en/platform/)** | You need Istio/Envoy mesh capture, SESSIFY session context, or the observability dashboard |
| **Testing** (this section) | You need Java record/replay, the JVM agent, policies, replay semantics, installation, commands, and automation |
:::

## Why record-and-replay

Traditional integration tests require maintaining environments, seed data, and hand-written cases. Softprobe Testing instead:

* **No application code changes** — bytecode instrumentation via `sp-agent.jar`
* **High coverage from real traffic** — production or staging requests become replay cases
* **Isolated replay** — databases, HTTP clients, Redis, RPC, caches, and more are mocked from recordings so you do not need live downstreams in the test environment
* **Safe WRITE paths** — recorded dependency behavior is replayed without dirtying shared databases during regression
* **Lower noise** — compare rules, time mocking, and ignore nodes handle timestamps, random IDs, and environment-specific fields

## How the pieces fit together

| Component | Role |
|-----------|------|
| **Your Java service** | The application under test, started with `-javaagent:…/sp-agent.jar` |
| **Softprobe Java agent** | Records and replays at runtime; mocks dependencies during replay |
| **Softprobe backend** (`:8090`) | Deployed via Helm. Stores cases (MongoDB), serves policies, runs replay plans, computes diffs |
| **`sp` command** (optional) | Registers apps, applies policies, starts replay, inspects failures — see [Commands](/en/testing/commands/) |
| **Dashboard / workbench** (optional) | Visual diff and trace review |

```mermaid
flowchart LR
  App[JVM app under test]
  Agent[Java agent]
  Backend[sp-backend]
  App --> Agent
  Agent --> Backend
```

## Record → replay → compare (short)

1. **Record** — Agent captures entry traffic (e.g. HTTP `Servlet`) and dependency calls (`HttpClient`, `Database`, `Redis`, …) while the app handles real requests.
2. **Store** — Backend persists each interaction; replay hot path uses Redis-backed mock cache.
3. **Replay** — Schedule service sends recorded entry requests to your **test instance** (`targetEnv` URL). The agent returns recorded dependency responses instead of calling real downstreams.
4. **Compare** — Engine diffs recorded vs replay traffic; policies define what to ignore or how strictly to match.

Details: [Record traffic](/en/testing/recording) · [How it works](/en/testing/how-it-works) · [Replay and diff](/en/testing/replay-and-diff)

## Platform agent ≠ Java agent

The [Platform](/en/platform/advanced-guides/agent-architecture) **SP-Istio agent** runs in Envoy sidecars and captures mesh HTTP traffic. **Softprobe Testing** uses the **JVM agent** attached with `-javaagent`. They solve different problems; both can feed the same backend for correlation in SaaS deployments.

## Who should read this section

* **Java developers** adopting record/replay for the first time
* **QA / release engineers** running regression without full downstream stacks
* **Platform engineers** wiring `sp-backend` and agent startup in K8s or VM images
* **Automation authors** — start here for concepts, then [Commands](/en/testing/commands/) for `sp` automation

## Quick links

* [Getting Started](/en/testing/getting-started) — Try the 5-minute [Travel OTA](https://github.com/softprobe/demo-ota) demo and onboard your application.
* [Install Softprobe](/en/testing/installation/) — Install, set up, launch coding, diagnose, and upgrade with `sp`.
* [Record traffic](/en/testing/recording) — Core workflow step 1: build the case corpus.
* [Java agent](/en/testing/java-agent) — Attach, JVM flags, and production safety.
* [Policies overview](/en/testing/policies) — YAML by lifecycle phase.
* [Replay and diff](/en/testing/replay-and-diff) — Core workflow step 2: regression run.
* [Webhook and CI/CD](/en/testing/webhook-and-ci) — Post-deploy replay and pipeline gates.
* [Supported frameworks](/en/testing/supported-frameworks)

---

---
url: /en/testing/getting-started.md
---

# Getting Started with Softprobe Testing

Welcome to Softprobe Testing! This guide provides a single, cohesive path to get you started with record-and-replay in minutes using our pre-built Travel OTA demo application, followed by a quick wrapup on how to connect your own Java service.

::: tip Pick your interface
The steps below for registering an app, viewing recordings, and replaying can all be done in the **Web console** or with the **`sp` command**. Use the switch at the top of each card to choose one — your choice is remembered and stays consistent across the whole docs site.
:::

***

## Prerequisites

* **Java 8 or higher** (Java 17/21 recommended) installed and on your `PATH`.
* **`sp` command** installed (`curl -fsSL https://install.softprobe.ai/install.sh | bash`) — see [Install the CLI](/en/testing/installation/).
* Your **Softprobe Helm chart** is deployed and running (with the Softprobe backend available, e.g. at `http://<your-backend-host>:8090`) — not deployed yet? Follow [Server (Helm)](/en/testing/installation/server) first.

***

## 1. Verify Java Installation

Ensure Java is configured correctly and available in your terminal:

```bash
java -version
```

## 2. Download the Demo Application

Download the pre-built [Travel OTA](https://github.com/softprobe/demo-ota) application JAR (either [click to download travel-ota.jar](https://github.com/softprobe/demo-ota/releases/download/v1.1.0/travel-ota.jar) directly via your browser, or run the command below):

```bash
curl -L -O https://github.com/softprobe/demo-ota/releases/download/v1.1.0/travel-ota.jar
```

## 3. Download the Softprobe Agent

Download the Softprobe Java agent JAR (either [click to download sp-agent.jar](https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar) directly via your browser, or run the command below):

```bash
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
```

See [Download Java agent](/en/testing/download-java-agent) for immutable release versions.

## 4. Register the Application

Register the demo application in Softprobe to receive a unique `appId` (a 16-character hex identifier):

1. Open your Softprobe Dashboard.
2. Navigate to **Apps** and click **Create App**.
3. Enter `travel-ota` as the name and click **Save**.
4. Copy the generated **App ID**.

Run the following command to register the app with `sp`:

```bash
export SP_API_URL=http://localhost:8090   # Point to your Helm/local backend
sp app create travel-ota
```

Save the `appId` returned in the JSON response.

## 5. Start the Application with the Agent

Start the demo app with the `-javaagent` flag, passing your `appId`:

```bash
java -javaagent:sp-agent.jar \
     -Dsp.app.id=<your-app-id> \
     -Dsp.api.url=http://localhost:8090 \
     -jar travel-ota.jar
```

*(If your Helm backend is hosted at another address, replace `http://localhost:8090` with your actual backend URL).*

The application is now running locally at <http://localhost:8080>.

## 6. Perform a Booking (Generate Traffic)

Open <http://localhost:8080> in your browser:

1. Click **Search** to view available flights.
2. Select a flight and click **Book**.
3. Complete the checkout/payment process.

The Softprobe agent automatically intercepts and captures this entire transaction.

## 7. View Recorded Data

1. Log in to your Softprobe Dashboard.
2. Go to the **Workbench** or **Recordings** tab.
3. Select `travel-ota` from the app dropdown.
4. Browse the recorded traces and inspect the deep dependency graphs.

List cases recorded for the app in the last 10 minutes:

```bash
sp record case list --app <your-app-id> --since -10m
```

## 8. Replay the Recordings

Replay executes recorded transactions against a target environment with automated dependency mocking (your database and downstreams do not need to be set up).

1. Navigate to the **Replays** tab and click **New Replay Plan**.
2. Select `travel-ota`.
3. Choose the cases to replay.
4. Set the target environment to `http://localhost:8080` and click **Run**.

Trigger the replay plan pointing to your local running instance:

```bash
sp replay run --app <your-app-id> --env http://localhost:8080
```

Watch the replay status until it reaches a terminal state:

```bash
sp replay status <replay-plan-id> --watch
```

***

## Onboard Your Own Application

Onboarding your own service is **exactly the same** as running the Travel OTA demo! Your application is just another `.jar` file started with the same Softprobe Java Agent and your custom App ID.

To connect your own application:

1. **Register Your App**: Create a new 16-character App ID either via **Apps** → **Create App** in the Web UI, or run:
   ```bash
   sp app create <your-app-name>
   ```
2. **Attach the Agent**: Start your own JVM service with the same `-javaagent` flag, passing your new App ID and Helm backend address:
   ```bash
   java -javaagent:sp-agent.jar \
        -Dsp.app.id=<your-new-app-id> \
        -Dsp.api.url=http://<your-backend-host>:8090 \
        -jar your-own-application.jar
   ```
3. **Verify and Replay**: Record traffic, list cases, and trigger replays exactly as you did with the demo app.

## Next: the core workflow

The demo is just the start. Take your own application through the **core workflow**:

1. **[Record traffic](/en/testing/recording)** — capture real cases from production/staging and build your regression corpus
2. **[Replay & diff](/en/testing/replay-and-diff)** — run a regression before every release
3. **[Review diffs](/en/testing/review-diffs-in-the-web-ui)** — understand failing cases and accept the differences that aren't bugs
4. **[Configure compare rules](/en/testing/compare-rules-web-ui)** — turn always-changing fields into rules so they stop false-alarming

For deeper configuration, see [Java Agent Configuration](/en/testing/java-agent) (JVM properties, Tomcat/Docker) and the [Policies Overview](/en/testing/policies) (declarative YAML for CI/GitOps).

---

---
url: /en/testing/how-it-works.md
---

# How record-and-replay works

Softprobe Testing captures a **transaction**: one inbound request plus every dependency the code touched on that path. On replay, the same code path runs again; external calls are satisfied from stored data instead of live systems.

## End-to-end sequence

```mermaid
sequenceDiagram
  participant App as App under test
  participant Agent as Java agent
  participant Storage as sp-backend storage
  participant Redis as Redis mock cache
  participant Mongo as MongoDB
  participant Schedule as Replay schedule

  Note over App,Mongo: Recording
  App->>Agent: dependency call
  Agent->>Storage: upload SpMocker
  Storage->>Mongo: persist

  Note over Schedule,Redis: Replay prep
  Schedule->>Storage: preload mocks
  Storage->>Redis: record category keys

  Note over App,Redis: Replay
  App->>Agent: same call path
  Agent->>Storage: query mock
  Storage->>Redis: get recorded response
  Storage-->>Agent: Zstd payload
  Agent-->>App: mocked dependency

  Note over Schedule,Mongo: Compare
  Schedule->>Storage: record vs replay lists
  Storage->>Schedule: diff results
```

## Recording phase {#recording-phase}

When recording is enabled, the agent intercepts:

| Kind | Examples | What is stored |
|------|----------|----------------|
| **Entry** | `Servlet`, `DubboProvider`, `NettyProvider` | Inbound API request and response |
| **Dependency** | `HttpClient`, `Database`, `Redis`, `DubboConsumer`, `DynamicClass`, … | Outbound request and response per call |

Each interaction is a **mocker** row keyed by `appId`, trace/case identifiers, category, and operation name.

**How to produce cases:** [Record traffic](/en/testing/recording) · **Recording scope config:** [Recording policy](/en/testing/policies#recording-policy)

::: tip
Cases are created only from **instrumented traffic**. There is no supported workflow to hand-author cases in the CLI; send real or synthetic traffic through the app while the agent is recording.
:::

## Replay phase

A **replay plan** selects recorded cases and drives entry HTTP traffic to **`targetEnv`** — the base URL of the service under test (for example `http://order-service.test:8080`). This is **not** the same as `SP_API_URL`, which points at sp-backend.

During replay:

1. The schedule service sends the recorded entry request to `targetEnv`.
2. The app under test must run with the agent attached (recording frequency can be set to zero on the replay machine).
3. On each dependency call, the agent asks storage for the **recorded response** matching that call.
4. Storage serves from Redis when preloaded; otherwise it loads from MongoDB.

**Entry vs dependency behavior:**

* **Entry** (`entryPoint` categories): storage records the replay request; the live app still executes your controller/handler.
* **Dependency**: storage returns the recorded mock body so the app does not hit the real database or external API.

## Compare phase

After replay, the compare engine pairs recorded and replay mockers. Failures mean a mismatch on the main response or a dependency (missing call, wrong value, extra call).

Typical diff patterns:

* **Value diff** — same dependency was called but response body differs
* **Missing call** — replay did not invoke a dependency that was recorded
* **Extra call** — replay invoked something not present in the recording

Use [compare policy](/en/testing/policies) and [Replay and diff](/en/testing/replay-and-diff) to ignore noisy fields (timestamps, tokens, IPs).

## Pedagogical example

Consider a method that parses an IP string and calls a validator:

```java
public Integer parseIp(String ip) {
    int result = 0;
    if (checkFormat(ip)) {
        String[] ipArray = ip.split("\\.");
        for (int i = 0; i < ipArray.length; i++) {
            result = result << 8;
            result += Integer.parseInt(ipArray[i]);
        }
    }
    return result;
}
```

**Recording** — the agent saves arguments and return values when `needRecord()` is true.

**Replay** — the agent short-circuits with stored results so `checkFormat` and parsing behave as they did during capture, even if the test environment differs.

Dynamic classes (local cache, encryption helpers, system time) use the same model; configure them via recording policy and mock policy rather than changing application code.

## Storage layout (conceptual)

| Store | Role |
|-------|------|
| **MongoDB** | Durable recordings, replay plans, compare results |
| **Redis** | Hot mock cache during replay (`record:{category}:{recordId}:…`) |

When self-hosted, `sp-backend` is typically a single process on port **8090** serving API, storage, and schedule.

## Related

* [Java agent](/en/testing/java-agent)
* [Policies](/en/testing/policies)
* [Replay and diff](/en/testing/replay-and-diff)
* [CLI concepts](/en/testing/agents/concepts)

---

---
url: /en/testing/installation/server.md
---

# Install Softprobe Server

Install the unified Softprobe backend on Kubernetes with Helm. The chart deploys **Redis** in-cluster and either **bundled MongoDB** or connects to your **existing MongoDB** server.

Chart **v4.3.x+** also enables the [unified log pipeline](#unified-log-pipeline) by default (Vector, Parquet PVC, compaction). Fresh installs need only the MongoDB and encryption keys below — no separate `logPipeline` block required.

**Prerequisites:** Kubernetes 1.24+, Helm 3.x, GCR pull credentials from Softprobe, and `encryption.secretKey` for at-rest payload encryption.

For **bundled** MongoDB, your cluster needs a default or configured `StorageClass` for the MongoDB PVC.

## MongoDB modes (pick one)

Configure **exactly one** mode in your values file. `helm install` fails if both are set or neither is set.

| Mode | Set in values | Chart deploys MongoDB? |
|------|---------------|------------------------|
| **Bundled** | `mongodb.bundled.auth.password` | Yes — Deployment, Service, PVC |
| **External** | `mongodb.connectionString` | No |

**Shared external MongoDB:** multiple Helm releases can use one MongoDB host. Put a **unique database name** in each connection string (for example `acme_prod_sp_storage_db`).

Download the example values file (always the current version):

[values.example.yaml](https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/latest/values.example.yaml)

## Install

### 1. Helm repo and namespace

```bash
helm repo add softprobe \
  https://storage.googleapis.com/softprobe-published-files/helm/sp-backend
helm repo update

kubectl create namespace softprobe
```

### 2. GCR pull secret

```bash
kubectl create secret docker-registry softprobe-gcr-pull \
  --docker-server=https://gcr.io \
  --docker-username=_json_key \
  --docker-password="$(cat softprobe-registry-puller.json)" \
  --namespace softprobe
```

::: warning
Use both `gcr.io` and `https://gcr.io` auth entries. See the chart README if plain `kubectl create secret docker-registry` causes `ImagePullBackOff`.
:::

### 3. Values file

Copy the example to `values.yaml` and edit for your environment. **Pick one MongoDB block below.**

#### Mode A — Bundled MongoDB (in-cluster)

Chart deploys MongoDB 7 and Redis 7 alongside sp-backend.

```yaml
image:
  tag: "v4.3.10"
  pullSecrets:
    - name: softprobe-gcr-pull

mongodb:
  bundled:
    auth:
      password: "CHANGE_ME_mongo_password"
    # Optional: storageClass, storageSize, resources, placement — see values.example.yaml

encryption:
  enabled: true
  secretKey: "CHANGE_ME_base64_32_byte_key"
```

#### Mode B — External MongoDB

Use your existing MongoDB server. Do **not** set `mongodb.bundled.auth.password`.

```yaml
image:
  tag: "v4.3.10"
  pullSecrets:
    - name: softprobe-gcr-pull

mongodb:
  connectionString: "mongodb://USER:PASS@mongo.host:27017/your_release_sp_storage_db?authSource=admin"

encryption:
  enabled: true
  secretKey: "CHANGE_ME_base64_32_byte_key"
```

### 4. Helm install

Use the chart version and image tag from your Softprobe release (`v4.3.10` → chart `4.3.10`, image `v4.3.10`).

```bash
helm install softprobe softprobe/sp-backend \
  --version 4.3.10 \
  --namespace softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

## Verify

```bash
kubectl get pods -n softprobe
kubectl port-forward -n softprobe svc/softprobe-sp-backend 8090:8090
curl -s http://127.0.0.1:8090/actuator/health
```

**Bundled mode:** expect pods for `mongodb`, `redis`, `sp-backend`, and (chart v4.3.x+) `log-vector`.

**External mode:** expect `redis`, `sp-backend`, and `log-vector` (no `{release}-mongo` pod).

## Upgrade an existing release

Use the same release name, namespace, and `values.yaml` you used at install. A Softprobe release tag maps to Helm as:

| Release tag | Chart `--version` | `image.tag` |
|-------------|-------------------|-------------|
| `v4.3.10` | `4.3.10` | `v4.3.10` |

### Before you upgrade

1. **Keep your existing `values.yaml`** — you do not need to replace it. Helm merges your file with the new chart defaults for any key you omitted.
2. **Older file without `logPipeline`?** If you installed on v4.3.5 or earlier with only `image`, `mongodb`, and `encryption`, bump `--version` and `image.tag` only. Missing keys inherit chart defaults — **`logPipeline.enabled` is `true`**, so Vector, the Parquet PVC (local mode), compaction, and retention are added on upgrade. Use `--dry-run` first to preview new resources.
3. **Review optional overrides** — download the current [values.example.yaml](https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/latest/values.example.yaml) and merge only what you need (PVC `storageClass`, `placement`, S3 backend). Do **not** change `encryption.secretKey` — existing encrypted payloads depend on it.
4. **Keep your MongoDB mode** — do not switch between bundled and external MongoDB on upgrade.
5. **Confirm registry access** — the `softprobe-gcr-pull` secret must still be valid for the new `image.tag`.
6. **Preview the diff** (optional):

```bash
helm repo update
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false \
  --dry-run
```

### Upgrade from the Helm repo

Typical upgrade — same `values.yaml` as install, new chart and image version:

```bash
helm repo update
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

Replace `softprobe` with your release name if different. Pin **`image.tag`** to the semver release Softprobe gave you — not `latest`.

**Example — old values file, no `logPipeline` block:** your file still looks like this:

```yaml
image:
  tag: "v4.3.5"
  pullSecrets:
    - name: softprobe-gcr-pull
mongodb:
  bundled:
    auth:
      password: "your-existing-password"
encryption:
  enabled: true
  secretKey: "your-existing-key"
```

Upgrade command (no edits required):

```bash
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

Helm adds log-pipeline resources from chart defaults. After rollout, instrument workloads with `-Dsp.api.url` pointing at sp-backend (see [Agent log export](#agent-log-export)).

### Upgrade from a downloaded chart package

If you install offline or verify SHA-256 from GCS:

```bash
curl -fLO "https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/v4.3.10/sp-backend-4.3.10.tgz"

helm upgrade softprobe ./sp-backend-4.3.10.tgz \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

### Verify after upgrade

```bash
kubectl rollout status -n softprobe deploy/softprobe-sp-backend
kubectl get pods -n softprobe
kubectl get pods,cronjob,pvc -n softprobe | grep -E 'log-vector|log-parquet|compaction|retention'
kubectl port-forward -n softprobe svc/softprobe-sp-backend 8090:8090
curl -s http://127.0.0.1:8090/actuator/health
```

Expect a rolling restart of `sp-backend` (and Redis if the chart template changed). Bundled MongoDB data on the existing PVC is preserved. `sp-backend` may take up to ~2 minutes to become ready after the new pod starts (JVM warm-up).

On v4.3.10+ you should also see `log-vector` and a `log-parquet` PVC (local storage). Instrument workloads with:

```text
-Dsp.api.url=http://<release>-sp-backend.<namespace>.svc.cluster.local:8090
```

Log export uses `{sp.api.url}/v1/logs`; sp-backend proxies to Vector internally.

### Customize or disable the log pipeline

The pipeline is **on by default**. Merge overrides from [values.example.yaml](https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/latest/values.example.yaml) only when you need non-default storage, placement, or S3:

```yaml
logPipeline:
  parquet:
    storageSize: 100Gi
    storageClass: managed-csi
```

To **disable** on upgrade, add before running `helm upgrade`:

```yaml
logPipeline:
  enabled: false
```

Full options: [Unified log pipeline](#unified-log-pipeline).

### Upgrade troubleshooting

| Symptom | Check |
|---------|--------|
| `ImagePullBackOff` after upgrade | New `image.tag` exists in GCR; `softprobe-gcr-pull` secret valid |
| `helm upgrade` fails on MongoDB | Still set **one** of `mongodb.connectionString` or `mongodb.bundled.auth.password` — do not clear both |
| New log Parquet PVC pending | Cluster `StorageClass` — set `logPipeline.parquet.storageClass` |
| Log query empty after enabling pipeline | Agent OTLP endpoint and trace bounds — see [log pipeline troubleshooting](#troubleshooting) |

## Unified log pipeline {#unified-log-pipeline}

Enable correlated log ingest, Parquet storage, and trace-id query (`sp logs` / `GET /api/recorder/logs`) from the **sp-backend** Helm chart.

**Prerequisites:** a healthy `sp-backend` release on chart **v4.3.x+**. The pipeline is **enabled by default** (`logPipeline.enabled: true`). Instrumented workloads need `-Dsp.api.url` pointing at sp-backend (see [Agent log export](#agent-log-export)).

### What the chart deploys

When `logPipeline.enabled: true`, Helm adds:

| Resource | Purpose |
|----------|---------|
| **Vector** (`{release}-log-vector`) | OTLP log ingest (gRPC/HTTP + agent JSON on `:4320`) |
| **rclone gateway** (`{release}-log-rclone`, local + Azure Blob) | Shared S3 gateway Deployment + Service that Vector and sp-backend use to write/read Parquet |
| **Parquet PVC** (local mode only) | Durable storage mounted only by the rclone gateway; all other pods reach it via the gateway's S3 endpoint |
| **Compaction CronJob** (all backends) | Merges closed-hour minute files → `part-hourly.parquet` (DuckDB) |
| **Retention CronJob** (optional, all backends) | Prunes Parquet older than `ttlDays` |

sp-backend is wired for Parquet reads and exports its own diagnostic logs when the pipeline is enabled (`OTEL_ENABLED=true`, `OTEL_LOGS_EXPORTER=otlp-filtered`).

### Configure in Helm

The chart enables the pipeline by default. Override in your values file (or `--set` on install/upgrade) when you need non-default storage, placement, or to disable:

```yaml
logPipeline:
  enabled: true   # default; set false to disable
  storage:
    backend: local   # local | s3 | azure_blob — see "Choose where logs are stored"
  parquet:
    storageSize: 100Gi
    # storageClass: managed-csi   # optional — cluster default if omitted
  retention:
    ttlDays: 4               # set "" to disable retention CronJob
    cleanupSchedule: "0 3 * * *"
  compaction:
    enabled: true            # local storage only
    schedule: "15 * * * *"   # previous closed UTC hour
  # Pin Vector + maintenance jobs to the same node pool as sp-backend when using taints:
  placement:
    nodeSelector:
      workload: softprobe-backend
    tolerations:
      - key: workload
        operator: Equal
        value: backend
        effect: NoSchedule
```

**Upgrading from an older `values.yaml` without a `logPipeline` block?** You do not need to add one — chart defaults apply and the pipeline is deployed on upgrade. See [Upgrade an existing release](#upgrade-an-existing-release).

Example upgrade with explicit overrides (optional):

```bash
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

Pin **`image.tag`** to a semver release (for example `v4.3.10`), not `latest`, so the backend matches your chart version.

### Verify log pipeline

```bash
kubectl get pods,cronjob,pvc -n softprobe | grep -E 'log-vector|log-parquet|compaction|retention'
kubectl port-forward -n softprobe svc/softprobe-sp-backend 8090:8090
```

Run a canned lookup (replace trace id and bounds):

```bash
export SP_API_URL=http://127.0.0.1:8090
sp logs --trace-id <32-hex> --since 2026-06-27T10:00:00Z --until 2026-06-27T10:05:00Z
```

Or:

```bash
curl -s "$SP_API_URL/api/recorder/logs?trace_id=<id>&since=2026-06-27T10:00:00Z&until=2026-06-27T10:05:00Z"
```

v1 has **no** dedicated pipeline health API — a successful trace-id lookup confirms ingest, storage, and query wiring.

The same Vector Deployment also stores Softprobe **metrics** under a sibling `metrics/` Parquet dataset. See [Metrics data plane](./metrics-data-plane) for `POST /v1/metrics` and `GET /api/recorder/metrics`.

### Agent log export {#agent-log-export}

Point the Java agent at sp-backend only:

```text
-Dsp.api.url=http://<release>-sp-backend.<namespace>.svc.cluster.local:8090
```

For release `softprobe` in namespace `softprobe`:

```text
-Dsp.api.url=http://softprobe-sp-backend.softprobe.svc.cluster.local:8090
```

Correlated application and agent logs export to `{sp.api.url}/v1/logs` during record and replay. sp-backend proxies agent JSON to Vector `:4320` and OTLP to `:4318` when the log pipeline is enabled.

Optional advanced override — direct Vector ingest (bypasses backend proxy):

```text
-Dsp.otel.exporter.otlp.log.endpoint=http://<release>-log-vector.<namespace>.svc.cluster.local:4320/v1/logs
```

Legacy capture flags (`sp.record.user.log`, `sp-capture-log`, `sp.user.log.level`, etc.) are not used in v1.

### Choose where logs are stored

Set `logPipeline.storage.backend` to pick where Parquet log files live. Pick one:

| Backend | `logPipeline.storage.backend` | Best for | Credentials you provide |
|---------|-------------------------------|----------|-------------------------|
| **Local disk** (default) | `local` | Single cluster, simplest setup | None (in-cluster volume) |
| **S3-compatible** | `s3` | AWS S3, MinIO, GCS, any S3 API | `access-key-id` + `secret-access-key` |
| **Azure Blob** | `azure_blob` | Azure Storage accounts | `account-key` |

Querying, compaction, and retention behave identically across all three — only the storage location and credentials differ.

#### Option 1 — Local disk (default)

Nothing to set up. The chart creates a PersistentVolumeClaim and stores Parquet there. Adjust size/class only if needed:

```yaml
logPipeline:
  storage:
    backend: local
  parquet:
    storageSize: 100Gi
    storageClass: ""      # cluster default; or e.g. managed-csi (AKS), gp3 (EKS)
```

Back up by snapshotting the `{release}-log-parquet` PVC.

#### Option 2 — S3-compatible bucket

**Step 1 — create the credentials secret.** It must contain exactly these two keys:

```bash
kubectl create secret generic softprobe-log-s3-credentials -n softprobe \
  --from-literal=access-key-id='AKIA...' \
  --from-literal=secret-access-key='...'
```

| Secret key | Value |
|------------|-------|
| `access-key-id` | Bucket access key ID |
| `secret-access-key` | Bucket secret access key |

**Step 2 — point the chart at your bucket and that secret:**

```yaml
logPipeline:
  storage:
    backend: s3
    s3:
      bucket: my-softprobe-logs
      endpoint: https://s3.amazonaws.com   # or your MinIO / GCS / other S3 endpoint
      region: us-east-1
      forcePathStyle: true                 # keep true for MinIO
      prefix: ""                           # optional key prefix inside the bucket
      existingSecret: softprobe-log-s3-credentials
```

The bucket must already exist. No Parquet PVC is created in this mode.

#### Option 3 — Azure Blob container

**Step 1 — create the credentials secret.** It must contain exactly one key, `account-key`, holding your storage account access key:

```bash
kubectl create secret generic softprobe-log-azure-credentials -n softprobe \
  --from-literal=account-key='<storage account key>'
```

| Secret key | Value |
|------------|-------|
| `account-key` | Storage account access key (Azure portal → **Storage account → Security + networking → Access keys**, or `az storage account keys list --account-name <account> --query '[0].value' -o tsv`) |

**Step 2 — point the chart at your container and that secret:**

```yaml
logPipeline:
  storage:
    backend: azure_blob
    azureBlob:
      container: my-softprobe-logs
      accountName: <account>
      endpoint: https://<account>.blob.core.windows.net
      prefix: ""                           # optional prefix inside the container
      existingSecret: softprobe-log-azure-credentials
```

The container must already exist. Softprobe authenticates with the account name + access key (Azure Shared Key). No Parquet PVC is created in this mode.

#### Compaction & retention (all backends)

Both run automatically and identically for `local`, `s3`, and `azure_blob` — no cloud lifecycle rules required:

* **Compaction** (`logPipeline.compaction`, hourly, on by default): merges each closed UTC hour's minute files into a single `part-hourly.parquet`.
* **Retention** (`logPipeline.retention.ttlDays`, default `4`): deletes Parquet older than N days. Set `ttlDays: ""` to keep logs forever.

> Compaction uses the `softprobe/duckdb:1.1.3` image (`linux/amd64`). On Apple Silicon dev clusters, build/load an `arm64` build (`make duckdb-image DUCKDB_PLATFORM=linux/arm64`) and override `logPipeline.compaction.image`.

**Security:** end users and Agent Skills **must never** receive bucket or storage-account credentials — they query only through `sp logs` / `GET /api/recorder/logs`.

### Helm values reference

| Value | Description |
|-------|-------------|
| `logPipeline.enabled` | Deploy Vector, storage, and query wiring (default `true`) |
| `logPipeline.storage.backend` | `local` (PVC), `s3`, or `azure_blob` |
| `logPipeline.parquet.storageSize` / `storageClass` | Local Parquet PVC size and class |
| `logPipeline.vector.image` | Vector image (default `timberio/vector:0.56.0-debian`) |
| `logPipeline.vector.resources` | CPU/memory for the log collector |
| `logPipeline.otlp.httpPort` / `grpcPort` / `agentJsonPort` | OTLP ports (defaults `4318` / `4317` / `4320`) |
| `logPipeline.retention.ttlDays` | Prune TTL in days; `""` disables retention CronJob |
| `logPipeline.retention.cleanupSchedule` | Retention CronJob schedule (default `0 3 * * *`) |
| `logPipeline.compaction.enabled` / `schedule` / `image` | Hourly compaction for every backend (default on, `15 * * * *`, `softprobe/duckdb:1.1.3`) |
| `logPipeline.placement` | `nodeSelector` / `tolerations` / `affinity` for Vector and maintenance CronJobs |
| `logPipeline.agentLogEndpointProperty` | Documented JVM property: `sp.otel.exporter.otlp.log.endpoint` |

`logPipeline.parquet.localRoot` exists in chart defaults (`/data/parquet/logs`) and must stay aligned with the rclone bucket layout — operators normally **do not** override it.

### Maintenance jobs

| CronJob | When | What |
|---------|------|------|
| `{release}-log-vector-retention` | `ttlDays` set (any backend) | AWS CLI deletes Parquet objects older than TTL over the S3 endpoint (native bucket, or the rclone gateway for local/Azure Blob) |
| `{release}-log-vector-compaction` | `compaction.enabled` (any backend) | DuckDB merges previous hour's `part-*.parquet` → `part-hourly.parquet` over the S3 endpoint, then AWS CLI prunes the minute objects |

Check last run:

```bash
kubectl get cronjob,jobs -n softprobe -l 'app.kubernetes.io/component=log-pipeline-maintenance'
kubectl logs -n softprobe job/<compaction-job-name>
```

### Out of scope (v1)

* Iceberg, ad hoc SQL, direct Parquet access for end users.
* Dual-write to local PVC and S3.
* Dedicated log-pipeline health/status API.

## Uninstall

```bash
helm uninstall softprobe -n softprobe
```

Bundled MongoDB PVCs are retained by default. Delete manually if required:

```bash
kubectl delete pvc -n softprobe -l app.kubernetes.io/instance=softprobe
```

## Java agents

Point instrumented applications at the in-cluster service:

```text
-Dsp.api.url=http://softprobe-sp-backend.softprobe.svc.cluster.local:8090
```

## Troubleshooting

| Symptom | Check |
|---------|--------|
| `helm install` fails on MongoDB | Set **one** of `mongodb.connectionString` or `mongodb.bundled.auth.password` |
| `sp-backend` pod `Init:0/1` (bundled) | MongoDB or Redis not ready — `kubectl get pods -n softprobe` |
| `sp-backend` slow start | JVM warm-up — up to ~2 minutes (startup probe) |
| `ImagePullBackOff` | Missing `softprobe-gcr-pull` secret or wrong `image.tag` |
| Mongo PVC pending (bundled) | No StorageClass — set `mongodb.bundled.storageClass` |
| External MongoDB connection errors | URI reachable from cluster; unique DB name; correct `authSource` |
| Empty `GET /api/recorder/logs` but data expected | Partial `part-hourly.parquet` from interrupted compaction — delete hourly file or wait for next compaction; confirm minute `part-*.parquet` files exist |
| Vector pod not ready | `kubectl logs -n softprobe deploy/<release>-log-vector -c vector` |
| Compaction `ImagePullBackOff` on arm64 | Override `logPipeline.compaction.image` with a local `arm64` build |
| Agent logs missing | `sp.api.url` must reach sp-backend; log pipeline enabled; trace must have `trace_id` on export |
| sp-backend logs missing | `logPipeline.enabled` auto-enables OTLP export on sp-backend |

## Next step

After sp-backend is healthy, install Softprobe on developer machines: [Install Softprobe Client](./).

For a shared team web workbench on Linux, see [Spcode Service](./index#spcode-service) on the client install page.

Related: [`sp logs`](/en/testing/commands/logs) · [Log correlation IDs](/en/testing/reference/log-correlation-ids)

---

---
url: /en/testing/installation/metrics-data-plane.md
---
# Metrics data plane

Softprobe metrics use the **same collector and Parquet store family as logs**: OTLP → Vector → one-minute aggregate → Parquet under a `metrics/` dataset → product HTTP query. This is for **ad-hoc diagnosis** (for example “are we receiving agent logs?”), not Prometheus/Grafana dashboards.

Requires chart **v4.3.x+** with the [unified log pipeline](./server#unified-log-pipeline) enabled (default). Metrics reuse that Vector Deployment — no second time-series database and no extra pods.

## Ingest — `POST /v1/metrics`

Clients POST OpenTelemetry metrics to Softprobe. sp-backend proxies to Vector when the telemetry pipeline is ready.

| Content-Type | Payload |
|--------------|---------|
| `application/x-protobuf` | OTLP `ExportMetricsServiceRequest` |
| `application/json` | OTLP metrics JSON (`resourceMetrics` …) |

**Accepted formats:** OTLP protobuf and OTLP JSON only. Softprobe does **not** accept a flat-JSON metrics dialect on this endpoint. OTLP JSON is accepted at Softprobe and forwarded to Vector as protobuf.

| Pipeline state | Response |
|----------------|----------|
| Ready | `200` after accept/proxy |
| Not ready | `503` — Softprobe does **not** return success while dropping data |

Example (OTLP JSON):

```bash
curl -sS -X POST "$SP_API_URL/v1/metrics" \
  -H 'Content-Type: application/json' \
  --data-binary @export-metrics.json
```

Instrumented backends also **emit** catalog counters in-process to the same Vector metrics path (no HTTP self-POST).

## Query — `GET /api/recorder/metrics`

Bounded, read-only lookups over Softprobe Parquet under `metrics/`.

**Required query parameters:**

| Parameter | Description |
|-----------|-------------|
| `metric_name` | Metric name to select |
| `since` | ISO-8601 UTC inclusive |
| `until` | ISO-8601 UTC exclusive — window is `[since, until)` |

Optional exact-match filters on promoted label columns only: `status`, `kind`, `source`, `result`, `content_type`. High-cardinality selectors (`trace_id`, full URL, exception message) and non-catalog keys such as `app_id` are rejected.

There is **no** product row `limit` and **no** max time-span beyond requiring valid bounds (same spirit as log query). Missing partitions return HTTP 200 with empty `rows`.

```bash
curl -sS "$SP_API_URL/api/recorder/metrics?metric_name=sp.logs.ingest.requests&since=2026-07-10T18:00:00Z&until=2026-07-10T18:05:00Z"
```

Example success shape:

```json
{
  "lookup": {
    "metric_name": "sp.logs.ingest.requests",
    "windows": [{ "since": "2026-07-10T18:00:00Z", "until": "2026-07-10T18:05:00Z" }],
    "filters": {}
  },
  "rows": [
    {
      "timestamp": "2026-07-10T18:01:00Z",
      "metric_name": "sp.logs.ingest.requests",
      "metric_type": "sum",
      "service_name": "sp-backend",
      "value": 3,
      "status": "ok",
      "kind": "agent_json"
    }
  ],
  "warnings": []
}
```

Error bodies must not expose Parquet paths, bucket names, or storage credentials.

## Backend P0 catalog (R1)

Softprobe emits these series from the log ingest/forward path:

| Name | Meaning |
|------|---------|
| `sp.logs.ingest.requests` | Every `POST /v1/logs` attempt (`status`, `kind`) |
| `sp.logs.ingest.records` | Records after convert (`kind`, `source`) |
| `sp.logs.forward.results` | Export outcome (`success` / `failure` / `skipped_*`) |
| `sp.logs.forward.duration_ms` | Export timing histogram |

After Vector’s one-minute aggregate, expect queryable rows within about a minute (CI polls up to **70 seconds**).

## Out of scope

* Prometheus scrape, Grafana, or PromQL as the Softprobe product path
* `sp metrics` CLI (HTTP API is the R1 contract)
* Agent-side `sp.agent.logs.*` emitters (later round)
* Direct Parquet or storage credentials for end users

---

---
url: /en/testing/installation.md
---

# Install Softprobe Client

::: tip Prerequisite
Deploy the Softprobe backend on your cluster first: [Install Softprobe Server](./server).
:::

Install Softprobe with the global installer:

```bash
curl -fsSL https://install.softprobe.ai/install.sh | bash
```

The default install adds the Softprobe CLI (`sp`), the Java agent, and the web UI used for testing.

After install, confirm `sp` is available:

```bash
sp -v
```

If that prints a version, you are ready for the steps below.

## Setup

`sp setup` configures the self-hosted or on-prem Softprobe API URL.

```bash
sp setup
```

If you omit `--api-url`, the wizard **prompts** for your API URL. On Linux it may also ask whether to install Spcode Service ([below](#spcode-service)).

| Flag | Description |
|------|-------------|
| `--api-url` | API URL (use this to skip the prompt) |

Model provider configuration belongs to `sp code`, not `sp setup`.

For non-interactive or scripted setup:

```bash
sp setup --api-url http://127.0.0.1:8090 --json
```

The URL is stored in Softprobe config at `~/.config/softprobe/config.jsonc`.

## Spcode Service (Linux only) {#spcode-service}

On **Linux**, `sp setup` can optionally install **Spcode Service** — a shared web workbench your team opens in the browser on the corp network. For a private UI on your own machine, see [Launch Softprobe Web UI Manually](./code) instead.

```bash
sp setup --api-url http://sp-backend.corp:8090 --install-spcode-service
sp setup --install-spcode-service --spcode-service-port 5000
sp setup --uninstall-spcode-service
```

| Flag | Description |
|------|-------------|
| `--install-spcode-service` | Install the shared web workbench (**Linux only**) |
| `--uninstall-spcode-service` | Stop and remove Spcode Service |
| `--spcode-service-port` | Listen port when installing (default `4096`) |

`--install-spcode-service` and `--uninstall-spcode-service` cannot be combined.

### Install

```bash
# Interactive: API URL → opt-in → port (default 4096)
sp setup

# Non-interactive
sp setup --api-url http://sp-backend.corp:8090 --install-spcode-service

# Automation without TTY
sudo sp setup --api-url http://sp-backend.corp:8090 --install-spcode-service
```

Install Softprobe and run `sp setup` as the account that should own the machine’s Softprobe settings. Spcode Service uses those same settings (backend URL, MCP, agent instructions, skills). After you change them, restart the service.

### Operations

| Task | Command |
|------|---------|
| Status | `systemctl status spcode-web.service` |
| Stop / start | `systemctl stop spcode-web.service` / `systemctl start spcode-web.service` |
| Logs | `journalctl -u spcode-web -n 50 --no-pager` |

If start fails, inspect `journalctl -u spcode-web` before retrying.

Optional MCP tools, agent instructions, and skills: [Client configuration](./configuration).

### Uninstall

```bash
sp setup --uninstall-spcode-service
```

Removes the systemd unit. Softprobe config files under your install account remain unless you delete them.

## Next

The CLI is ready — now bring your Java application in:

1. [Supported frameworks](/en/testing/supported-frameworks) — confirm your stack can be instrumented
2. [Attach the Java agent](/en/testing/java-agent) — attach the agent and register your app
3. [Launch the Web UI](./code) — view recordings and replays in the browser

Tools and advanced: [Doctor](./doctor) (troubleshooting) · [Upgrade](./upgrade) · [Client configuration](./configuration) (MCP, `AGENTS.md`, skills)

---

---
url: /en/testing/supported-frameworks.md
---

# Supported frameworks (Java)

Softprobe Testing instruments common Java stacks through the agent’s module plugins. The list below reflects currently documented support; new integrations are added continuously in [`sp-agent-java`](https://github.com/softprobe/softprobe/tree/main/backend/sp-agent-java).

::: tip
If your stack is not listed, check whether it uses a supported client (for example any JDBC layer backed by MyBatis or Hibernate). Entry points must be one of the HTTP/RPC provider modules the agent weaves.
:::

## Foundation

* Java Executors
* System time (`DynamicClass` — force-mocked on replay by default policy)
* User-configured dynamic types

## Cache

* Caffeine Cache
* Guava Cache
* Spring Cache

## Spring / HTTP entry

* Spring Boot 1.4+, 2.x+
* Servlet API 3+, 5+

## HTTP clients

* Apache HttpClient 4.0+
* OkHttp 3.0 – 4.11
* Spring WebClient 5.0+
* Spring `RestTemplate`
* Feign 9.0+
* Elasticsearch Client 7.x

## Redis

* `RedisTemplate`
* Jedis 2.10+, 4+
* Redisson 3.0+
* Lettuce 5.x, 6.x

## Persistence

* MyBatis 3.x, MyBatis-Plus, TkMyBatis
* Hibernate 5.x

## NoSQL

* MongoDB driver 3.x, 4.x

## RPC

* Apache Dubbo 2.x, 3.x
* Alibaba Dubbo 2.x
* SOFA RPC (provider/consumer categories in backend model)

## Auth

* Spring Security 5.x
* Apache Shiro 1.x
* jCasbin 1.x
* Auth0 JWT 3.x
* JJWT 0.1+, jjwt-api 0.10+

## Netty

* Netty server 3.x, 4.x

## Config

* Apollo Config 1.x, 2.x

## Backend dependency categories

Replay matching uses **dependency** categories (not entry types) in mock policy. Common names aligned with [`MockCategoryType`](https://github.com/softprobe/softprobe):

| Category | Typical use |
|----------|-------------|
| `HttpClient` | Outbound HTTP |
| `Database` | JDBC / ORM SQL |
| `Redis` | Redis commands |
| `DubboConsumer` / `SofaConsumer` | RPC client |
| `QMessageProducer` | Message publish |
| `DynamicClass` | Time, random, cache, encryption helpers |
| `ConfigFile` | Config reads (often skipped in compare) |

Entry categories (`Servlet`, `DubboProvider`, …) define what was recorded as the **main** API under test.

## Related

* [Java agent](/en/testing/java-agent)
* [Mock policy](/en/testing/policies#mock-policy)
* [How it works](/en/testing/how-it-works)

---

---
url: /en/testing/download-java-agent.md
---

# Download Java agent

Download the latest Java agent:

```bash
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
```

Pin a specific release by replacing `latest` with a version:

```bash
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/v4.3.9/sp-agent.jar
```

## Available versions

The mutable `latest` alias always points at the current Java agent. Immutable releases are listed below from `install.softprobe.ai`.

Scriptable version list:

```bash
curl -fsSL https://install.softprobe.ai/artifacts/agent/versions.json
```

## Next

Attach the agent and configure JVM flags: [Attach and configure](/en/testing/java-agent).

---

---
url: /en/testing/java-agent.md
---

# Softprobe Java agent

The Softprobe Java agent (`sp-agent.jar`) attaches to your JVM with `-javaagent`. It instruments frameworks at bytecode level (similar in *deployment* to an OpenTelemetry Java agent) but its purpose is **test data capture and replay-time mocking**, not generic distributed tracing.

::: warning Not the Istio/Envoy agent
Mesh capture is documented under [Platform agent architecture](/en/platform/advanced-guides/agent-architecture). This page covers the **JVM** agent only.
:::

## Prerequisites

* Java service you can restart with JVM flags
* **sp-backend** reachable from the agent host (default `http://127.0.0.1:8090` locally)
* Registered **`appId`** — create with `sp app create` and pin the same id on every instance

## Startup command

Attach the agent with `-javaagent` and the JVM properties below:

```bash
java \
  -javaagent:sp-agent.jar \
  -Dsp.app.id=<appId> \
  -Dsp.api.url=http://127.0.0.1:8090 \
  -jar your-service.jar
```

The agent may also resolve an app id automatically from jar name or environment; explicit `-Dsp.app.id` avoids mismatches between record and replay. Legacy docs and some configs still use **`sp.service.name`** — treat it as an alias in older deployments; prefer **`sp.app.id`** for new setups.

| Property | Points to | Purpose |
|----------|-----------|---------|
| `-Dsp.app.id` | — | Registered application id (16-char hex from `sp app create`). **Pin this** in every environment that shares recordings. |
| `-Dsp.api.url` | **sp-backend** (e.g. `:8090`) | **Required** — sp-backend base URL (must include `http://` or `https://`). Env fallback: `SP_API_URL`. Record, replay, mock, compare, **and correlated log export** (`{sp.api.url}/v1/logs`). |

When `sp.api.url` is set and the server [unified log pipeline](./installation/server.md#unified-log-pipeline) is enabled, logs are proxied to Vector internally — you do **not** need a separate Vector URL on the agent.

### Optional: direct Vector override

For advanced setups (bypassing the backend proxy), set:

```bash
-Dsp.otel.exporter.otlp.log.endpoint=http://<vector-host>:4320/v1/logs
```

This JVM property wins over `{sp.api.url}/v1/logs`.

Without `sp.api.url` (and without the override above), record and replay still work, but application logs are not exported and `sp logs` will be empty for that trace.

## Environment tags

Tag recorded traffic for filtering and replay scope:

```bash
-Dsp.mocker.tags=env=staging
```

Recorded mockers carry `env=<value>` so you can replay only traffic from a given environment. Match the same tag in a policy via `selector.envTags` — see [Policy YAML guide · Common fields](/en/testing/policy-yaml-guide#common-fields).

## Alternative deployment patterns

### `sp.agent.conf` file

```properties title="META-INF/sp/sp.agent.conf (baked into agent JAR)"
sp.api.url=http://127.0.0.1:8090
```

All-in-one and Helm installs bake this at packaging time so operators only need `-javaagent:sp-agent.jar` and `-Dsp.app.id`. Override with `SP_API_URL` or `-Dsp.api.url` when redirecting to another backend.

### Tomcat / `JAVA_OPTS`

Set agent flags in `catalina.sh` or `JAVA_TOOL_OPTIONS` so every worker JVM loads the agent on startup.

### Coexistence with OpenTelemetry

If another `-javaagent` conflicts (for example OpenTelemetry), add ignore prefixes:

```bash
-Dsp.ignore.type.prefixes=io.opentelemetry
-Dsp.ignore.classloader.prefixes=io.opentelemetry
```

Comma-separate multiple prefixes.

### Debug logging

```bash
-Dsp.enable.debug=true
```

## Agent status

`sp app status <appId>` reports **`online`**, **`offline`**, or **`never`** from instance heartbeats (default offline threshold ~60 seconds). Status reflects running agents, not merely app registration.

During recording, legacy UIs showed **WORKING** / **SLEEPING** / **UNSTART** per instance; the same idea applies: the agent must be injected and recording enabled to produce cases.

## What a complete case looks like

A healthy recorded case typically includes:

* **Servlet** (or other entry type) — main API request/response
* **Database**, **Redis**, **HttpClient**, … — dependency mockers in call order
* **DynamicClass** — optional, for configured cache/time/encryption methods

List cases after traffic: `sp record case list --app <appId> --json`.

## Production safety

To limit impact on live traffic, the agent implements **backpressure** when overloaded or when storage is unhealthy.

### Queue overflow

Recording tasks enter an in-memory queue (default capacity **1024**).
2\. If the queue is full, recording stops immediately.
3\. After ~30s, a health task lowers sampling (~20%) and retries.
4\. If still full after ~5 minutes, frequency drops again until a minimum (~once per hour).
5\. When the queue recovers (~10 minutes later), normal recording resumes.

### Storage health

1. If sp-storage calls fail or time out, recording stops immediately.
2. After ~10s, recording resumes while health is sampled.
3. If metrics stay unhealthy for ~3 minutes, frequency is reduced in steps like queue overflow until storage recovers.

Combined with [recording policy](/en/testing/policies) sampling and desensitization, this keeps production risk bounded.

## Replay-side agent

The **same** agent JAR must be attached on the instance that receives replay traffic. Set recording to minimal or zero on dedicated replay hosts so you only mock, not capture new production-like volume unintentionally.

## Next

Agent attached and `sp app status` shows online? Onboarding is done → head into the core workflow with **[Record traffic](/en/testing/recording)**.

Related: [Download Java agent](/en/testing/download-java-agent) · [Supported frameworks](/en/testing/supported-frameworks) · [Getting started](/en/testing/getting-started)

---

---
url: /en/testing/installation/code.md
---

# Launch Softprobe Web UI Manually

Most testing work happens in the **Softprobe web UI** in your browser — record, replay, and inspect traces there.

This page is for starting that UI **on your own dev machine** when you want a private session on your laptop or workstation. It is not the same as a **shared web workbench** that your team opens from one URL on the network; that optional server install is covered under [Spcode Service](./#spcode-service) on the install page.

## Start the UI on your dev box

```bash
sp code web --port 4096
```

Then open `http://127.0.0.1:4096` (or the host and port you chose). The UI uses **your** Softprobe config and backend URL from `sp setup`.

You can also open the terminal coding experience without the browser:

```bash
sp code
```

If Softprobe cannot start the web UI, run `sp doctor` and follow the remediation, or run `sp upgrade` if the install is out of date.

## Dev machine vs shared workbench

| | On your dev box (this page) | Shared web workbench (optional) |
|--|-------------------------------|----------------------------------|
| **Who** | You, on your machine | Your organization, one URL for the team |
| **Typical use** | Local testing while you develop | Colleagues in the same network open the same workbench |
| **How** | `sp code web` in your shell | `sp setup --install-spcode-service` during install ([Spcode Service](./#spcode-service)) |
| **Config** | Your Softprobe settings from `sp setup` | Same Softprobe settings as the account that installed Spcode Service |

Use the dev-box flow when you are the only person on that machine. Choose the shared workbench when several people need browser access without each running their own process.

## Next

With the UI opening, your onboarding toolkit is complete. Haven't run the full loop yet? Walk through [Getting started](/en/testing/getting-started); if you already have recorded cases, this UI is where you'll [Review diffs](/en/testing/review-diffs-in-the-web-ui) after a replay.

---

---
url: /en/testing/installation/doctor.md
---

# Doctor

Use `sp doctor` to check a self-hosted Softprobe installation:

```bash
sp doctor
sp doctor --json
```

The doctor surface reports backend reachability and internal coding engine installation with remediation for failures.

When **`spcode-web.service`** is registered (Spcode Service installed via [`sp setup --install-spcode-service`](/en/testing/installation/#spcode-service)), `sp doctor` also checks that the service is healthy and can reach the configured Softprobe backend.

---

---
url: /en/testing/installation/upgrade.md
---

# Upgrade

Upgrade Softprobe with:

```bash
sp upgrade
```

The command uses the global installer lifecycle so the CLI, Java agent, and internal coding engine update as one Softprobe install. Partial component failures are reported visibly.

If **Spcode Service** is installed (`spcode-web.service`), restart after upgrading `sp` or `spcode`:

```bash
sudo systemctl restart spcode-web.service
journalctl -u spcode-web -n 20 --no-pager
```

---

---
url: /en/testing/recording.md
---

# Record traffic

Recording means letting your agent-attached application **handle real requests as usual** — every request that flows through, together with the dependency calls it triggers (database, HTTP, Redis, …), is automatically stored as a **case**: the raw material for replay.

This page follows `order-service` as the running example: the agent is attached per [Attach the Java agent](/en/testing/java-agent), and the app is registered (you have its `appId`) as in the quick start. All four workflow steps use this same application.

::: tip Records out of the box — no policy needed first
The built-in global default policy (priority 0) makes recording work out of the box, and already excludes `/health` and similar probes. You only need to write an app-level policy when you want to tune sampling, time windows, or operation scope — see [Tune what gets recorded](#tune-what-gets-recorded) at the end of this page.
:::

## Step 1 · Confirm the agent is online

Start the application in the **environment whose traffic you want to capture** (usually production or staging):

```bash
java -javaagent:sp-agent.jar \
     -Dsp.app.id=<your appId> \
     -Dsp.api.url=http://<backend-host>:8090 \
     -jar order-service.jar
```

Confirm the agent has reported in:

```bash
sp app status <your appId> --json
```

In multi-environment deployments, tag instances (e.g. `-Dsp.mocker.tags=env=prod`) — you will use the same tags later to filter cases and match policies.

## Step 2 · Let real traffic flow

Nothing to do — user requests, business calls, and load-test traffic passing through the app are captured automatically. In environments without natural traffic (e.g. staging), send a few business requests to the APIs yourself.

::: tip Cases are recorded, never hand-written
The CLI cannot author cases manually. Want more cases? Send more traffic through the app.
:::

## Step 3 · Confirm cases exist

```bash
sp record case list --app <your appId> --since -1h --json
```

Cases showing up in the list means step 1 of the workflow is done — move on to [Replay](/en/testing/replay-and-diff).

To check completeness per trace, use `sp record completeness <traceId> --json`.

## No cases? Check this table

| Symptom | Check first |
|---------|-------------|
| Empty `sp record case list` | An app-level policy set `ratePerHundredSeconds` to 0; outside `timeWindow`; operation `exclude`d |
| Agent shows not recording | `machineCountLimit` too low; another instance holding the quota |
| Few cases | Sampling cap; narrow `include` whitelist |
| No upload at all | `appId` doesn't match the policy `selector`; `SP_API_URL` unreachable; agent offline |

## Keep record and replay environments separate

| Environment | Agent recording | Role |
|-------------|-----------------|------|
| Production / staging | On | Build the case corpus from real traffic |
| Test / CI replay host | Off or minimal | Avoid recording a second corpus during replay |

Use consistent `sp.mocker.tags` (e.g. `env=prod`) when filtering cases by source environment.

## Tune what gets recorded {#tune-what-gets-recorded}

When the defaults don't fit — you want to control the sampling rate, record only some APIs, or restrict recording hours — write an app-level `RecordingPolicy` (`priority > 0` overrides the global default):

```bash
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
```

Tunables: `ratePerHundredSeconds` (sampling), `timeWindow` (hours), `operations.include/exclude` (API scope), `serializeSkip`, `timeMock`. Field-by-field reference and full examples: [Policy YAML guide · RecordingPolicy](/en/testing/policy-yaml-guide#recordingpolicy).

::: warning Avoid `machineCountLimit: 1` in production
This caps how many instances may record concurrently in an env group. With `1`, the first instance can hold the slot after it goes away, leaving others in "not recording". Prefer omitting the field (unlimited) or setting it ≥ your instance count.
:::

::: info Two known boundaries

* `spec.sensitiveData` on the record path does **not** change what is stored yet; for masking at view time see [SensitivePolicy](/en/testing/policy-yaml-guide#related-configuration).
* Changing `operations` include/exclude also affects the **replay schedule's** operation scope.
  :::

## Next

Your cases are in the corpus → **[Replay & diff](/en/testing/replay-and-diff)**: turn them into a regression run.

---

---
url: /en/testing/replay-and-diff.md
---

# Replay and diff

Replay turns the cases you collected in [Record](/en/testing/recording) into a **regression run**: the original entry requests are sent to your **test instance** as-is, dependency calls (database, external HTTP, …) are automatically mocked from recorded data, and when the run finishes each case gets an automatic pass/fail from comparing recorded vs replayed responses.

Continuing the `order-service` example: production traffic has built a corpus of cases, and now you want to verify a new build in the test environment for regressions.

## Step 1 · Prepare the test instance

Run the **new build** in the test environment, same agent, same `appId`:

```bash
java -javaagent:sp-agent.jar \
     -Dsp.app.id=<your appId> \
     -Dsp.api.url=http://<backend-host>:8090 \
     -jar order-service-new.jar
```

Note its base URL, e.g. `http://order-service.test:8080` — this is **`targetEnv`**, the destination for replayed traffic.

::: warning Record in prod, replay in test
Replay sends **real HTTP requests** to `targetEnv` (only downstream dependencies are mocked), so the replay target should be a non-production instance unless you explicitly accept the risk. Also turn recording off (or near zero) on the replay host so the run doesn't capture a second corpus on top of the replay.
:::

## Step 2 · Start the replay

```bash
sp replay run --app <your appId> --env http://order-service.test:8080 --json
```

The command returns a `planId`. Watch it to completion:

```bash
sp replay status <planId> --watch
```

::: tip Don't mix up the two URLs
`--env` (`targetEnv`) is the address of the **service under test**; `SP_API_URL` is the address of the **sp-backend service**. Confusing them is the most common integration mistake — see [CLI concepts](/en/testing/agents/concepts#replay-target-url-targetenv).
:::

What happens during the run: the schedule service preloads the cases' mocks into Redis, then re-sends each recorded entry request to `targetEnv`; your service executes its real business code, but on every dependency call the agent returns the **recorded** response — no real database or external system is touched; replay-side traffic is stored and automatically compared against the recorded side.

sp-backend logs **`Replay send start`** before each dispatched entry call and **`Replay send done`** / **`Replay send failed`** after — the entry/exit boundary for replay HTTP dispatch. See [Replay send log markers](/en/testing/reference/replay-send-log-markers).

## Step 3 · Read the results

A case **passes** when compare finds no material differences. **Failed** cases show a diff scene:

* **Value diff** — the dependency was called, but the response body differs
* **Missing call** — a dependency called during record was not called during replay
* **Main response diff** — the entry response differs from the recording

Quick triage from the command line:

```bash
sp replay case list --plan <planId> --json     # which cases failed
sp diagnose replay <planId> --failed-only --out-dir .sp-work --json   # failure detail + diff artifacts on disk
```

Once you have a difference's `diffId`, inspect the full single diff: `sp replay diff get <diffId> --out-dir .sp-work --json`.

## Failures? Don't call them bugs yet

**Most failures are not bugs.** Timestamps, random IDs, pod IPs, and session tokens change on every run — they will always "differ" without anything being wrong. The last two workflow steps exist for exactly this:

* **[Review diffs](/en/testing/review-diffs-in-the-web-ui)** — read each diff in the workbench, accept the ones that aren't real bugs, and let true failures stand out
* **[Configure compare rules](/en/testing/compare-rules-web-ui)** — turn always-changing fields into rules so future replays stop false-alarming

Rules can also be declared in YAML (`sp policy compare`) for CI and GitOps — see [Policy YAML guide · CompareRulePolicy](/en/testing/policy-yaml-guide#comparerulepolicy).

## Terminology

| Concept | Meaning |
|---------|---------|
| `targetEnv` / `--env` | Base URL of the service receiving replayed entry traffic |
| `planId` | Container for the whole run |
| `planItemId` | One operation (API path) within the plan |
| `replayId` | One replay execution of a single case |
| Case | One recorded entry request + its dependency mockers |

## Replay scope

Which cases replay is determined by the plan request's time range and operation filters, plus the recording policy's `operations` include/exclude. To expand coverage, go back to [Record](/en/testing/recording) and record more traffic.

## Automation

Humans review diffs in the workbench; CI and AI agents should use `sp diagnose replay <planId> --json` and the `--out-dir` artifacts from the [output contract](/en/testing/agents/output-contract). For deploy-triggered replays and pipeline gates, see [Webhook and CI/CD](/en/testing/webhook-and-ci).

## Next

The run finished with failing cases → **[Review diffs](/en/testing/review-diffs-in-the-web-ui)**: understand them and clear the noise.

---

---
url: /en/testing/review-diffs-in-the-web-ui.md
---

# Review replay diffs

After a replay run from [Replay](/en/testing/replay-and-diff), the workbench shows which cases failed and why. This page covers the human review workflow in the **Web console** — reading a diff, and accepting the differences that aren't real bugs. Prefer the command line? `sp replay case list --plan <planId>` and `sp diagnose replay <planId>` yield the same failure data (see [Replay and diff](/en/testing/replay-and-diff)).

Most differences aren't bugs. Timestamps, random tokens, and IDs change on every call — they'll always "differ" without anything being wrong. Your job is to accept those, so the real failures stand out.

## Accept a difference: one-off, or permanent

There are two ways to make a difference stop counting, and picking the right one is the whole game:

* **Mark it passed on this case** — *"In this run, this difference is fine."* Applies now, only to this run. Use it after you've reviewed a case and decided it isn't a real bug.
* **Write a compare rule** — *"From now on, never compare this field."* Permanent, applies to every future replay. Use it for fields that are different by nature (timestamps, random IDs) that should never be compared.

One line to choose: **just changing this run's result → mark it passed; want it to stick forever → write a rule.**

| | Mark passed | Compare rule |
|---|---|---|
| Scope | Just this run | Every replay, from now on |
| Lifespan | One-off — gone on the next replay | Permanent — saved in your policy |
| Use it for | A case you've reviewed and accepted | A field that always changes |

::: tip A rule only applies on the next replay
A compare rule doesn't touch a run that already finished. To apply a new rule to the run you're looking at without replaying, use [Recompare](#recompare-apply-rules-to-this-run).
:::

## Open a case and read its differences

1. Open a **replay run** and pick a **failed case** — the left panel lists them with red dots.
2. **Click a span** in the call tree on the right.

The diff opens: the recorded response on the left, the replayed response on the right. Fields that differ are highlighted; the header shows how many differences the case has.

![The diff drawer open next to the call tree](/img/docs/replay/trace-diff-open.png)

## Ignore a field

**This is the one you'll reach for most.** A field like a timestamp differs on every replay and you never want to compare it.

1. In the diff, **hover the line** that differs. An **eye-off icon** appears at the left edge — click it. (Or right-click the line, or use the **Ignore rules** button at the top.)
2. Choose **Ignore this field's differences**.
3. Pick **This case only**.

The field is struck through, the difference is gone, and the case's pass rate updates.

![The ignore menu](/img/docs/replay/ignore-menu.png)

### Ignore it more widely

**This case only** fixes just the case in front of you. But a timestamp differs on *every* case, so ignoring it one case at a time is tedious. To stop comparing a field more widely, pick a bigger scope in the same menu:

| Pick | Ignores the field for | Takes effect |
|---|---|---|
| **This case only** | Just this one case | Now |
| **This endpoint only** | Every case of this endpoint | Next replay |
| **All endpoints** | The whole application | Next replay |

The two wider scopes write a **compare rule**, which takes effect on the *next* replay — so the run you're looking at doesn't change yet. Softprobe shows a **Recompare** button to apply it now; see [Recompare](#recompare-apply-rules-to-this-run).

::: tip Ignore every field with the same name
The menu also offers **Ignore all "…" fields** — handy when the same field name (say `updatedAt`) appears in several places and you want them all gone at once.
:::

## Mark a case passed

Sometimes a difference is real but you're OK with it — an intentional change, or a Softprobe artifact. Instead of ignoring fields one by one, accept the **whole case**.

1. On the failed case, click **Mark as passed**.
2. Pick a reason: **Difference is expected (by design)**, or **Softprobe collection/comparison issue**.
3. Optionally add a note, then confirm.

The case moves to passed and the pass rate updates. Your reason and note are kept for later review.

![The mark-as-passed form](/img/docs/replay/mark-passed.png)

## Recompare: apply rules to this run {#recompare-apply-rules-to-this-run}

A rule you just wrote only kicks in on the **next** replay — it doesn't change a run that already finished. **Recompare** applies your latest rules to the run you're looking at now, re-checking its stored responses. It does **not** replay any traffic.

When you add or remove a rule from the diff, a **Recompare** button appears in the run header. Click it. Softprobe re-checks the run and updates the case list, counts, and pass rate together.

![Recompare done, showing how many cases changed](/img/docs/replay/recompare-done.png)

::: warning Recompare is not replay
Replay re-sends traffic to your service. Recompare only re-judges responses that are already stored, against your current rules — nothing hits your service.
:::

## Un-ignore a field

Changed your mind? **Right-click the struck-through line** and choose **Un-ignore (restore comparison)**. The field goes back to being compared.

You don't need to remember how you ignored it — Softprobe removes whatever is hiding the difference (the case mark, or the rule) and tells you what it removed. If the rule lives in your global defaults, it points you to the compare-rules page instead of leaving you stuck.

## See what's already ignored

To see everything ignored on a case without opening each span, click the **"N ignored"** chip at the top of the case. A panel lists every ignored field, grouped by call, with the rule that caught it. Hover any row to un-ignore it right there.

![The ignored-summary panel](/img/docs/replay/ignored-summary.png)

## Next

Ignoring the same field on every single case? Time to turn it into a rule → **[Configure compare rules](/en/testing/compare-rules-web-ui)**: configure once, and every future replay stops false-alarming.

---

---
url: /en/testing/compare-rules-web-ui.md
---

# Configure compare rules

Compare rules decide which replay differences **don't count**. The principle: *what you configure is skipped; everything else is compared strictly.* Accepting diffs one by one in [Review diffs](/en/testing/review-diffs-in-the-web-ui) only covers one run; turning always-changing fields (timestamps, random IDs) into rules stops the false alarms on every future replay — this step closes the loop.

This page covers setting rules in the **Web console's visual editor** — pick a rule type, fill in a field or two, click add. The visual editor is the friendlier path for day-to-day use; the exact same rules can be written declaratively in the [Policy YAML guide](/en/testing/policy-yaml-guide) for GitOps and CI.

## Where to configure rules

There are two places, for two scopes:

| | Global default rules | Application rules |
|---|---|---|
| Open from | Settings → Compare Rules | Workbench → Configuration → Compare Rules |
| Applies to | Every application | The current application only |
| Rule types | All types, in six tabs | The three common types plus per-endpoint overlays; the rest via YAML |

An application panel links to the global defaults at the top, so you always see what else is in effect.

![The application compare-rules panel](/img/docs/replay/compare-rules-panel.png)

The global page groups the rule types into six tabs — one per type below.

![The six rule-type tabs](/img/docs/replay/rules-global-tabs.png)

## Ignore a field by path

**The most common rule — stop comparing a field.** Ideal for volatile fields: timestamps, trace IDs, random tokens. The field, and everything under it, is removed from the comparison.

**How to add it:** open the **Ignore by path** tab, type the field path in **Ignore fields**, and click **Add**.

![The path tab](/img/docs/replay/rule-exclude-path.png)

**What to type:** a field path. Both `data.traceId` (dot form) and `/data/traceId` (JSON Pointer) work. Use `*` for one level and `**` for any depth, e.g. `/data/*/updatedAt`.

::: tip The whitelist input (rarely needed)
The same tab has an **Include paths (whitelist)** input at the top. Add anything here and **only** those paths are compared, everything else ignored — the opposite of the ignore list. Leave it empty (the normal case) to compare everything.
:::

## Ignore an entire dependency type

**A coarse switch — drop all differences from one kind of downstream call.** For example, ignore every Redis difference, or every difference from one database query.

**How to add it:** open the **Dependency types** tab. Enter the **type** (e.g. `Redis`, `Database`, `Dubbo`, `HttpClient`) and, optionally, a specific dependency **name**. Leave the name empty to ignore the whole type.

![The dependency types tab](/img/docs/replay/rule-ignore-category.png)

In a run, a call ignored this way shows an **"Entire category ignored"** chip instead of per-field strikethroughs — the whole call is dropped at once.

## Ignore by a condition (CEL)

**The most flexible rule — ignore a difference when a condition is true.** Use it when matching by path or field name isn't enough, e.g. "ignore any field whose recorded and replayed values are both timestamps." After the comparison runs, each difference is tested against your condition; if it matches, the difference is dropped.

**How to add it:** open the **Ignore by condition (CEL)** tab, click **Add rule**, optionally name it, and write the condition. A template picker offers ready-made conditions.

![The CEL rules tab with example rules](/img/docs/replay/rule-cel.png)

Variables you can use: `left` / `right` (recorded / replayed value), `path` / `pointer` (field path), `fieldName`, `category`, `time_tolerance_ms`. Helpers: `isUUID`, `isIP`, `isTimestamp`, `toTimestamp`, `toNumber`.

Examples:

* Both values are timestamps: `isTimestamp(left) && isTimestamp(right)`
* A generated ID: `fieldName == "requestId" && isUUID(right)`

## Normalize a value before comparing

**Round or reshape a value so noise doesn't register** — e.g. round a float so precision differences don't count.

**How to add it:** open the **Value transform** tab, enter the field **path** and a CEL **expression** on the value (`value` is the field's original value), then click **Add transform**.

![The value transform tab](/img/docs/replay/rule-transform.png)

Example — round to two decimals: path `/data/orders/*/total`, expression `math.round(value * 100) / 100`.

## Decode an encoded field before comparing

**Decode base64/gzip JSON so the comparison sees real data**, instead of reporting "these two encoded strings differ."

**How to add it:** open the **Decompress** tab, enter the field **path**, and pick the **codec** (`Base64 + JSON`, `Gzip + Base64 + JSON`, or `Plain JSON`).

![The decompress tab](/img/docs/replay/rule-decompress.png)

## Match array elements when order varies

**Compare an unordered array as a set, not by position.** By default arrays are compared by index — when element order changes between record and replay, that produces false "missing / new element" differences.

**How to add it:** open the **Array matching** tab, enter the array **path**, pick a **strategy**, and (for `By key`) enter the **key field(s)**. Click **Add array config**.

![The array matching tab](/img/docs/replay/rule-arrays.png)

| Strategy | When to use |
|---|---|
| By index | Default — compare position by position. |
| By key | Pair elements by a key field (enter the key, e.g. `orderId`). |
| By LCS | Longest common subsequence — best-effort alignment without a key. |

## Apply rules to specific endpoints only

The rules above apply to every endpoint. To add rules for just some endpoints, use the **Per-endpoint** section in an application panel. Click **Add endpoint rules**, enter the endpoints to match — exact names and/or glob patterns like `/api/order/*` — then fill in that group's own rule table.

## Where quick rules from the diff go

When you [ignore a field in a run](/en/testing/review-diffs-in-the-web-ui#ignore-a-field), it writes into one of these rule types:

| Diff action | Becomes |
|---|---|
| Ignore this field's differences | An ignore-by-path rule |
| Ignore all "…" fields | A CEL rule matching that field name |
| Set as array key | An array rule (By key) |

Scoped to an endpoint, it lands in a per-endpoint group; scoped to the whole app, at the top level.

## Next

With rules in place, go back to [Replay](/en/testing/replay-and-diff) and run again (or use **Recompare** in [Review diffs](/en/testing/review-diffs-in-the-web-ui) to verify immediately) — whatever remains in the failure list is now worth looking at.

To automate the whole workflow — deploy-triggered replays and pipeline gates — see [Webhook and CI/CD](/en/testing/webhook-and-ci); for rules in GitOps, see the [Policy YAML guide](/en/testing/policy-yaml-guide).

---

---
url: /en/testing/policies.md
---

# Policies overview

Softprobe Testing uses **declarative YAML policies** (`apiVersion: softprobe.ai/v1`), merged by `metadata.priority` and applied by sp-backend at runtime.

::: tip Policies are optional tuning, not a prerequisite
Built-in global defaults (priority 0) make recording and replay work out of the box. You only write your own policy when you want to **change** the default behavior — control sampling, narrow operation scope, ignore noisy fields (`priority > 0` overrides). Get the core workflow running first, then come back to tighten.
:::

Separate **how to run each phase** from **policy configuration**:

| Phase | Operations | Policy config |
|-------|------------|---------------|
| **1 · Record** | [Record traffic](/en/testing/recording) | [RecordingPolicy](#recording-policy) below |
| **2 · Replay** | [Replay and diff](/en/testing/replay-and-diff) | [MockPolicy](#mock-policy), [CompareRulePolicy](#compare-policy) below |

Field reference and full examples: [Policy YAML guide](/en/testing/policy-yaml-guide) · [sp policy command](/en/testing/commands/policy)

## CLI quick reference

```bash
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
sp policy mock apply -f mock.yaml --json
sp policy compare apply -f compare.yaml --json
```

Higher **`metadata.priority`** wins on conflicts. Built-in priority-0 globals exist; use `priority > 0` on app policies.

## Lifecycle phases

| Phase | Kind | When to configure | CLI |
|-------|------|-------------------|-----|
| **1 · Record** | `RecordingPolicy` | Before traffic | `sp policy recording` |
| **2 · Replay** | `MockPolicy` | Before `sp replay run` | `sp policy mock` |
| **2 · Replay** | `CompareRulePolicy` | Before `sp replay run` | `sp policy compare` |

Full lifecycle: [Getting started](/en/testing/getting-started)

***

## RecordingPolicy {#recording-policy}

**For the [Record](/en/testing/recording) stage · apply before traffic**

Controls **what the agent records**: sampling, time window, operation include/exclude, serialize skip, record-time time mock.

* **Sampling** — `ratePerHundredSeconds` (`0` = no record), optional `machineCountLimit` (omit = unlimited)
* **Time window** — `daysOfWeek`, `from` / `to` (agent JVM local timezone)
* **Operations** — `exclude` globs; non-empty `include` switches to whitelist mode
* **Serialize skip** — `serializeSkip` by class and field names
* **`timeMock`** — fix `java.time.*` at record time

**Operational steps:** [Record traffic](/en/testing/recording)

**YAML fields and examples:** [Policy YAML guide · RecordingPolicy](/en/testing/policy-yaml-guide#recordingpolicy)

::: info Notes

* `spec.sensitiveData` is **not** applied on the agent record path yet; use `matchTolerance` for mock-key noise and `SensitivePolicy` for view-time masking (see YAML guide).
* Changing `operations` include/exclude updates **replay schedule** operation scope without a separate schedule edit.
  :::

```bash
sp policy recording validate -f my-recording.yaml --json
```

***

## MockPolicy {#mock-policy}

**For the [Replay](/en/testing/replay-and-diff) stage · apply before `sp replay run`**

Controls **whether dependencies are mocked** at replay, mock-key tolerance, cross-app dependencies, and fallback when no mock matches.

* **`mockByDefault`** — mock all deps by default (`skipMock` exceptions), or the inverse with `forceMock`
* **`Category:operationGlob`** — e.g. `HttpClient:/payment/**` (**not** entry types like `Servlet`)
* **`matchTolerance`** — ignore volatile headers, query params, body paths
* **`multiServiceDependencies`** — mock downstream apps from the same session
* **`fallback`** — `FAIL` (default), `PASS_THROUGH`, `RETURN_DEFAULT`

Global defaults **force-mock** `DynamicClass:SystemTime.**` and `RandomSource.**`; user `skipMock` has no effect.

**YAML fields and examples:** [Policy YAML guide · MockPolicy](/en/testing/policy-yaml-guide#mockpolicy)

[Full dependency category list](/en/testing/policy-yaml-guide#dependency-categories)

***

## CompareRulePolicy {#compare-policy}

**For the [Replay](/en/testing/replay-and-diff) stage · apply before `sp replay run`**

Controls **diff noise** during replay comparison (not mock behavior).

* **`excludePaths` / `includePaths`** — JSON Pointer (with globs)
* **`defaults.timeToleranceMs`** and CEL **`validations`** — drop diffs by rule (including per-`category` rules)
* **`operationSpecs`** — per-entry-operation overlays (**do not** put operation names on `selector`)

**YAML fields and examples:** [Policy YAML guide · CompareRulePolicy](/en/testing/policy-yaml-guide#comparerulepolicy)

Use `sp replay diff` after replay, then tighten policy rather than changing application code.

***

## Dynamic classes (not RecordingPolicy)

Register methods in **dynamic class configuration** (dashboard/API), not in `RecordingPolicy`. Control replay mocking via **MockPolicy** `UserDynamic` / `DynamicClass` rules. See [Policy YAML guide · Related configuration](/en/testing/policy-yaml-guide#related-configuration).

## Policy kinds and server modules

| Kind | CLI | Server module |
|------|-----|---------------|
| `RecordingPolicy` | `sp policy recording` | `RecordingPolicyService` |
| `MockPolicy` | `sp policy mock` | `MockPolicyService` |
| `CompareRulePolicy` | `sp policy compare` | `CompareRulesService` |

Example files ship in `sp-policy-rules/src/main/resources/examples/`.

## Agent workflow

```bash
# Always validate before apply
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
```

CI should fail on `valid: false` or a non-zero exit.

## GitOps

```bash
sp policy recording export prod-policy-id -o policies/recording-prod.yaml
git commit -m "chore: sync recording policy"
```

See [GitOps policies](/en/testing/examples/gitops-policies).

## Relationship to legacy config

`sp config legacy schedule` and Mongo `ServiceCollectConfiguration` are **not** the source of truth after the policy-rules migration. Use `sp policy recording` for operation include/exclude that affects both agent and replay scope.

## Related

* [Record traffic](/en/testing/recording)
* [Replay and diff](/en/testing/replay-and-diff)
* [Policy YAML guide](/en/testing/policy-yaml-guide)
* [CLI: policy command](/en/testing/commands/policy)

---

---
url: /en/testing/webhook-and-ci.md
---

# Webhook-triggered replay and CI/CD integration

Run record-and-replay regression automatically after a deploy or merge against a **test** instance, then gate the pipeline on compare results. This page covers two trigger styles, how to collect results, and GitHub Actions / Jenkins patterns.

::: tip Record first, then automate replay
Webhooks and CI do not create cases for you. Walk the core workflow first — [Record](/en/testing/recording) and [Replay](/en/testing/replay-and-diff) (`appId`, policies, reachable `targetEnv`) — then hand it to automation.
:::

## Should you use the `sp` CLI or REST APIs?

| Scenario | Recommendation |
|----------|----------------|
| Full pipelines (GitHub Actions, Jenkins, GitLab CI) | **`sp` CLI** (`--json`, stable exit codes, built-in polling) |
| Deploy hooks (Argo CD, K8s Job, shell) that only need “deploy OK → start replay” | **GET webhook** (one-line `curl`) |
| Custom orchestration with your own HTTP client and request bodies | **REST** (`POST /api/createPlan` + report APIs) |

**Rule of thumb:** When the pipeline must **create a plan → wait → list failures → pass/fail**, use **`sp replay run --watch`** and **`sp replay case list --failed`** per the [output contract](/en/testing/agents/output-contract). Use GET webhook to **trigger only**; fetch results in a later step with `sp` or authenticated report APIs.

See [replay command](/en/testing/commands/replay) and [authentication](/en/testing/agents/authentication).

## Architecture (brief)

```mermaid
sequenceDiagram
  participant CI as CI/CD or webhook
  participant API as sp-backend schedule + report
  participant SUT as Service under test targetEnv
  CI->>API: createPlan
  API->>SUT: replay recorded entry HTTP
  SUT-->>API: replay traffic and compare results
  CI->>API: poll progress / query failed cases
```

* **`SP_API_URL`**: sp-backend (storage, schedule, report) — **not** the service under test.
* **`targetEnv`**: base URL of the test instance (e.g. `http://order-service.test:8080`). Do not confuse with `SP_API_URL` — [CLI concepts: targetEnv](/en/testing/agents/concepts#replay-target-url-targetenv).

## Option 1: Webhook (GET) trigger

The schedule service exposes **GET** `/api/createPlan` for deploy webhooks, alerts, or simple `curl` calls. The server sets `operator` to `Webhook` and, by default, selects cases from the last **24 hours** for the app (narrow with time query params).

### Example request

```bash
curl -sS -G "${SP_API_URL}/api/createPlan" \
  -H "access-token: ${SP_TOKEN}" \
  --data-urlencode "appId=YOUR_APP_ID" \
  --data-urlencode "targetEnv=http://your-service.test:8080" \
  --data-urlencode "planName=post-deploy-smoke" \
  --data-urlencode "caseCountLimit=50" \
  --data-urlencode "enableMock=true"
```

### Query parameters

| Parameter | Required | Description |
|-----------|----------|-------------|
| `appId` | yes | Registered application id |
| `targetEnv` | yes | Test instance base URL (`http://` or `https://`) |
| `caseSourceFrom` | no | Case window start (Unix **milliseconds**); default ~24h ago |
| `caseSourceTo` | no | Case window end (ms); default now |
| `caseCountLimit` | no | Max cases to replay |
| `planName` | no | Display name for the plan |
| `operationIds` | no | Repeatable; restrict to operation ids |
| `enableMock` | no | `true` / `false` |
| `caseTags` | no | JSON string for tag filters |

### Response

On success, `result` is `1` and `data` contains **`replayPlanId`** (use as `planId` when polling):

```json
{
  "result": 1,
  "desc": "success",
  "data": {
    "replayPlanId": "plan-abc123"
  }
}
```

::: warning
GET webhook **only creates** a plan; it does not wait for completion. Poll progress or query the report in a follow-up CI step.
:::

::: info POST vs GET
`POST /api/createPlan` (used by `sp replay run`) accepts the full `BuildReplayPlanRequest` (time window, `operationCaseInfoList`, pressure settings, etc.). For complex case selection, use POST or the CLI instead of stretching GET params.
:::

## Option 2: CLI / REST create (recommended for CI)

Same as [Replay and diff](/en/testing/replay-and-diff):

```bash
export SP_API_URL=https://your-tenant.softprobe.ai
export SP_TOKEN="${SP_TOKEN}"

sp replay run \
  --app YOUR_APP_ID \
  --env http://your-service.test:8080 \
  --from -24h \
  --name "ci-${GITHUB_SHA:-build}" \
  --watch \
  --json
```

`--watch` polls `GET /api/progress?planId=…` until finished. The CLI may still exit **0** when compare failures exist — pipelines must check failed cases (below).

REST equivalent:

```http
POST /api/createPlan
Content-Type: application/json
access-token: <JWT>

{
  "appId": "YOUR_APP_ID",
  "targetEnv": "http://your-service.test:8080",
  "caseSourceFrom": 1717000000000,
  "caseSourceTo": 1717086400000,
  "enableMock": true,
  "planName": "ci-build-42"
}
```

## Getting results

### 1. Poll progress

```bash
sp replay status plan-abc123 --watch --json
```

REST: `GET /api/progress?planId=plan-abc123` on the same `SP_API_URL`. When `percent` reaches 100, scheduling is done.

### 2. Pass / fail gate

After progress finishes:

```bash
sp replay case list --plan plan-abc123 --failed --limit 1 --json
```

Fail the job if `data.items` is non-empty. For diff artifacts:

```bash
sp replay diff get <diffId> --out-dir .sp-work --json
sp diagnose replay plan-abc123 --failed-only --out-dir .sp-work --json
```

See [Example: diagnose replay failure](/en/testing/examples/agent-diagnose-replay).

### 3. Dashboard (humans)

Open the same `planId` in the workbench or dashboard for diff trees; automation should rely on CLI/`--json`.

## GitHub Actions example

Run after the **test** deployment with the agent attached. Secrets: `SP_API_URL`, `SP_TOKEN`.

```yaml
name: Softprobe replay gate

on:
  workflow_dispatch:
  push:
    branches: [main]

jobs:
  replay-regression:
    runs-on: ubuntu-latest
    env:
      SP_API_URL: ${{ secrets.SP_API_URL }}
      SP_TOKEN: ${{ secrets.SP_TOKEN }}
      SP_APP_ID: ${{ vars.SP_APP_ID }}
      SP_TARGET_ENV: http://my-service.test:8080
    steps:
      - name: Install sp
        run: |
          curl -fsSL -o sp "${SP_DOWNLOAD_URL:-https://install.softprobe.ai/artifacts/sp/latest}/sp-linux-amd64"
          chmod +x sp && sudo mv sp /usr/local/bin/

      - name: Preflight
        run: |
          sp health --json
          sp record case list --app "$SP_APP_ID" --since -24h --json

      - name: Run replay and wait
        id: replay
        run: |
          sp replay run \
            --app "$SP_APP_ID" \
            --env "$SP_TARGET_ENV" \
            --from -24h \
            --name "gha-${GITHUB_SHA}" \
            --watch \
            --json | tee replay.ndjson
          PLAN_ID=$(grep '"command":"replay run"' replay.ndjson | tail -1 | jq -r '.data.planId // .data.replayPlanId')
          echo "plan_id=$PLAN_ID" >> "$GITHUB_OUTPUT"

      - name: Fail if any case failed
        run: |
          COUNT=$(sp replay case list --plan "${{ steps.replay.outputs.plan_id }}" --failed --json \
            | jq '[.data.items[]?] | length')
          if [ "$COUNT" -gt 0 ]; then
            echo "Replay had $COUNT failed case(s)"
            exit 1
          fi
```

Optional fire-and-forget webhook after deploy:

```yaml
      - name: Trigger replay webhook
        run: |
          curl -fsS -G "$SP_API_URL/api/createPlan" \
            -H "access-token: $SP_TOKEN" \
            --data-urlencode "appId=$SP_APP_ID" \
            --data-urlencode "targetEnv=$SP_TARGET_ENV" \
            --data-urlencode "planName=deploy-${{ github.run_id }}"
```

Policy validation on PRs: [CI policy gate example](/en/testing/examples/ci-policy-gate).

## Jenkins example

Bind `SP_TOKEN` as credentials:

```groovy
pipeline {
  agent any
  environment {
    SP_API_URL = credentials('sp-api-url')
    SP_TOKEN   = credentials('sp-token')
    SP_APP_ID  = 'YOUR_APP_ID'
    SP_TARGET_ENV = 'http://my-service.test:8080'
  }
  stages {
    stage('Replay') {
      steps {
        sh '''
          sp replay run \
            --app "$SP_APP_ID" \
            --env "$SP_TARGET_ENV" \
            --from -24h \
            --name "jenkins-${BUILD_NUMBER}" \
            --watch \
            --json > replay.ndjson
          PLAN_ID=$(grep '"command":"replay run"' replay.ndjson | tail -1 | jq -r '.data.planId // .data.replayPlanId')
          FAILED=$(sp replay case list --plan "$PLAN_ID" --failed --json | jq '[.data.items[]?] | length')
          test "$FAILED" -eq 0
        '''
      }
    }
  }
}
```

Use `curl -G …/api/createPlan` in a post-deploy step; wait and gate in a later stage with `sp replay status <planId> --watch`.

## Security and operations

* Store **`SP_TOKEN` in CI secrets** only — [authentication](/en/testing/agents/authentication).
* If webhook URLs are reachable from the internet, restrict access and always send **`access-token`**.
* Replay sends **real HTTP** to `targetEnv`; point webhooks at test/staging instances — [Replay and diff](/en/testing/replay-and-diff).
* Lower or disable recording on replay hosts during runs.

## Related

* [Replay and diff](/en/testing/replay-and-diff)
* [Getting started](/en/testing/getting-started)
* [CLI: replay](/en/testing/commands/replay)
* [API mapping](/en/testing/reference/api-mapping)
* [CLI quickstart](/en/testing/getting-started)

---

---
url: /en/testing/agents/overview.md
---
# For AI agents

This page is the primary entry point for authors of **OpenCode / spcode**, **Claude Code**, **Codex**, **Cursor**, and other agent hosts that invoke SoftProbe through shell tools.

## Feed these docs to your agent

The whole documentation set is published in [llmstxt.org](https://llmstxt.org/) format so an agent can ingest it without scraping HTML:

| URL | Contents |
|-----|----------|
| [`/llms.txt`](/llms.txt) | Curated index of every page with descriptions — a small entry point |
| [`/llms-full.txt`](/llms-full.txt) | The entire docs corpus in one plain-text file |
| `‹any page›.md` | The Markdown source of a single page (append `.md` to its URL) |

Point your agent at `/llms.txt` first; it will follow links into the full text or per-page `.md` as needed.

## Design goals

1. **One process, one job** — each tool call runs a single `sp` command with explicit flags. Do not rely on shell aliases or interactive prompts.
2. **Always pass `--json`** for machine parsing unless you are showing output to a human in chat.
3. **Use artifacts for large payloads** — diff bodies, log downloads, and record queries write files under `--out-dir`; stdout returns paths and summaries only.
4. **Backend data only** — use API commands (`sp app`, `sp replay`, `sp record`, …) for cluster state.

## Recommended tool shape

Wrap `sp` as a shell tool with a fixed argv prefix:

```bash
sp --json --profile "${SP_PROFILE:-default}" <subcommand> ...
```

Environment variables agents should set in the host config:

| Variable | Purpose |
|----------|---------|
| `SP_API_URL` | Backend base URL (e.g. `http://127.0.0.1:8090`) |
| `SP_TOKEN` | JWT from `sp auth login` or CI secret |
| `SP_PROFILE` | Named profile in `${XDG_CONFIG_HOME}/softprobe/config.jsonc` or `sp.jsonc` |
| `SP_CONFIG` | Extra config file path loaded after `sp.jsonc` and before `--config` |

`sp` also reads `${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc` for
shared SoftProbe settings and `${XDG_CONFIG_HOME:-~/.config}/softprobe/sp.jsonc`
for CLI-specific overrides. Agents should prefer `SP_API_URL` and `SP_TOKEN`
when running in ephemeral CI containers.

## Lifecycle command order

Use these job-oriented commands before low-level building blocks:

1. `sp doctor --json`
2. `sp app create <name> --json` → save `data.appId`
3. `sp policy recording apply -f … --json`
4. Download `sp-agent.jar` from `install.softprobe.ai`, then run `sp agent command --app <id> --agent-jar ./sp-agent.jar --json`
5. Start the app with `data.startCommand`; send traffic
6. `sp record case list --app <id> --since -1h --json`
7. `sp policy mock apply` / `sp policy compare apply`
8. `sp replay run --app <id> --env <service-base-url> --json`
9. `sp replay status <planId> --json` or `sp diagnose replay <planId> --json`

## Typical skill flows

### Diagnose a failed replay

```mermaid
sequenceDiagram
  participant Agent
  participant SP as sp CLI
  participant API as sp-backend

  Agent->>SP: app list --json
  SP->>API: GET /api/applications/list
  Agent->>SP: replay case list --plan X --failed --json
  SP->>API: report/storage APIs
  Agent->>SP: replay diff get diffId --out-dir .sp-work --json
  SP->>API: GET /api/report/queryDiffMsgById/{id}
  Agent->>Agent: Read artifact file locally
```

Commands: see [Diagnose replay failure](/en/testing/examples/agent-diagnose-replay).

### Change policy and re-run

1. `sp policy recording validate -f policy.yaml --json`
2. `sp policy recording apply -f policy.yaml --json`
3. `sp replay run --app … --env <service-base-url> --json`
4. `sp replay status <planId> --watch --json`

## Migrating from `sp_api` (spcode plugin)

The OpenCode `@spcode/plugin` tool `sp_api` uses named endpoints (`diff_detail`, `query_replay_case`, …). The `sp` CLI exposes the same operations as **stable subcommands**.

| Old | New |
|-----|-----|
| `sp_api endpoint=list_applications` | `sp app list --json` |
| `sp_api endpoint=diff_detail diffId=…` | `sp replay diff get … --out-dir … --json` |
| `sp_api endpoint=find_traces_by_attr …` | `sp trace find … --json` |

Skills should be updated to call `sp` so behavior is consistent outside OpenCode.

## Error handling for agents

| Exit code | Meaning | Agent action |
|-----------|---------|--------------|
| `0` | Success | Parse stdout JSON |
| `1` | API or business error | Read stderr JSON `error` field; may retry or ask user |
| `2` | Usage / missing config / auth | Fix argv or run `sp auth login` / set `SP_TOKEN` |
| `3` | Auth required | Refresh token |

See [Output contract](./output-contract) and [Exit codes](/en/testing/reference/exit-codes).

## What not to call

* Agent write APIs (`POST /api/storage/record/save`, batch saves) — instrumentation only
* OTLP ingestion (`/v1/traces`) — use collectors
* Mutating `sp ops` without `--confirm` — blocked by design

## Related pages

* [Output contract](./output-contract)
* [Versioning](./versioning)
* [Commands](/en/testing/commands/)

---

---
url: /en/testing/agents/introduction.md
---
# Introduction

SoftProbe is a record-and-replay testing system for Java services. It attaches to an application as a `-javaagent`, observes real traffic in the background, records request and dependency data, then replays recorded cases later with automatic mocking and automatic comparison.

The SoftProbe CLI (`sp`) is the command-line interface to the SoftProbe backend (**sp-backend**). It is specified **AI-agent first**: stable JSON output, predictable exit codes, and artifact files for large diagnosis payloads.

## How SoftProbe Works

SoftProbe has four moving parts:

| Part | Role |
|------|------|
| **Instrumented application** | Your Java service, started with `-javaagent:/path/to/sp-agent.jar`. |
| **SoftProbe Java agent** | Weaves bytecode at runtime, similar operationally to an OpenTelemetry Java agent. It records HTTP, DB, cache, RPC, config, auth, dynamic class, and other dependency data without application code changes. |
| **SoftProbe backend** | Stores recorded cases, policies, replay plans, replay logs, diff results, extraction rules, app config, and agent heartbeats. |
| **`sp` CLI** | Registers apps, configures policies/rules, checks agent status, queries recorded data, starts replay plans, and diagnoses replay failures. |

The normal lifecycle is:

1. **Setup** the CLI and confirm the backend is reachable.
2. **Create an app** and get an `appId`.
3. **List the app** and confirm it is visible to the current user or agent token.
4. **Set recording policy** so the agent knows what traffic to capture.
5. **Install the Java agent** (`sp-agent.jar`) on the service host or image.
6. **Run the app with the agent in recording mode**, pinning `sp.app.id` to the registered `appId`, then send real or test traffic through the app.
7. **List recorded cases** and verify the expected requests were captured.
8. **Set replay policy**: mock policy and compare policy decide what is mocked and how differences are judged.
9. **Create a replay plan** from the recorded cases and run replay.
10. **Check replay results** with case, diff, log, trace, and mock-tree commands.

Replay is therefore not the first action in a fresh system. A replay plan requires recorded cases from an app that was actually run with the SoftProbe agent.

## Agent Startup Example

The agent is attached with a JVM `-javaagent` flag. Pin the app id explicitly in production; do not rely on inferred service names.

```bash
java \
  -javaagent:/opt/softprobe/sp-agent.jar \
  -Dsp.app.id=a1b2c3d4e5f67890 \
  -Dsp.api.url=http://127.0.0.1:8090 \
  -jar order-service.jar
```

`sp.app.id` is the identity SoftProbe uses to isolate recordings, pull config, and match replay data. It is separate from `OTEL_SERVICE_NAME`, which only describes telemetry resource attributes.

## Single public surface

All documented `sp` commands call **sp-backend** over HTTP (`:8090` by default). The Java agent also talks to backend services to pull config, send heartbeats, and upload recording/replay observability data.

## Who uses `sp`?

1. **AI coding agents** — replay failure diagnosis, trace lookup by business ID, policy validation, automated re-runs
2. **CI/CD** — policy gates on pull requests, scheduled replay smoke tests
3. **Platform engineers** — same commands as agents, often without `--json`

The web UI remains available for visual inspection and manual review; the CLI covers **scriptable** and **agent** workflows focused on record-and-replay.

## Relationship to other tools

| Tool | Role |
|------|------|
| **Web UI** | Visual diff, trace inspection, and console review |
| **Java agent** | In-process instrumentation, recording, replay-time mocking, and replay observability |
| **`spcode` CLI** | Local autonomous AI engine and interactive terminal workspace. Shared SSO configuration. |
| **[Policy YAML guide](/en/testing/policy-yaml-guide)** | Schema and examples for `sp policy` inputs |

## Documentation map

* [Softprobe Testing](/en/testing/) — product overview (Java agent, record/replay, policies)
* [For AI agents](./overview) — start here if you write skills or tools
* [spcode CLI](/en/testing/installation/code) — local autonomous AI engine and interactive terminal workspace
* [Commands](/en/testing/commands/) — platform, investigation, and administration

---

---
url: /en/testing/agents/concepts.md
---
# Concepts

For the Java record-and-replay product (agent, policies, replay semantics), see [Softprobe Testing](/en/testing/).

For Istio/Envoy mesh capture and SESSIFY session context (separate from the Java agent), see [Platform core concepts](/en/platform/advanced-guides/concepts).

## Application (`appId`)

A registered service under test. Recording, replay, policies, and extraction rules are all scoped by **`appId`**.

| Field | Role |
|-------|------|
| `appName` | Unique label supplied at registration (`sp app create <appName>`). |
| `appId` | System-generated id (16-character hex). Configure the Java agent and CLI with this value. |

After registration, save `data.appId` from the create response. Attach the SoftProbe Java agent to your JVM with that id and your sp-backend URL, then confirm connectivity with `sp app status <appId>` or `sp app list --json`.

**Agent status** (`online`, `offline`, `never`) is derived from instance heartbeats, not from the app document alone. The server marks an app `offline` when the freshest heartbeat is older than the configured threshold (default 60 seconds).

**CLI reference:** [sp app](/en/testing/commands/app)

## Java agent

The SoftProbe Java agent is attached to the service under test with `-javaagent:/path/to/sp-agent.jar`. It operates like an observability agent operationally, but its purpose is test data capture and replay:

* During **recording**, it observes real requests and dependency interactions and uploads mocker data keyed by `appId` and trace/case ids.
* During **replay**, it restores recorded dependency behavior according to mock policy and emits replay data for comparison.
* It reports heartbeat/status so `sp app status <appId>` can tell whether an app is `online`, `offline`, or `never`.

Minimum startup flags:

```bash
java \
  -javaagent:/opt/softprobe/sp-agent.jar \
  -Dsp.app.id=<appId> \
  -Dsp.api.url=http://<sp-backend-host>:8090 \
  -jar app.jar
```

Pin `sp.app.id` for every production-like deployment. If the id changes between recording and replay, SoftProbe cannot reliably find the original cases or mock data.

## Replay target URL (`targetEnv`)

Replay does not use a symbolic environment label such as `staging` or `prod`. The schedule service field **`targetEnv`** is the **base URL of the running service** that will receive replayed HTTP traffic.

When you run `sp replay run --env <url>`, the CLI sends that value as `targetEnv` on `POST /api/createPlan`. The schedule module parses it as a URI (`DefaultDeploymentEnvironmentProviderImpl`) and builds a `ServiceInstance` whose `url` is that string. Replay senders then issue requests to that URL (for example `DefaultHttpReplaySender` uses `instanceRunner.getUrl()`).

Requirements:

* Use a reachable base URL for the app under test, including scheme and host (and port when not default), for example `http://travel-ota:8080` or `https://order-service.internal:8443`.
* The URL must parse as a URI with a non-empty host; otherwise plan validation fails with *requested target env unable load active instance*.
* This is independent of **`SP_API_URL`** / `api_url` in CLI config, which points at sp-backend (storage, report, schedule APIs), not at the service being replayed.

Optional **`sourceEnv`** on the same request is a separate URI used only when you need a non-default source deployment; the demo stack often leaves it as `pro`.

**CLI reference:** [sp replay](/en/testing/commands/replay) (`--env` → `targetEnv`)

## Recording policy

Declarative YAML (`kind: RecordingPolicy`) controlling what the agent records: sampling, operation include/exclude, time windows, sensitive-field scrubbing.

* Managed via `sp policy recording`
* Schema: [Policy YAML guide](/en/testing/policy-yaml-guide)

## Mock policy

Declarative YAML (`kind: MockPolicy`) controlling replay-time mocking: skip/force mock, tolerance, dependencies.

* `sp policy mock`

## Compare rules

Declarative YAML (`kind: CompareRulePolicy`) controlling diff behavior during replay comparison.

* `sp policy compare`

Policies merge by `metadata.priority`; global defaults ship in `sp-policy-rules` JAR resources.

## Replay plan

A batch replay job with a `planId`. Created by `sp replay run`, tracked with `sp replay status`. Replay plans consume cases already recorded by an instrumented app; a fresh app with no recorded traffic has nothing meaningful to replay.

Schedule service endpoints: `/api/createPlan`, `/api/progress`, `/api/stopPlan`.

## Trace, replay, and plan ids {#trace-replay-and-plan-ids}

Platform ids tie together recording, replay, diff, and **correlated log search**. For a full reference — what each id means, where to find it, and how to triage unified logs — see **[Log correlation IDs](/en/testing/reference/log-correlation-ids)**.

| ID | Meaning | Log lookup (v1) |
|----|---------|-----------------|
| `traceId` | W3C trace id for a recorded or replayed request flow | **`sp logs --trace-id …`** or `GET /api/recorder/logs?trace_id=…` — **only v1 key** |
| `replayId` | One replay **attempt** of a case | Diff/diagnose only — copy **`traceId`** from the same case row for logs |
| `planId` | Replay plan container from `sp replay run` | Case list / diagnose — per-case **`traceId`** for logs |
| `planItemId` | Operation-level item within a plan | Same — not a log lookup key |
| `diffId` | Comparison result row for deep diff fetch | Use `sp replay diff get` — not a log lookup key |

**Where ids appear in CLI output**

| Command | Fields |
|---------|--------|
| `sp replay run --json` | `planId` |
| `sp replay case list --plan … --json` | `replayId`, `traceId`, plan item ids |
| `sp replay metadata <replayId> --json` | `traceId`, linked recording metadata |
| `sp trace find … --json` | `traceId` when resolving business attributes |
| `sp diagnose replay <planId> --json` | Failed cases with ids for follow-up |

Agents should obtain `traceId` via `sp trace find` when users supply business attributes (orderId, caseId) instead of trace ids. For log diagnosis after a replay failure, use **`traceId`** from the failed replay case or the e2e **Softprobe correlation** block (`trace_id` field) — not `replayId` as a log query key.

## Historical coupling: schedule ↔ recording

Replay operation include/exclude lists on schedule configuration are **populated from recording policy at read time**, not stored independently on the schedule document.

Implications:

* Changing recording policy can change which operations appear in replay scope without editing schedule config.
* Diagnosis skills must not assume schedule Mongo documents are the sole source of operation filters.

See server comments on `ScheduleConfigurableHandler` in sp-tr-api.

Test cases are created only via **recording** (instrumented app traffic). Manual case authoring is not part of the CLI workflow.

## Related

* [For agents](./overview)
* [Commands](/en/testing/commands/)

---

---
url: /en/testing/agents/authentication.md
---
# Authentication

Console APIs require the HTTP header:

```http
access-token: <JWT>
```

(Defined as `Constants.ACCESS_TOKEN` in sp-tr-api.)

## Non-interactive login (agents + CI)

### Email verification flow

1. Request code (human step or separate automation):

   ```http
   GET /api/login/getVerificationCode/{userName}
   ```

2. Exchange code for token:

   ```bash
   sp auth login --email user@corp.com --code 123456 --json
   ```

   **REST:** `POST /api/login/verify` with body `{ "userName", "verifyCode", ... }`

3. CLI writes token to the shared config file in
   `${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc` unless `--no-save`.

### CI / agent hosts

**CLI** (record, replay, app management) uses your user JWT:

```bash
export SP_TOKEN="eyJ..."
sp app list --json
```

**Java agent** on Softprobe Cloud uses a **tenant API key** (long-lived, scoped to the org):

```bash
export SP_TENANT_API_KEY="…"   # from sp tenant key ensure or dashboard Settings
export SP_TENANT_ID="35"
sp agent command --app <appId> --json
```

Never commit tokens or API keys to git. Rotate on leak.

### Token refresh

```bash
sp auth refresh --user user@corp.com --json
```

**REST:** `GET /api/login/refresh/{userName}`

Add `--no-save` when an agent or CI job should receive a token without touching
local config:

```bash
export SP_TOKEN="$(sp auth refresh --user user@corp.com --no-save --json | jq -r .data.token)"
```

## Guest login

If enabled on the server:

```bash
sp auth login --guest --json
```

**REST:** `POST /api/login/loginAsGuest`

## OAuth

OAuth flows are browser-based. Agents should use pre-provisioned `SP_TOKEN` rather than driving OAuth interactively.

Document for humans: `GET /api/login/oauthInfo/{oauthType}`, `POST /api/login/oauthLogin`.

## Who am I

```bash
sp auth whoami --json
```

Decodes JWT `userName` or calls profile endpoint when implemented.

## Errors

| Situation | Exit code |
|-----------|-----------|
| Missing token with `--json` | `3` (`AUTH_REQUIRED`) |
| Expired or invalid token | `1` (`API_ERROR`) |

## Related

* [config](/en/testing/installation/configuration)
* [auth command](/en/testing/commands/auth)

---

---
url: /en/testing/agents/output-contract.md
---
# Output contract

All public `sp` commands follow this contract when `--json` is set.

## CLI envelope (stdout on success)

```json
{
  "ok": true,
  "command": "app list",
  "data": { }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `ok` | boolean | Always `true` on exit 0 |
| `command` | string | Normalized command name for logging |
| `data` | object | Command-specific payload (see [JSON types](/en/testing/reference/json-types)) |

## CLI envelope (stderr on failure, exit 1)

```json
{
  "ok": false,
  "command": "replay run",
  "error": {
    "code": "API_ERROR",
    "message": "Human-readable summary",
    "httpStatus": 500,
    "backend": { }
  }
}
```

`backend` optionally contains the raw SoftProbe `Response` or schedule `CommonResponse` body for debugging.

## Backend response shapes

### sp-tr-api (`Response`)

Most console APIs return:

```json
{
  "responseStatusType": { "responseCode": 0, "responseDesc": "success" },
  "body": { }
}
```

The CLI maps `responseCode !== 0` to exit code `1`.

### sp-replay-schedule (`CommonResponse`)

Replay control (`createPlan`, `progress`, …) uses:

```json
{
  "result": 1,
  "desc": "success",
  "data": { }
}
```

The CLI maps `result !== 1` (per server convention) to exit code `1`.

## Artifacts (large output)

When response bodies exceed an internal threshold (recommended: 64 KiB) or when the command always produces files (diff detail, log download):

**stdout:**

```json
{
  "ok": true,
  "command": "replay diff get",
  "data": {
    "artifact": ".sp-work/diff-abc123.json",
    "summary": {
      "diffId": "abc123",
      "diffResultCode": 1,
      "categoryName": "..."
    }
  }
}
```

Agents should **read the artifact file** with their file tool, not expect full payloads on stdout.

Default `--out-dir`: `.sp-work/` in the current working directory (override per invocation).

## Pagination

List commands accept:

| Flag | Default | Description |
|------|---------|-------------|
| `--page` | `1` | 1-based page index |
| `--limit` | `20` | Page size (max 100 unless documented) |
| `--fields` | all | Comma-separated field filter |

Paginated JSON:

```json
{
  "ok": true,
  "command": "replay case list",
  "data": {
    "items": [],
    "page": 1,
    "pageSize": 20,
    "total": 142,
    "hasMore": true
  }
}
```

## Human-readable mode (no `--json`)

* Tables on stdout
* Errors on stderr as plain text
* Colors only when stdout is a TTY and `--pretty` is set (optional; not required for v1 implementers)

## Non-interactive rule

If `--json` is set and authentication is missing, the CLI must **not** prompt. Exit `3` with:

```json
{
  "ok": false,
  "error": { "code": "AUTH_REQUIRED", "message": "Set SP_TOKEN or run sp auth login" }
}
```

---

---
url: /en/testing/agents/versioning.md
---
# Versioning and stability

## Binary version

`sp version` prints the CLI build version (semver). Agents may log this on every session for support correlation.

## Command argv stability

| Change type | Policy |
|-------------|--------|
| New subcommand | Allowed anytime |
| New optional flag | Allowed anytime |
| Renaming subcommand or flag | Deprecate for one minor release; print stderr warning |
| Removing subcommand or flag | Major version bump only |

Document deprecations in release notes and in command `--help` text.

## JSON field stability

| Change type | Policy |
|-------------|--------|
| New fields in `data` | Allowed; agents must ignore unknown fields |
| Renaming or removing fields | Major version bump, or add `dataVersion` in envelope |
| Changing field types | Major version bump |

Recommended: include `"schemaVersion": 1` in the CLI envelope in a future release.

## API backend compatibility

The CLI targets **sp-backend** unified deployment. Backend URL is configurable; the CLI does not pin to separate storage/schedule hostnames (those are internal to sp-backend).

When backend APIs change, update [API mapping](/en/testing/reference/api-mapping) in the same PR as the CLI implementation.

---

---
url: /en/testing/examples.md
---

# Examples

Testing examples show how to automate Softprobe with `sp`.

* Diagnose replay failures
* Look up traces and correlated logs
* Run CI policy gates
* Manage GitOps policies

---

---
url: /en/testing/commands.md
---
# Commands

Reference for public `sp` subcommands. Use `--json` on API-backed commands. See [Output contract](/en/testing/agents/output-contract).

## Lifecycle (recommended)

Job-oriented commands that follow record-and-replay order:

| Command | Synopsis |
|---------|----------|
| [setup](./setup) | Configure self-hosted backend URL; optional Spcode Service (Linux) |
| [demo](./demo) | `start`, `traffic`, `replay`, `status`, `stop` — [Travel OTA demo](https://github.com/softprobe/demo-ota) stack |
| [agent](./agent) | `download`, `command` — install jar and JVM flags |
| [record](./record) | `case list` — recorded entry cases before replay |
| [diagnose](./diagnose) | `replay`, `trace` — bundled investigation workflows |

## Platform

Connect, authenticate, manage apps and policies, run replay plans.

| Command | Synopsis |
|---------|----------|
| [config](./config) | Profiles, URL, init |
| [auth](./auth) | Login, whoami, refresh |
| [app](./app) | List, create, agent status, recent replays |
| [policy](./policy) | Recording, mock, compare YAML policies |
| [replay](./replay) | Run, status, stop, rerun plans |
| [health](./health) | Cluster health |
| `version` | CLI version string |

## Investigation

Recorded data, traces, and replay failures.

| Command | Synopsis |
|---------|----------|
| [record](./record) | Query recordings and completeness |
| [trace](./trace) | Find traces by business attributes |
| [logs](./logs) | Correlated logs by `trace_id` — see [Log correlation IDs](/en/testing/reference/log-correlation-ids) |
| [replay case](./replay-case) | List cases, metadata, mock tree |
| [replay diff](./replay-diff) | Diff artifacts, compare results |
| [extraction-rule](./extraction-rule) | Business attribute extraction rules |

Investigation commands support `--out-dir`, `--page`, and `--limit` unless noted.

## Administration

Groups, system config, diagnostics, legacy APIs.

| Command | Synopsis |
|---------|----------|
| [group](./group) | User groups and app grants |
| [grant](./group) | App grant listing (`grant list`) |
| [system](./system) | System config keys |
| [ops](./ops) | Storage and schedule diagnostics |
| [config legacy](./config-legacy) | Legacy `/api/config/*` (deprecated) |

## Global flags

```text
--json          Machine-readable stdout (required for agents)
--profile       Config profile name
--api-url       Override backend URL
--token         Override JWT (else SP_TOKEN / config)
--config        Extra config file (JSONC overlay)
--quiet         Suppress non-error stderr
--out-dir       Directory for artifact files
--page          Page number (investigation list commands)
--limit         Page size / max items
```

## Cheat sheet

```bash
sp config init && sp auth login --email u@c.com --code 123456 --json
sp doctor --json
sp app create my-svc --json
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
sp agent command --app <appId> --agent-jar ./sp-agent.jar --json   # copy startCommand into your run script
sp record case list --app <appId> --since -1h --json
sp replay run --app <appId> --env http://your-service:8080 --from -24h --json
sp replay status <planId> --watch --json
sp diagnose replay <planId> --failed-only --out-dir .sp-work --json
```

## Related

* [Quickstart](/en/testing/getting-started)
* [For AI agents](/en/testing/agents/overview)
* [Examples](/en/testing/examples/agent-diagnose-replay)
* [API mapping](/en/testing/reference/api-mapping)

---

---
url: /en/testing/policy-yaml-guide.md
---

# Policy YAML guide

Softprobe Testing uses **declarative YAML policies** (`apiVersion: softprobe.ai/v1`) for recording, replay mocking, and diff comparison. Policies are versioned resources scoped to apps (and optionally environments and operations), merged by priority, and applied by sp-backend at runtime.

Manage the three CLI-supported kinds with:

```bash
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
sp policy mock apply -f mock.yaml --json
sp policy compare apply -f compare.yaml --json
```

| Kind | What it controls | CLI |
|------|------------------|-----|
| `RecordingPolicy` | Sampling, time windows, operation include/exclude, serialize skip, time mock | `sp policy recording` |
| `MockPolicy` | Skip/force mock, lookup tolerance, cross-app deps, fallback | `sp policy mock` |
| `CompareRulePolicy` | Ignore paths, decompress, transforms, array matchers, CEL validations | `sp policy compare` |

JSON schemas live in the backend module `sp-policy-rules` (`src/main/resources/schema/`). Use them in your editor via `# yaml-language-server: $schema=...` for completions.

## Shared document shape

Every policy has the same outer structure:

```yaml
apiVersion: softprobe.ai/v1
kind: RecordingPolicy   # or MockPolicy / CompareRulePolicy
metadata:
  name: my-app-recording
  priority: 100
  description: "optional"
  enabled: true         # RecordingPolicy / MockPolicy only
selector:
  appIds: [my-app-id]
  envTags:
    env: [prod]
spec:
  # kind-specific
```

### Common fields

| Field | Type | Required | Notes |
|-------|------|----------|-------|
| `apiVersion` | string | yes | Must be `softprobe.ai/v1` |
| `kind` | string | yes | One of the three kinds above |
| `metadata.name` | string | yes | Unique id (lowercase-dash) |
| `metadata.priority` | integer | no | Default `0`. Higher merges later and wins conflicts |
| `metadata.description` | string | no | Free text |
| `metadata.enabled` | boolean | no | Recording/Mock only; default `true` |
| `selector.matchAll` | boolean | no | Apply to every app (system defaults use this at priority 0) |
| `selector.appIds` | string\[] | one of\* | Exact app ids |
| `selector.appIdPattern` | string | one of\* | Glob, e.g. `order-*` |
| `selector.excludeAppIds` | string\[] | no | Subtract from match |
| `selector.envTags` | map | no | Tag key → list of allowed values, e.g. `env: [prod, staging]`. Matches agent tags from `-Dsp.mocker.tags=env=prod`. Conjunctive: every declared key must match. Omitted = all environments |
| `selector.operationNames` | string\[] | no | Exact operation names. **Recording and Mock only** — rejected on CompareRulePolicy selector |
| `selector.operationNamePatterns` | string\[] | no | Globs, e.g. `/api/order/*`. **Recording and Mock only** |

\*Selector must include at least one of `matchAll`, `appIds`, or `appIdPattern`.

**Selector dimensions** are conjunctive: app match AND (optional) env match AND (optional) operation match. Empty operation constraints mean “all operations for matched apps.”

### Merging

Multiple policies matching the same app are merged in **priority ascending** (higher number applied last):

| Kind | Merge behavior |
|------|----------------|
| **Compare** | Scalars overridden; lists/maps unioned; last-write-wins on keys |
| **Recording** | Scalars overridden; lists unioned; `timeMock` sticky-true (any policy sets true → true); `serializeSkip` merged by `className` + field name union |
| **Mock** | Scalars overridden; `skipMock`/`forceMock` unioned; `matchTolerance` and `multiServiceDependencies` keyed by pattern/app with last-write-wins |

Priority-0 `default-global-*-policy.yaml` files ship inside sp-backend; set `priority > 0` on your policies to override.

### Runtime pipeline

```text
YAML → parse → validate → Mongo → resolve + merge → compile → PolicyCache (fresh 60s, stale 24h)
```

If the cache cannot load from Mongo in time, callers degrade safely (recording: do not record; mock: pass-through).

### Authoring tips

1. Use JSON Schema in your editor for inline validation.
2. Avoid `matchAll: true` in app-specific policies unless you intend a global rule — defaults already provide a floor.
3. Comment the **why** in YAML; the schema documents the **what**.

***

## RecordingPolicy

Controls **what the agent records**: sampling, schedule, operation filters, optional scrubbing metadata, serialization skips, and record-time time mock.

### `spec` fields

| Section | Field | Type | Semantics |
|---------|-------|------|-----------|
| `sampling` | `ratePerHundredSeconds` | integer ≥ 0 | Max requests recorded per 100-second window. `0` = no recording |
| `sampling` | `machineCountLimit` | integer ≥ 1 or omit | Max concurrent recording instances in the env group. **Omit = unlimited.** Avoid `1` unless you understand quota pinning |
| `timeWindow` | `daysOfWeek` | `MON`…`SUN` or `*` | Omit section = 24/7 |
| `timeWindow` | `from` / `to` | `HH:mm` | Both required together; `from` strictly before `to` (validator). Agent uses JVM local timezone |
| `operations` | `exclude` | string\[] | Globs not recorded (deny list) |
| `operations` | `include` | string\[] | When non-empty: **whitelist** — only these ops recorded; `exclude` still applies after include |
| `sensitiveData` | `headers` | string\[] | Case-insensitive header names (see note below) |
| `sensitiveData` | `bodyPaths` | string\[] | JSONPath; must start with `$` |
| `sensitiveData` | `queryParams` | string\[] | Query parameter names |
| `sensitiveData` | `placeholder` | string | Replacement text; default `***` |
| `serializeSkip` | `className` | string | Java class |
| `serializeSkip` | `fieldNames` | string\[] | Fields to skip in bytecode serialization |
| | `timeMock` | boolean | Mock `java.time.*` at record time for deterministic replays |
| | `extras` | map string→string | Forwarded to agent `extendField` |

::: warning `sensitiveData` on the record path
The backend forwards `sensitiveData` to the agent wire DTO, but the **Java agent does not apply scrubbing at record time yet**. For replay mock-key noise, use `MockPolicy.spec.matchTolerance`. For view/query-time masking, use `SensitivePolicy` (REST API; no `sp policy` CLI today).
:::

Replay schedules read include/exclude from the compiled recording policy when building operation scope — changing recording policy can change replay scope without editing the schedule document.

### Full example

```yaml
apiVersion: softprobe.ai/v1
kind: RecordingPolicy
metadata:
  name: order-service-prod
  description: "Prod recording — 50 reqs/100s, business hours, scrub metadata."
  priority: 100
  enabled: true
selector:
  appIds: [order-service]
  envTags:
    env: [prod]
  operationNamePatterns:
    - "/api/order/*"
spec:
  sampling:
    ratePerHundredSeconds: 50
    machineCountLimit: 3
  timeWindow:
    daysOfWeek: [MON, TUE, WED, THU, FRI]
    from: "09:00"
    to: "18:00"
  operations:
    exclude:
      - "/health"
      - "/metrics/**"
      - "/actuator/**"
    # include:
    #   - "/api/order/create"
    #   - "/api/order/pay"
  sensitiveData:
    headers: [authorization, cookie, x-api-key]
    bodyPaths:
      - "$.password"
      - "$.user.phone"
    queryParams: [token, sessionId]
    placeholder: "***"
  serializeSkip:
    - className: com.example.order.domain.Order
      fieldNames: [internalNote]
  timeMock: false
  extras: {}
```

### Whitelist-only recording example

```yaml
apiVersion: softprobe.ai/v1
kind: RecordingPolicy
metadata:
  name: order-service-whitelist
  priority: 110
selector:
  appIds: [order-service]
spec:
  sampling:
    ratePerHundredSeconds: 100
  operations:
    include:
      - "/api/order/**"
    exclude:
      - "/api/order/internal/**"
```

***

## MockPolicy {#mockpolicy}

Controls **replay-time** dependency mocking: which calls use recorded data vs real downstream, mock-key tolerance, multi-app dependency maps, and behavior when no mock exists.

### `spec` fields

| Section | Field | Type | Semantics |
|---------|-------|------|-----------|
| | `mockByDefault` | boolean | Default `true`: mock all dependencies except `skipMock`. `false`: call real downstream except `forceMock` |
| `operations` | `skipMock` | string\[] | `Category:operationGlob` — bypass mock, hit real downstream |
| `operations` | `forceMock` | string\[] | `Category:operationGlob` — must mock; wins over `skipMock` |
| `matchTolerance` | `operationPattern` | string | Glob on **entry** operation |
| `matchTolerance` | `ignoreHeaders` | string\[] | Dropped from mock fingerprint |
| `matchTolerance` | `ignoreQueryParams` | string\[] | Dropped from mock fingerprint |
| `matchTolerance` | `ignoreBodyPaths` | string\[] | JSON paths ignored in mock lookup |
| `multiServiceDependencies` | `downstreamApp` | string | Other app id |
| `multiServiceDependencies` | `operations` | string\[] | Exact downstream ops to mock from session |
| `multiServiceDependencies` | `operationPatterns` | string\[] | Globs (one of `operations` or `operationPatterns` required) |
| `fallback` | `strategy` | enum | `FAIL` (default), `PASS_THROUGH`, `RETURN_DEFAULT` |
| `fallback` | `defaultResponse` | object | Required when `RETURN_DEFAULT`: `statusCode`, `contentType`, `body`, `headers` |

### Dependency categories for `skipMock` / `forceMock` {#dependency-categories}

Use **`Category:operationGlob`** — bare globs without a category prefix are **rejected**.

| Category | Typical `operationGlob` examples |
|----------|----------------------------------|
| `HttpClient` | `/payment/charge`, `/internal/**` |
| `Database` | `select_*`, `*` |
| `Redis` | `GET user:*` |
| `DubboConsumer` | `com.foo.Service#method` |
| `SofaConsumer` | RPC operation names |
| `DubboStreamProvider` | Stream ops |
| `QMessageProducer` | Topic or message id |
| `ConfigFile` | Config keys |
| `UserDynamic` | `com.foo.Cache.get` (user-configured dynamic class) |
| `DynamicClass` | `SystemTime.**`, `RandomSource.**` (built-in; global default force-mocks these — user `skipMock` has no effect) |
| `Encryption` | `com.foo.Crypto.encrypt` |
| `IbmMQ` | MQ destinations |
| `SoapClient` | SOAP actions |
| `DSR` | DSR operations |

Glob in the operation segment: `*` (one path segment), `**` (multiple segments), `?` (single character).

::: warning Entry types are not mock policy keys
Do not use `Servlet`, `DubboProvider`, or other **entry** categories in mock rules — only **dependency** categories above.
:::

### Full example

```yaml
apiVersion: softprobe.ai/v1
kind: MockPolicy
metadata:
  name: order-service-replay
  description: "Replay defaults for order-service"
  priority: 100
  enabled: true
selector:
  appIds: [order-service]
  envTags:
    env: [staging, prod]
spec:
  mockByDefault: true
  operations:
    skipMock:
      - "HttpClient:/health"
      - "HttpClient:/internal/admin/**"
    forceMock:
      - "HttpClient:/payment/charge"
      - "Database:*"
  matchTolerance:
    - operationPattern: "/api/order/**"
      ignoreHeaders: [x-request-id, x-trace-id, x-b3-*]
      ignoreQueryParams: [_t, timestamp]
  multiServiceDependencies:
    - downstreamApp: payment-service
      operations: ["/charge", "/refund"]
    - downstreamApp: inventory-service
      operationPatterns: ["/reserve", "/release"]
  fallback:
    strategy: FAIL
```

### Pass-through fallback example

```yaml
apiVersion: softprobe.ai/v1
kind: MockPolicy
metadata:
  name: order-service-isolated-replay
  priority: 100
selector:
  appIds: [order-service]
spec:
  fallback:
    strategy: PASS_THROUGH
```

### `RETURN_DEFAULT` fallback example

```yaml
apiVersion: softprobe.ai/v1
kind: MockPolicy
metadata:
  name: order-service-default-mock
  priority: 100
selector:
  appIds: [order-service]
spec:
  operations:
    forceMock:
      - "HttpClient:/legacy/ping"
  fallback:
    strategy: RETURN_DEFAULT
    defaultResponse:
      statusCode: 200
      contentType: application/json
      body: '{"status":"ok"}'
      headers:
        x-mock: "true"
```

***

## CompareRulePolicy {#comparerulepolicy}

Controls **diff noise** during replay comparison: ignored paths, decompression, transforms, array matching, and post-diff CEL filters.

::: warning Operation scope
Do **not** put `operationNames` or `operationNamePatterns` on `selector` — the validator rejects them. Use `spec.operationSpecs[]` for per-operation overlays.
:::

### `spec` fields

| Section | Field | Type | Semantics |
|---------|-------|------|-----------|
| `defaults` | `timeToleranceMs` | integer ≥ 0 | Used by CEL `time_tolerance_ms` and timestamp rules |
| `defaults` | `ignoreHeaderPatterns` | string\[] | Header name globs skipped in compare |
| | `includePaths` | string\[] | JSON Pointer whitelist; empty = compare all |
| | `excludePaths` | string\[] | JSON Pointer blacklist (pre-filter) |
| `decompress[]` | `path` | string | JSON Pointer or glob (`/data/**`) |
| `decompress[]` | `codec` | enum | `PLAIN_JSON`, `BASE64_JSON`, `GZIP_BASE64_JSON` |
| `transforms[]` | `path` | string | JSON Pointer |
| `transforms[]` | `expression` | string | CEL expression to normalize value before compare |
| `arrays[]` | `path` | string | Array field path |
| `arrays[]` | `strategy` | enum | `BY_INDEX` (default) or `BY_KEY` |
| `arrays[]` | `keys` | string\[] | Required when `BY_KEY` |
| `arrays[]` | `references[]` | object | `field`, `target`, `targetKey` for FK-style array linking |
| `validations[]` | `id` | string | Unique rule id |
| `validations[]` | `name` | string | Display name |
| `validations[]` | `expression` | string | CEL; must evaluate to bool |
| `validations[]` | `action` | enum | `DROP` only (ignore this diff) |
| `validations[]` | `enabled` | boolean | Default `true` |
| `validations[]` | `message` | string | Human-readable reason |
| `operationSpecs[]` | `operationNames` / `operationNamePatterns` | string\[] | Which entry ops this overlay applies to |
| `operationSpecs[]` | `spec` | object | Same leaf fields as top-level spec (no nested `operationSpecs`) |

Paths use **JSON Pointer** syntax (`/foo/bar`). Globs: `*` = one segment, `**` = any depth.

### CEL variables and helpers

Available in `validations` and `transforms`:

| Name | Type | Meaning |
|------|------|---------|
| `left` | string | Recorded value |
| `right` | string | Replay value |
| `path` | string | JSON Pointer path |
| `pointer` | string | Alias for path |
| `fieldName` | string | Leaf field name |
| `category` | string | Dependency category, e.g. `DATABASE` |
| `time_tolerance_ms` | int | From `defaults.timeToleranceMs` |
| `isTimestamp(s)` | function | True if string is a known timestamp format |
| `toTimestamp(s)` | function | Epoch millis |
| `matches` | method | Regex on strings |

### Full example (app defaults + per-operation overlay)

```yaml
apiVersion: softprobe.ai/v1
kind: CompareRulePolicy
metadata:
  name: order-service-compare
  description: "Compare rules for order-service"
  priority: 100
selector:
  appIds: [order-service]
spec:
  defaults:
    timeToleranceMs: 60000
    ignoreHeaderPatterns: [x-request-id, x-b3-*]
  excludePaths:
    - "/response/headers/date"
    - "/response/body/metadata/generatedAt"
  decompress:
    - path: "/response/body/payload"
      codec: GZIP_BASE64_JSON
  arrays:
    - path: "/response/body/items"
      strategy: BY_KEY
      keys: [skuId]
  validations:
    - id: ignore-uuids
      name: Ignore UUID pairs
      expression: >-
        left.matches("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
        && right.matches("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
      action: DROP
      enabled: true
      message: Both sides are UUIDs
    - id: ignore-db-body
      name: Skip raw SQL body on DATABASE
      expression: 'category == "DATABASE" && fieldName == "body"'
      action: DROP
      enabled: true
  operationSpecs:
    - operationNamePatterns: ["/api/order/export"]
      spec:
        excludePaths:
          - "/response/body/exportJobId"
        validations:
          - id: export-timestamp-tolerance
            expression: >-
              isTimestamp(left) && isTimestamp(right)
              && math.abs(toTimestamp(left) - toTimestamp(right)) <= time_tolerance_ms
            action: DROP
            enabled: true
```

***

## Related configuration {#related-configuration}

### Dynamic classes

**Not** part of `RecordingPolicy`. Configure methods in **Dynamic class configuration** (dashboard or storage API). At replay, they appear under `UserDynamic` or built-in `DynamicClass` (e.g. `SystemTime.*`, `RandomSource.*`). Control mock behavior with `MockPolicy` `forceMock` / `skipMock`, not recording policy.

Matching at replay: exact parameter match first, then fuzzy match by signature.

### SensitivePolicy

Fourth policy kind for **query/view-time** PII masking (REST API; no `sp policy` CLI). Separate from `RecordingPolicy.spec.sensitiveData`.

| Field | Notes |
|-------|-------|
| `spec.fieldNameRules[]` | `pattern` (Java regex), `type`: `NAME`, `ID_CARD`, `PHONE`, `EMAIL`, `PASSPORT`, `DEFAULT`, `NONE` |
| `spec.contentRules[]` | Same shape; matches field **values** |

```yaml
apiVersion: softprobe.ai/v1
kind: SensitivePolicy
metadata:
  name: order-service-sensitive
  priority: 100
selector:
  appIds: [order-service]
spec:
  fieldNameRules:
    - pattern: "(?i)^password$"
      type: NAME
  contentRules:
    - pattern: "^1[3-9]\\d{9}$"
      type: PHONE
```

## Related

* [Policies overview](/en/testing/policies)
* [Replay and diff](/en/testing/replay-and-diff)
* [CLI: policy command](/en/testing/commands/policy)

---

---
url: /en/testing/installation/configuration.md
---

# Client configuration

This page is the reference for client-side (`sp` CLI, Java agent, coding engine) config files and advanced options. Softprobe uses one XDG namespace shared by `sp`, `sp code`, and the Java agent.

| Path | Purpose |
|------|---------|
| `~/.config/softprobe/config.jsonc` | Shared backend URL and credentials |
| `~/.config/softprobe/sp.jsonc` | CLI profiles and overrides |
| `~/.config/softprobe/spcode.jsonc` | Internal coding engine settings |
| `~/.local/share/softprobe/agent/sp-agent.jar` | Installed Java agent |
| `~/.local/share/softprobe/bin/spcode` | Internal coding engine binary |

Use `sp setup` for the backend URL. Use `sp code` for coding-engine and model-provider readiness.

## Spcode Service (Linux systemd)

When you install Spcode Service with [`sp setup --install-spcode-service`](./#spcode-service), the shared workbench uses **the same Softprobe settings as the account that ran install** (backend URL, MCP, agent instructions, skills).

Configure Softprobe as that install account. After you change settings the service should pick up, restart it:

```bash
sudo systemctl restart spcode-web.service
```

## MCP tools and agent instructions {#spcode-service-mcp-agents}

`sp setup` / `sp code` do **not** create coding-engine MCP settings or global agent instructions. Add them under:

| File | Purpose |
|------|---------|
| `~/.config/spcode/opencode.jsonc` | MCP servers and engine settings |
| `~/.config/spcode/AGENTS.md` | Global agent instructions |

These paths apply to both personal `sp code web` and Linux Spcode Service. Restart the UI process (or `spcode-web.service`) after you edit them.

### Example: Feishu / Lark MCP + `AGENTS.md`

::: warning Prerequisite: Node.js
This MCP example launches `@larksuiteoapi/lark-mcp` with **`npx`**, so the host must have **Node.js** installed (which provides `npx`). Softprobe install does **not** install Node.js for you.

Check:

```bash
node -v
npx -v
```

If those commands are missing, install Node.js (LTS) from [nodejs.org](https://nodejs.org/), or with your OS package manager (for example `apt install nodejs npm` on Debian/Ubuntu, or `dnf install nodejs` on RHEL/Fedora). For Spcode Service, confirm `npx -v` also works when run as root (`sudo npx -v`), because the service process runs as root.
:::

1. Create `~/.config/spcode/opencode.jsonc` (replace placeholders with your app credentials; never commit real secrets):

```jsonc
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "lark-mcp": {
      "type": "local",
      "enabled": true,
      "command": [
        "npx",
        "-y",
        "@larksuiteoapi/lark-mcp",
        "mcp",
        "-a",
        "YOUR_FEISHU_APP_ID",
        "-s",
        "YOUR_FEISHU_APP_SECRET",
        "--token-mode",
        "tenant_access_token",
        "--domain",
        "https://open.feishu.cn",
        "-t",
        "docx.v1.document.get,docx.v1.document.rawContent"
      ]
    }
  }
}
```

The host also needs network access to Feishu/Lark (`https://open.feishu.cn`).

2. Create `~/.config/spcode/AGENTS.md` so agents know how to use that MCP. Example (Chinese customer):

```md
# Softprobe 服务端 Agent 说明

关于应用、Git 分支、录制/回放仓库信息，**不要硬编码**。

请使用 **lark-mcp** MCP 服务，从以下飞书文档获取最新内容：

https://example.feishu.cn/docx/YOUR_DOC_TOKEN

优先调用 `docx.v1.document.rawContent`（或 lark-mcp 中的等价工具）读取该文档，并以返回的表格/文本作为应用名称与分支配置的唯一真实来源。
```

Replace the Feishu doc URL with your own. You can point agents at any MCP you configure in `opencode.jsonc`, not only Feishu.

## Skills {#skills}

Skills are folders with a `SKILL.md` file. Softprobe loads them automatically from:

| Location | Path |
|----------|------|
| Global (this machine) | `~/.config/spcode/skill/<name>/SKILL.md` or `~/.config/spcode/skills/<name>/SKILL.md` |
| Project | `.opencode/skill/<name>/SKILL.md` or `.opencode/skills/<name>/SKILL.md` in the opened repo |

Each `SKILL.md` needs frontmatter with at least `name` and `description`. Example:

```md
---
name: my-skill
description: Use when the user asks about release checklists.
---

# My skill

Steps the agent should follow…
```

To load skills from another directory, set `skills.paths` in `~/.config/spcode/opencode.jsonc`.

There is no separate “create skill” command — add the folder and file yourself (or ask the coding agent to help). After changing global skills used by Spcode Service, restart `spcode-web.service`.

---

---
url: /en/testing/reference/exit-codes.md
---
# Exit codes

Contract for agents and CI scripts invoking `sp`.

| Code | Name | Meaning |
|------|------|---------|
| `0` | Success | Parsed JSON on stdout when `--json` |
| `1` | API\_ERROR | Backend returned error or HTTP non-success (includes `NO_RECORDED_CASES` on `replay run`) |
| `2` | USAGE / PROFILE\_NOT\_FOUND / CONFIG\_\* | Invalid flags, missing config, parse errors, unknown profile |
| `3` | AUTH\_REQUIRED | No token available in non-interactive mode |

## stderr with `--json`

On exit `1` or `2` or `3`, stderr contains a JSON error object:

```json
{
  "ok": false,
  "command": "app list",
  "error": {
    "code": "AUTH_REQUIRED",
    "message": "Set SP_TOKEN or run sp auth login"
  }
}
```

## Config error codes (exit `2`)

| Code | Cause |
|------|-------|
| `CONFIG_MISSING` | No `config.jsonc` / `sp.jsonc`; run `sp config init` |
| `CONFIG_PARSE_ERROR` | Invalid JSONC or schema validation failure |
| `CONFIG_WRITE_ERROR` | Could not write config file |
| `PROFILE_NOT_FOUND` | Selected profile missing from merged config |

## Agent retry guidance

| Code | Retry? |
|------|--------|
| 0 | No |
| 1 | Sometimes — after fixing data or if transient 5xx |
| 2 | No — fix invocation |
| 3 | After refresh/login |

---

---
url: /en/testing/reference/json-types.md
---
# JSON types

CLI-level envelope: see [Output contract](/en/testing/agents/output-contract).

## Common `data` shapes

### ApplicationListItem

Used in `sp app list` → `data.items[]`.

```json
{
  "appId": "string",
  "appName": "string",
  "name": "string",
  "agentStatus": "online | offline | never",
  "lastSeenAt": 0,
  "agentVersion": "string",
  "env": "string",
  "tags": ["string"],
  "worktreeDirectory": "string"
}
```

### ApplicationCreateResult

Used in `sp app create` → `data`.

```json
{
  "success": true,
  "appId": "string",
  "msg": "string"
}
```

### AgentStatus

Used in `sp app status` → `data`.

```json
{
  "appId": "string",
  "status": "online | offline | never",
  "instanceCount": 0,
  "lastSeenAt": 0,
  "agentVersion": "string"
}
```

### PolicyValidateResult

```json
{
  "valid": true,
  "errors": [{ "path": "spec.sampling.rate", "message": "..." }],
  "warnings": []
}
```

### ReplayPlanCreated

```json
{
  "planId": "string",
  "result": 1,
  "desc": "string"
}
```

### ReplayProgress

```json
{
  "planId": "string",
  "status": "RUNNING | FINISHED | FAILED | CANCELLED",
  "percent": 0,
  "finished": false
}
```

### TraceSummary

```json
{
  "traceId": "string",
  "endpoint": "string",
  "status": "string",
  "durationMs": 0,
  "startedAt": "ISO-8601",
  "attrs": { "orderId": "ORD-123" }
}
```

### ArtifactResult

```json
{
  "artifact": "relative/path.json",
  "summary": {}
}
```

### PaginatedList

```json
{
  "items": [],
  "page": 1,
  "pageSize": 20,
  "total": 0,
  "hasMore": false
}
```

## Backend passthrough

When `--include-backend` (optional debug flag) is set, implementers may add `data._backend` with the raw `Response.body` or `CommonResponse.data`.

Agents should not depend on `_backend` in production skills.

## schemaVersion

Future CLI releases may add top-level `"schemaVersion": 1` in the envelope. Agents must ignore unknown top-level fields.

---

---
url: /en/testing/reference/api-mapping.md
---
# API mapping

Authoritative **REST ↔ CLI** reference for sp-backend (default port **8090**). Header `access-token` required on console APIs unless noted.

**Legend:** Category `platform` | `investigation` | `admin` | `—` (not exposed)

## Auth

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/login/getVerificationCode/{userName}` | (precursor to login) | platform |
| POST | `/api/login/verify` | `sp auth login` | platform |
| GET | `/api/login/refresh/{userName}` | `sp auth refresh` | platform |
| POST | `/api/login/loginAsGuest` | `sp auth login --guest` | platform |
| POST | `/api/login/oauthLogin` | — | — |
| GET | `/api/login/userProfile/{userName}` | `sp auth whoami` (future) | investigation |

## Applications

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/applications/list` | `sp app list` | platform |
| POST | `/api/applications/create` | `sp app create` | platform |
| GET | `/api/applications/{appId}/agent-status` | `sp app status` | platform |
| GET | `/api/applications/{appId}/replays/recent` | `sp app replays` | platform |
| GET | `/api/applications/{appId}/extraction-rules` | `sp extraction-rule list` | investigation |
| PUT | `/api/applications/{appId}/extraction-rules` | `sp extraction-rule apply` | investigation |
| POST | `/api/extraction-rules/preview` | `sp extraction-rule preview` | investigation |
| GET | `/api/agent/extraction-rules` | — (agent) | — |

## Policies

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/recording-policies/policies` | `sp policy recording list` | platform |
| GET | `/api/recording-policies/policies/{id}` | `sp policy recording get` | platform |
| POST | `/api/recording-policies/policies` | `sp policy recording apply` | platform |
| POST | `/api/recording-policies/policies/yaml` | `sp policy recording apply -f` | platform |
| DELETE | `/api/recording-policies/policies/{id}` | `sp policy recording delete` | platform |
| POST | `/api/recording-policies/policies/validate` | `sp policy recording validate` | platform |
| GET | `/api/recording-policies/policies/{id}/yaml` | `sp policy recording export` | platform |
| \* | `/api/mock-policies/*` | `sp policy mock …` | platform |
| \* | `/api/compare-rules/*` | `sp policy compare …` | platform |

## Replay control (schedule)

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/createPlan` | `sp replay run` (`targetEnv` = service base URL) | platform |
| GET | `/api/createPlan` | — (webhook only) | — |
| GET | `/api/progress` | `sp replay status` | platform |
| GET | `/api/stopPlan` | `sp replay stop` | platform |
| POST | `/api/reRunPlan` | `sp replay rerun` | platform |
| POST | `/api/createRealTimePlan` | `sp replay realtime create` | investigation |
| GET | `/api/queue/control/*` | `sp replay realtime queue …` | investigation |
| POST | `/api/compareCase` | `sp replay compare` | investigation |
| GET | `/api/queryNoise` | `sp replay noise query` | investigation |
| POST | `/api/excludeNoise` | `sp replay noise exclude` | investigation |

## Traces

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/traces/find-by-attr` | `sp trace find` | investigation |
| GET | `/api/traces/{traceId}/summary` | `sp trace get` | investigation |
| POST | `/api/traces/attr-rule-statistics` | `sp trace stats` | investigation |

## Storage record (read)

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/storage/record/query` | `sp record query` | investigation |
| GET | `/api/storage/record/trace/{traceId}` | `sp record trace` | investigation |
| GET | `/api/storage/record/completeness` | `sp record completeness` | investigation |
| POST | `/api/storage/record/save` | — | — |
| \* | `/api/storage/replay/query/*` | `sp replay case list` (partial) | investigation |
| GET | `/api/storage/replay-mock-tree/{replayId}` | `sp replay mock-tree` | investigation |

## Logs

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/record-logs/overview` | `sp record logs overview` | investigation |
| GET | `/api/record-logs/download` | `sp record logs download` | investigation |
| GET | `/api/replay-logs/overview` | `sp replay logs overview` | investigation |
| GET | `/api/replay-logs/download` | `sp replay logs download` | investigation |

## Reports (subset)

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/report/queryDiffMsgById/{id}` | `sp replay diff get` | investigation |
| POST | `/api/report/queryReplayCase` | `sp replay case list` | investigation |
| POST | `/api/report/init` | `sp replay report init` | investigation |
| \* | `/api/report/*` (others) | partial / future | investigation |

## Agent config

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/config/agent/load` | `sp config agent load` | investigation |
| POST | `/api/config/agent/agentStatus` | — or `sp app status` | investigation |

## Legacy config

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET/POST | `/api/config/{resource}/*` | `sp config legacy …` | investigation |

## User groups

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| \* | `/api/userGroup/*` | `sp group …` | investigation |
| \* | `/api/appGrant/*` | `sp grant …` | investigation |

## Health & ops

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/vi/health` | `sp health` | platform |
| GET | `/vi/storage/*` | `sp ops storage …` | investigation |
| GET | `/vi/schedule/monitor` | `sp ops schedule monitor` | investigation |

## Not exposed

| Method | Path | Reason |
|--------|------|--------|
| POST | `/v1/traces` | OTLP |
| POST | `/api/storage/record/save` | Agent write |
| \* | `/api/replay/local/*` | Local runner |

Controller source: `sp-tr-api/sp-web-api`, `sp-storage/sp-storage-web-api`, `sp-replay-schedule/sp-schedule-web-api`.

---

---
url: /en/testing/reference/log-correlation-ids.md
---
# Log correlation IDs

When a replay fails, you need a **`trace_id`** and a bounded time window to pull correlated logs from the application under test, the Java agent, and sp-backend in one query. This page explains what each id means, **where to find it**, and **how to triage** unified log results.

Requires the unified log pipeline (Vector → Parquet) enabled in your deployment or local compose stack. If the pipeline is disabled, log lookups fail fast with a clear error — they do not fall back to other log stores.

**Command reference:** [sp logs](/en/testing/commands/logs) · **API:** `GET /api/recorder/logs?trace_id=…&since=…&until=…`

***

## ID quick reference

| ID | v1 log lookup? | What it identifies | Use for log search |
|----|----------------|--------------------|--------------------|
| **`traceId`** | **Yes** (`--trace-id`) | W3C OpenTelemetry trace for one request flow | **Only v1 lookup key** — all agent/app/backend lines on that trace in `[since, until)` |
| **`replayId`** | No | One replay **attempt** of a recorded case | Find failed case + diff; copy **`traceId`** from the same row for logs |
| **`planId`** | No | A replay **plan** (batch from `sp replay run`) | Scope diagnose / case list; copy per-case **`traceId`** for logs |
| **`planItemId`** | No | One **case/operation** inside a plan | Same — use with case list, not as log key |
| **`diffId`** | No | Compare/diff result row | Use with `sp replay diff get`, not for logs |

Each log row carries **`source`**: `agent` (Java agent diagnostics), `app` (application-under-test logs captured by the agent), or `backend` (sp-backend diagnostics). API/Parquet use unprefixed column names (`source`, `replay_id`, …). OTLP wire format may use `sp.*` keys before Vector normalization.

**Not in v1 log query results:** `sessionId` / `sp.session_id`.

**Rejected in v1:** `--replay-id`, `--plan-id`, `--plan-item-id`, `--include-recording-log`, `GET /api/record-logs/*`, `GET /api/replay-logs/*`.

***

## Where to find `traceId` (primary for logs)

| Source | How |
|--------|-----|
| Failed replay case list | `sp replay case list --plan <planId> --failed --json` → `data.items[].traceId` |
| Replay metadata | `sp replay metadata <replayId> --json` → `traceId` |
| Diagnose workflow | `sp diagnose replay <planId> --failed-only --json` → then case list for `traceId` |
| CI / `make e2e` | **Softprobe correlation** block under pytest failure → field `trace_id` per case |
| Recorded cases | `sp record case list --app <appId> --since -24h --json` → entry case trace ids |

**Important:** For replay failures, use the **`traceId` on the replay case** (same value as the recorded trace — schedule puts it on `traceparent`). Do **not** guess from the newest recording list page or health-check traffic (`/index.html`, `/`) — those traces are unrelated noise.

`replayId`, `planId`, and `planItemId` in pytest output are for human reference and diff/diagnose commands only — not v1 log lookup keys.

***

## Query correlated logs

Every lookup requires **`trace_id`** plus **`since`** and **`until`**. Use **ISO-8601 UTC** (for example `2026-06-27T10:00:00Z`). `since` is **inclusive**; `until` is **exclusive** (`[since, until)`).

### CLI (primary)

```bash
sp logs --trace-id <traceId> --since <start> --until <end> [--json]
```

Add **`--json`** for scripts and AI agents. Human-readable text is the default.

### HTTP API (when `sp` is not in PATH)

```bash
export SP_API_URL="${SP_API_URL:-http://127.0.0.1:18090}"
TRACE_ID="2057ad46a7ce03d3955385f2a4142d29"
SINCE="2026-06-27T10:00:00Z"
UNTIL="2026-06-27T10:05:00Z"

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${SINCE}&until=${UNTIL}" \
  -H "Accept: application/json" -o /tmp/trace-logs.json
```

### Picking `since` / `until`

**When you have case timestamps (`recordTime` and `replayTime`):** run **two** ±2 minute lookups — one around each anchor — and merge rows. Never bridge record time to replay time in a single query. See [sp logs — Case-scoped lookup](/en/testing/commands/logs#case-scoped-lookup-dual-windows).

**Otherwise:**

1. Start with a window around the failure (for example five minutes before plan finish through one minute after).
2. Widen the window if row counts show zeros for a `source` you expect (`agent`, `app`, `backend`).
3. E2E after `make compose-parquet-clean`: rerun the session, then poll up to ~180s for Vector ingest.
4. v1 returns **all** matching rows in the window — redirect or pipe locally for large output:

```bash
sp logs --trace-id <id> --since … --until … > /tmp/trace.log
grep ERROR /tmp/trace.log | head -20
```

There is no `--limit` on v1 log lookups.

***

## Triage unified log results

After fetching logs, follow this order.

**`sp --json logs`** wraps the API body in `.data` — use `jq '.data.rows'`, `jq '.data.warnings'`. **`curl`** saves the API JSON directly — use `jq '.rows'`, `jq '.warnings'`.

```bash
# After sp --json logs (CLI envelope)
jq '.data.rows | length' /tmp/trace-logs.json
jq '[.data.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/trace-logs.json
jq '.data.warnings' /tmp/trace-logs.json
jq -r '.data.rows[] | select(.source=="backend" and .severity=="ERROR") | "\(.timestamp) \(.body)"' /tmp/trace-logs.json | head -20

# After curl GET /api/recorder/logs (API body at top level)
jq '.rows | length' /tmp/trace-logs.json
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/trace-logs.json
jq '.warnings' /tmp/trace-logs.json
jq -r '.rows[] | select(.source=="backend" and .severity=="ERROR") | "\(.timestamp) \(.body)"' /tmp/trace-logs.json | head -20
```

### Symptom guide

| What you see | Likely meaning | Next step |
|--------------|----------------|-----------|
| `rows` empty + `warnings` non-empty | Parquet reader/schema mismatch (e.g. stale backend image) | Rebuild sp-backend; confirm API rows use `source` not `sp.source` |
| `rows` empty + `warnings` empty | Wrong `trace_id`, narrow window, or ingest lag | Use replay case `traceId`; widen `[since, until)`; rerun replay |
| Rows from `agent`, `app`, and `backend` | Pipeline healthy for that trace | Read `body` for ERROR/WARN; use `sp diagnose` diffs for compare failures |
| Only `backend`, no `app`/`agent` | Agent export or app logging quiet | Check agent attach, `sp.enable.debug`, app logger levels |

Recording-phase and replay-phase lines for the **same business request** share one **`trace_id`** on replay (recorded trace reused on `traceparent`). A single trace lookup can include both without `--include-recording-log` (removed in v1).

***

## Reading the response

**HTTP API** top-level fields:

* `lookup` — type `trace`, value, and caller `[since, until)` bounds
* `rows[]` — each row: `timestamp`, `severity`, `body`, `service_name`, `source`, and optional `trace_id`, `span_id`, `replay_id`, `plan_id`, `plan_item_id`
* `warnings` — non-fatal schema-skip or similar (may be empty)

**`sp --json logs`** returns a CLI envelope: `{"ok":true,"command":"logs","data":{...}}` — use `.data.rows` and `.data.warnings` in scripts.

v1 responses do **not** include `source_summary`. Compute per-source counts locally with `jq` (see above).

See [Log query fields](/en/testing/commands/log-query-fields) for the full field reference.

***

## Pytest / `make e2e` failures

On replay-related test failures, pytest prints:

1. **Softprobe correlation** — `trace_id` (use for log query), plus `replay_id`, `plan_id`, `plan_item_id` for reference.
2. **Unified logs** — optional summary: row count and per-`source` counts when the hook ran curl.

Copy `trace_id` and the suggested `since`/`until` from the block, then run the triage commands above before diving into application code.

***

## Typical failure workflow

```text
1. sp replay case list --plan <planId> --failed --json
      → copy traceId (and replayId for diff/diagnose)

2. sp logs --trace-id <traceId> --since … --until … [--json]
      → triage: count → sources → warnings → read backend/agent/app bodies
      (curl GET /api/recorder/logs only when sp is unavailable)

3. sp diagnose replay <planId> --failed-only --out-dir .sp-work --json
      → read diff artifacts for field-level compare failures

4. If logs empty after clean Parquet: rerun replay/e2e and poll ingest
```

***

## API equivalent

```http
GET /api/recorder/logs?trace_id=<id>&since=<ts>&until=<ts>
```

Unsupported query parameters (`replay_id`, `plan_id`, `plan_item_id`, `include_recording_log`, …) are rejected before Parquet reads.

See [sp logs](/en/testing/commands/logs) for validation rules and JSON shape.

***

## Related

* [Concepts — replay plans and ids](/en/testing/agents/concepts#trace-replay-and-plan-ids)
* [Diagnose replay failure example](/en/testing/examples/agent-diagnose-replay)
* [sp replay case](/en/testing/commands/replay-case)
* [sp trace](/en/testing/commands/trace)

---

---
url: /en/testing/reference/replay-send-log-markers.md
---
# Replay send log markers

**When agents use this:** Interpret sp-backend schedule logs around each replay HTTP dispatch — the critical entry and exit boundary between replay scheduling and your application.

These structured lines are emitted by **sp-schedule** (`DefaultHttpReplaySender`) immediately before and after each entry HTTP call. They appear in correlated log lookups (`sp logs --trace-id …`, unified trace log downloads) when `sp.source` is **`backend`** and `service_name` is typically **`sp-backend`**.

Query logs with the case **`traceId`** (same value as `recordedTraceId` in the message when correlation succeeded). See [sp logs command](../commands/logs.md).

***

## Message prefixes

| Prefix | Severity | Meaning |
|--------|----------|---------|
| `Replay send start:` | INFO | Schedule is about to invoke the recorded entry HTTP request |
| `Replay send done:` | INFO | HTTP exchange completed; response status and duration are recorded |
| `Replay send failed:` | WARN | HTTP exchange failed before a normal completion |

Each prefix is followed by comma-separated **`key=value`** pairs (SLF4J structured message, not JSON).

***

## Structured fields

| Field | On | Description |
|-------|-----|-------------|
| `planId` | start, done, failed | Replay **plan** id for the batch run |
| `targetEnv` | start, done, failed | `true` when sending to the configured replay test URL (`--env` / `targetEnv`); `false` for record-side sends |
| `method` | start, done, failed | HTTP method (`GET`, `POST`, …) |
| `url` | start, done, failed | Full URL schedule invoked (path and query string included) |
| `recordedTraceId` | start, done, failed | W3C trace id from the **recording**; should match `trace_id` on the log row when correlation succeeded |
| `spanId` | start, done, failed | OpenTelemetry span id active on the send path |
| `httpStatus` | done, failed | HTTP status code from the response, or **`-1`** when no response was received (connection error, timeout, DNS failure) |
| `durationMs` | done, failed | Wall-clock milliseconds for the HTTP exchange |
| `error` | failed only | Exception or error message when the send did not complete successfully |

***

## Example lines

```text
Replay send start: planId=6a3f2aad59f0c4655b0f99da, targetEnv=true, method=POST, url=http://order-service.test:8080/api/orders, recordedTraceId=2057ad46a7ce03d3955385f2a4142d29, spanId=8d10c94a2a6f4e11

Replay send done: planId=6a3f2aad59f0c4655b0f99da, targetEnv=true, method=POST, url=http://order-service.test:8080/api/orders, recordedTraceId=2057ad46a7ce03d3955385f2a4142d29, spanId=8d10c94a2a6f4e11, httpStatus=200, durationMs=142

Replay send failed: planId=6a3f2aad59f0c4655b0f99da, targetEnv=true, method=POST, url=http://order-service.test:8080/api/orders, recordedTraceId=2057ad46a7ce03d3955385f2a4142d29, spanId=8d10c94a2a6f4e11, httpStatus=-1, durationMs=30001, error=Connection refused
```

***

## How to use them

1. Obtain **`traceId`** from `sp replay case list --plan <planId> --failed --json` or replay metadata.
2. Pull correlated logs for a time window around the failure (see [Diagnose replay failure example](../examples/agent-diagnose-replay.md)).
3. Filter **`sp.source=backend`** rows whose **`body`** contains `Replay send start`, `Replay send done`, or `Replay send failed`.

| Observation | Likely cause |
|-------------|--------------|
| No `Replay send start` in the window | Case may not have reached HTTP send (preload, schedule, or plan cancellation) — check plan status first |
| `start` without matching `done` or `failed` | Send may still be in flight, or logs were truncated — widen the time window |
| `failed` with `httpStatus=-1` | Network or timeout reaching **`targetEnv`** — verify `--env` URL, DNS, firewall, and that the app is running |
| `failed` with 4xx/5xx | Application returned an error HTTP status — inspect agent and application logs after the `start` timestamp |
| `done` with 2xx but compare still fails | Entry request reached the app; investigate agent mock/compare behavior after the `done` line |

***

## Related ERROR lines (same send attempt)

These are **not** the entry/exit markers but often appear on the same failed send:

| Message prefix | Severity | Meaning |
|----------------|----------|---------|
| `Replay send error:` | ERROR | Stack trace after a failed send (follows `Replay send failed`) |
| `Replay send result invalid:` | ERROR | Response received but missing expected trace/replay headers for result extraction |

***

## Related

* [Replay and diff](/en/testing/replay-and-diff)
* [Diagnose replay failure example](../examples/agent-diagnose-replay.md)
* [sp logs command](../commands/logs.md)

---

---
url: /zh/testing/reference/api-mapping.md
---
# API mapping

Authoritative **REST ↔ CLI** reference for sp-backend (default port **8090**). Header `access-token` required on console APIs unless noted.

**Legend:** Category `platform` | `investigation` | `admin` | `—` (not exposed)

## Auth

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/login/getVerificationCode/{userName}` | (precursor to login) | platform |
| POST | `/api/login/verify` | `sp auth login` | platform |
| GET | `/api/login/refresh/{userName}` | `sp auth refresh` | platform |
| POST | `/api/login/loginAsGuest` | `sp auth login --guest` | platform |
| POST | `/api/login/oauthLogin` | — | — |
| GET | `/api/login/userProfile/{userName}` | `sp auth whoami` (future) | investigation |

## Applications

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/applications/list` | `sp app list` | platform |
| POST | `/api/applications/create` | `sp app create` | platform |
| GET | `/api/applications/{appId}/agent-status` | `sp app status` | platform |
| GET | `/api/applications/{appId}/replays/recent` | `sp app replays` | platform |
| GET | `/api/applications/{appId}/extraction-rules` | `sp extraction-rule list` | investigation |
| PUT | `/api/applications/{appId}/extraction-rules` | `sp extraction-rule apply` | investigation |
| POST | `/api/extraction-rules/preview` | `sp extraction-rule preview` | investigation |
| GET | `/api/agent/extraction-rules` | — (agent) | — |

## Policies

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/recording-policies/policies` | `sp policy recording list` | platform |
| GET | `/api/recording-policies/policies/{id}` | `sp policy recording get` | platform |
| POST | `/api/recording-policies/policies` | `sp policy recording apply` | platform |
| POST | `/api/recording-policies/policies/yaml` | `sp policy recording apply -f` | platform |
| DELETE | `/api/recording-policies/policies/{id}` | `sp policy recording delete` | platform |
| POST | `/api/recording-policies/policies/validate` | `sp policy recording validate` | platform |
| GET | `/api/recording-policies/policies/{id}/yaml` | `sp policy recording export` | platform |
| \* | `/api/mock-policies/*` | `sp policy mock …` | platform |
| \* | `/api/compare-rules/*` | `sp policy compare …` | platform |

## Replay control (schedule)

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/createPlan` | `sp replay run` (`targetEnv` = service base URL) | platform |
| GET | `/api/createPlan` | — (webhook only) | — |
| GET | `/api/progress` | `sp replay status` | platform |
| GET | `/api/stopPlan` | `sp replay stop` | platform |
| POST | `/api/reRunPlan` | `sp replay rerun` | platform |
| POST | `/api/createRealTimePlan` | `sp replay realtime create` | investigation |
| GET | `/api/queue/control/*` | `sp replay realtime queue …` | investigation |
| POST | `/api/compareCase` | `sp replay compare` | investigation |
| GET | `/api/queryNoise` | `sp replay noise query` | investigation |
| POST | `/api/excludeNoise` | `sp replay noise exclude` | investigation |

## Traces

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/traces/find-by-attr` | `sp trace find` | investigation |
| GET | `/api/traces/{traceId}/summary` | `sp trace get` | investigation |
| POST | `/api/traces/attr-rule-statistics` | `sp trace stats` | investigation |

## Storage record (read)

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/storage/record/query` | `sp record query` | investigation |
| GET | `/api/storage/record/trace/{traceId}` | `sp record trace` | investigation |
| GET | `/api/storage/record/completeness` | `sp record completeness` | investigation |
| POST | `/api/storage/record/save` | — | — |
| \* | `/api/storage/replay/query/*` | `sp replay case list` (partial) | investigation |
| GET | `/api/storage/replay-mock-tree/{replayId}` | `sp replay mock-tree` | investigation |

## Logs

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/record-logs/overview` | `sp record logs overview` | investigation |
| GET | `/api/record-logs/download` | `sp record logs download` | investigation |
| GET | `/api/replay-logs/overview` | `sp replay logs overview` | investigation |
| GET | `/api/replay-logs/download` | `sp replay logs download` | investigation |

## Reports (subset)

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/api/report/queryDiffMsgById/{id}` | `sp replay diff get` | investigation |
| POST | `/api/report/queryReplayCase` | `sp replay case list` | investigation |
| POST | `/api/report/init` | `sp replay report init` | investigation |
| \* | `/api/report/*` (others) | partial / future | investigation |

## Agent config

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| POST | `/api/config/agent/load` | `sp config agent load` | investigation |
| POST | `/api/config/agent/agentStatus` | — or `sp app status` | investigation |

## Legacy config

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET/POST | `/api/config/{resource}/*` | `sp config legacy …` | investigation |

## User groups

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| \* | `/api/userGroup/*` | `sp group …` | investigation |
| \* | `/api/appGrant/*` | `sp grant …` | investigation |

## Health & ops

| Method | Path | CLI | Category |
|--------|------|-----|-------|
| GET | `/vi/health` | `sp health` | platform |
| GET | `/vi/storage/*` | `sp ops storage …` | investigation |
| GET | `/vi/schedule/monitor` | `sp ops schedule monitor` | investigation |

## Not exposed

| Method | Path | Reason |
|--------|------|--------|
| POST | `/v1/traces` | OTLP |
| POST | `/api/storage/record/save` | Agent write |
| \* | `/api/replay/local/*` | Local runner |

Controller source: `sp-tr-api/sp-web-api`, `sp-storage/sp-storage-web-api`, `sp-replay-schedule/sp-schedule-web-api`.

---

---
url: /zh/testing/agents/authentication.md
---
# Authentication

Console APIs require the HTTP header:

```http
access-token: <JWT>
```

(Defined as `Constants.ACCESS_TOKEN` in sp-tr-api.)

## Non-interactive login (agents + CI)

### Email verification flow

1. Request code (human step or separate automation):

   ```http
   GET /api/login/getVerificationCode/{userName}
   ```

2. Exchange code for token:

   ```bash
   sp auth login --email user@corp.com --code 123456 --json
   ```

   **REST:** `POST /api/login/verify` with body `{ "userName", "verifyCode", ... }`

3. CLI writes token to the shared config file in
   `${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc` unless `--no-save`.

### CI / agent hosts

**CLI**（录制、回放、应用管理）使用你的用户 JWT：

```bash
export SP_TOKEN="eyJ..."
sp app list --json
```

**Java agent** 在 Softprobe Cloud 上使用**租户 API key**（长期有效，作用域为该组织）：

```bash
export SP_TENANT_API_KEY="…"   # 来自 sp tenant key ensure 或仪表盘 Settings
export SP_TENANT_ID="35"
sp agent command --app <appId> --json
```

切勿将 token 或 API key 提交到 git。泄露后立即轮换。

### Token refresh

```bash
sp auth refresh --user user@corp.com --json
```

**REST:** `GET /api/login/refresh/{userName}`

Add `--no-save` when an agent or CI job should receive a token without touching
local config:

```bash
export SP_TOKEN="$(sp auth refresh --user user@corp.com --no-save --json | jq -r .data.token)"
```

## Guest login

If enabled on the server:

```bash
sp auth login --guest --json
```

**REST:** `POST /api/login/loginAsGuest`

## OAuth

OAuth flows are browser-based. Agents should use pre-provisioned `SP_TOKEN` rather than driving OAuth interactively.

Document for humans: `GET /api/login/oauthInfo/{oauthType}`, `POST /api/login/oauthLogin`.

## Who am I

```bash
sp auth whoami --json
```

Decodes JWT `userName` or calls profile endpoint when implemented.

## Errors

| Situation | Exit code |
|-----------|-----------|
| Missing token with `--json` | `3` (`AUTH_REQUIRED`) |
| Expired or invalid token | `1` (`API_ERROR`) |

## Related

* [config](/zh/testing/installation/configuration)
* [auth command](/zh/testing/commands/auth)

---

---
url: /zh/testing/commands.md
---
# Commands

Reference for public `sp` subcommands. Use `--json` on API-backed commands. See [Output contract](/zh/testing/agents/output-contract).

## Lifecycle (recommended)

Job-oriented commands that follow record-and-replay order:

| Command | Synopsis |
|---------|----------|
| [demo](./demo) | `start`、`traffic`、`replay`、`status`、`stop` —— [Travel OTA demo](https://github.com/softprobe/demo-ota) 一体化演示栈 |
| [setup](./setup) | 配置自托管后端 URL；可选 Spcode Service（Linux） |
| [agent](./agent) | `download`, `command` — install jar and JVM flags |
| [record](./record) | `case list` — recorded entry cases before replay |
| [diagnose](./diagnose) | `replay`, `trace` — bundled investigation workflows |

## Platform

Connect, authenticate, manage apps and policies, run replay plans.

| Command | Synopsis |
|---------|----------|
| [config](./config) | Profiles, URL, init |
| [auth](./auth) | Login, whoami, refresh |
| [app](./app) | List, create, agent status, recent replays |
| [policy](./policy) | Recording, mock, compare YAML policies |
| [replay](./replay) | Run, status, stop, rerun plans |
| [health](./health) | Cluster health |
| `version` | CLI version string |

## Investigation

Recorded data, traces, and replay failures.

| Command | Synopsis |
|---------|----------|
| [record](./record) | Query recordings, completeness |
| [logs](./logs) | 按 `trace_id` 关联日志 —— 见[日志关联 ID](/zh/testing/reference/log-correlation-ids) |
| [trace](./trace) | Find traces by business attributes |
| [replay case](./replay-case) | List cases, metadata, mock tree |
| [replay diff](./replay-diff) | Diff artifacts, compare results |
| [extraction-rule](./extraction-rule) | Business attribute extraction rules |

Investigation commands support `--out-dir`, `--page`, and `--limit` unless noted.

## Administration

Groups, system config, diagnostics, legacy APIs.

| Command | Synopsis |
|---------|----------|
| [group](./group) | User groups and app grants |
| [grant](./group) | App grant listing (`grant list`) |
| [system](./system) | System config keys |
| [ops](./ops) | Storage and schedule diagnostics |
| [config legacy](./config-legacy) | Legacy `/api/config/*` (deprecated) |

## Global flags

```text
--json          Machine-readable stdout (required for agents)
--profile       Config profile name
--api-url       Override backend URL
--token         Override JWT (else SP_TOKEN / config)
--config        Extra config file (JSONC overlay)
--quiet         Suppress non-error stderr
--out-dir       Directory for artifact files
--page          Page number (investigation list commands)
--limit         Page size / max items
```

## Cheat sheet

```bash
sp config init && sp auth login --email u@c.com --code 123456 --json
sp doctor --json
sp app create my-svc --json
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
sp agent command --app <appId> --agent-jar ./sp-agent.jar --json   # copy startCommand into your run script
sp record case list --app <appId> --since -1h --json
sp replay run --app <appId> --env http://your-service:8080 --from -24h --json
sp replay status <planId> --watch --json
sp diagnose replay <planId> --failed-only --out-dir .sp-work --json
```

## Related

* [Quickstart](/zh/testing/getting-started)
* [For AI agents](/zh/testing/agents/overview)
* [Examples](/zh/testing/examples/agent-diagnose-replay)
* [API mapping](/zh/testing/reference/api-mapping)

---

---
url: /zh/testing/agents/concepts.md
---
# Concepts

Java 录制回放产品说明（Agent、策略、回放语义）见 [Softprobe 测试](/zh/testing/)。

Istio/Envoy 网格采集与 SESSIFY 会话上下文（与 Java Agent 不同）见[平台核心概念](/zh/platform/advanced-guides/concepts)。

## Application (`appId`)

A registered service under test. Recording, replay, policies, and extraction rules are all scoped by **`appId`**.

| Field | Role |
|-------|------|
| `appName` | Unique label supplied at registration (`sp app create <appName>`). |
| `appId` | System-generated id (16-character hex). Configure the Java agent and CLI with this value. |

After registration, save `data.appId` from the create response. Attach the SoftProbe Java agent to your JVM with that id and your sp-backend URL, then confirm connectivity with `sp app status <appId>` or `sp app list --json`.

**Agent status** (`online`, `offline`, `never`) is derived from instance heartbeats, not from the app document alone. The server marks an app `offline` when the freshest heartbeat is older than the configured threshold (default 60 seconds).

**CLI reference:** [sp app](/zh/testing/commands/app)

## Java agent

The SoftProbe Java agent is attached to the service under test with `-javaagent:/path/to/sp-agent.jar`. It operates like an observability agent operationally, but its purpose is test data capture and replay:

* During **recording**, it observes real requests and dependency interactions and uploads mocker data keyed by `appId` and trace/case ids.
* During **replay**, it restores recorded dependency behavior according to mock policy and emits replay data for comparison.
* It reports heartbeat/status so `sp app status <appId>` can tell whether an app is `online`, `offline`, or `never`.

Minimum startup flags:

```bash
java \
  -javaagent:/opt/softprobe/sp-agent.jar \
  -Dsp.app.id=<appId> \
  -Dsp.api.url=http://<sp-backend-host>:8090 \
  -jar app.jar
```

Pin `sp.app.id` for every production-like deployment. If the id changes between recording and replay, SoftProbe cannot reliably find the original cases or mock data.

## Replay target URL (`targetEnv`)

Replay does not use a symbolic environment label such as `staging` or `prod`. The schedule service field **`targetEnv`** is the **base URL of the running service** that will receive replayed HTTP traffic.

When you run `sp replay run --env <url>`, the CLI sends that value as `targetEnv` on `POST /api/createPlan`. The schedule module parses it as a URI (`DefaultDeploymentEnvironmentProviderImpl`) and builds a `ServiceInstance` whose `url` is that string. Replay senders then issue requests to that URL (for example `DefaultHttpReplaySender` uses `instanceRunner.getUrl()`).

Requirements:

* Use a reachable base URL for the app under test, including scheme and host (and port when not default), for example `http://travel-ota:8080` or `https://order-service.internal:8443`.
* The URL must parse as a URI with a non-empty host; otherwise plan validation fails with *requested target env unable load active instance*.
* This is independent of **`SP_API_URL`** / `api_url` in CLI config, which points at sp-backend (storage, report, schedule APIs), not at the service being replayed.

Optional **`sourceEnv`** on the same request is a separate URI used only when you need a non-default source deployment; the demo stack often leaves it as `pro`.

**CLI reference:** [sp replay](/zh/testing/commands/replay) (`--env` → `targetEnv`)

## Recording policy

Declarative YAML (`kind: RecordingPolicy`) controlling what the agent records: sampling, operation include/exclude, time windows, sensitive-field scrubbing.

* Managed via `sp policy recording`
* Schema: [策略 YAML 指南](/zh/testing/policy-yaml-guide)

## Mock policy

Declarative YAML (`kind: MockPolicy`) controlling replay-time mocking: skip/force mock, tolerance, dependencies.

* `sp policy mock`

## Compare rules

Declarative YAML (`kind: CompareRulePolicy`) controlling diff behavior during replay comparison.

* `sp policy compare`

Policies merge by `metadata.priority`; global defaults ship in `sp-policy-rules` JAR resources.

## Replay plan

A batch replay job with a `planId`. Created by `sp replay run`, tracked with `sp replay status`. Replay plans consume cases already recorded by an instrumented app; a fresh app with no recorded traffic has nothing meaningful to replay.

Schedule service endpoints: `/api/createPlan`, `/api/progress`, `/api/stopPlan`.

## Trace、replay 与 plan ID {#trace-replay-and-plan-ids}

平台 ID 把录制、回放、diff 与**关联日志检索**串联起来。完整参考——每个 ID 的含义、在哪里获取、以及如何分诊统一日志——见 **[日志关联 ID](/zh/testing/reference/log-correlation-ids)**。

| ID | 含义 | 日志查询（v1） |
|----|------|----------------|
| `traceId` | 一次录制或回放请求流的 W3C trace id | **`sp logs --trace-id …`** 或 `GET /api/recorder/logs?trace_id=…` —— **唯一的 v1 键** |
| `replayId` | 一个 case 的一次回放**尝试** | 仅用于 diff/diagnose —— 查日志请从同一 case 行复制 **`traceId`** |
| `planId` | `sp replay run` 产生的回放计划容器 | case 列表 / diagnose —— 查日志用每个 case 的 **`traceId`** |
| `planItemId` | 计划内的操作级条目 | 同上 —— 不是日志查询键 |
| `diffId` | 用于深度 diff 拉取的比对结果行 | 用 `sp replay diff get` —— 不是日志查询键 |

**ID 在 CLI 输出中的位置**

| 命令 | 字段 |
|------|------|
| `sp replay run --json` | `planId` |
| `sp replay case list --plan … --json` | `replayId`、`traceId`、plan item id |
| `sp replay metadata <replayId> --json` | `traceId`、关联的录制元数据 |
| `sp trace find … --json` | 解析业务属性时的 `traceId` |
| `sp diagnose replay <planId> --json` | 带 id 的失败 case，供后续跟进 |

当用户提供业务属性（orderId、caseId）而非 trace id 时，Agent 应通过 `sp trace find` 获取 `traceId`。回放失败后做日志诊断时，使用失败回放 case 中的 **`traceId`** 或 e2e **Softprobe correlation** 块（`trace_id` 字段）——而不是把 `replayId` 当作日志查询键。

## Historical coupling: schedule ↔ recording

Replay operation include/exclude lists on schedule configuration are **populated from recording policy at read time**, not stored independently on the schedule document.

Implications:

* Changing recording policy can change which operations appear in replay scope without editing schedule config.
* Diagnosis skills must not assume schedule Mongo documents are the sole source of operation filters.

See server comments on `ScheduleConfigurableHandler` in sp-tr-api.

Test cases are created only via **recording** (instrumented app traffic). Manual case authoring is not part of the CLI workflow.

## Related

* [For agents](./overview)
* [Commands](/zh/testing/commands/)

---

---
url: /en/testing/commands/demo.md
---
# demo

Run the bundled [Travel OTA](https://github.com/softprobe/demo-ota) demo locally with Docker.

## Commands

| Command | Description |
|---------|-------------|
| `sp demo start [--watch]` | Create app, apply policies, start Docker stack |
| `sp demo traffic` | Send sample booking traffic |
| `sp demo replay [--watch]` | Create replay plan against localhost OTA |
| `sp demo status` | Stack + agent + recording status |
| `sp demo stop` | Stop Docker stack |

See [Getting Started](/en/testing/getting-started) for the full walkthrough.

---

---
url: /zh/testing/commands/demo.md
---
# demo

> 本页翻译可能滞后于英文版，如有出入以[英文版](/en/testing/commands/demo)为准。

使用 Docker 在本地运行内置的 [Travel OTA](https://github.com/softprobe/demo-ota) 演示。

## Commands

| Command | Description |
|---------|-------------|
| `sp demo start [--watch]` | 创建应用、应用策略并启动 Docker 栈 |
| `sp demo traffic` | 发送示例预订流量 |
| `sp demo replay [--watch]` | 针对 localhost OTA 创建回放计划 |
| `sp demo status` | 栈、agent 及录制状态 |
| `sp demo stop` | 停止 Docker 栈 |

完整操作流程参见[快速开始](/zh/testing/getting-started)。

---

---
url: /zh/testing/installation/doctor.md
---

# Doctor

使用 `sp doctor` 检查自托管 Softprobe 安装状态：

```bash
sp doctor
sp doctor --json
```

检查项包括后端可达性和内部编码引擎安装状态。

若已通过 [`sp setup --install-spcode-service`](/zh/testing/installation/#spcode-service) 安装 **Spcode Service**（`spcode-web.service` 已注册），`sp doctor` 还会检查服务是否健康，以及能否访问已配置的 Softprobe 后端。

---

---
url: /en/testing/examples/agent-attr-trace-lookup.md
---
# Example: Business ID → trace

**Audience:** AI agents

**Goal:** User says "order ORD-1234 failed in replay" without providing `traceId`.

## Steps

```bash
sp app list --json
sp trace find \
  --app my-app \
  --attr-name orderId \
  --attr-value ORD-1234 \
  --json
```

If multiple traces:

```bash
sp trace get <traceId> --json
```

Pick the trace matching time window or endpoint.

## Continue diagnosis

```bash
sp record query --trace-id <traceId> --out-dir .sp-work --json
sp record completeness <traceId> --json
```

If a replay already exists:

```bash
sp replay metadata <replayId> --json
```

## Skill note

Do **not** ask the user to open the UI to find traceId when extraction rules index their business attributes. Prefer `sp trace find` first.

## Related

* [trace](/en/testing/commands/trace)
* [Diagnose replay failure](./agent-diagnose-replay)

---

---
url: /zh/testing/examples/agent-attr-trace-lookup.md
---
# Example: Business ID → trace

**Audience:** AI agents

**Goal:** User says "order ORD-1234 failed in replay" without providing `traceId`.

## Steps

```bash
sp app list --json
sp trace find \
  --app my-app \
  --attr-name orderId \
  --attr-value ORD-1234 \
  --json
```

If multiple traces:

```bash
sp trace get <traceId> --json
```

Pick the trace matching time window or endpoint.

## Continue diagnosis

```bash
sp record query --trace-id <traceId> --out-dir .sp-work --json
sp record completeness <traceId> --json
```

If a replay already exists:

```bash
sp replay metadata <replayId> --json
```

## Skill note

Do **not** ask the user to open the UI to find traceId when extraction rules index their business attributes. Prefer `sp trace find` first.

## Related

* [trace](/zh/testing/commands/trace)
* [Diagnose replay failure](./agent-diagnose-replay)

---

---
url: /en/testing/examples/ci-policy-gate.md
---
# Example: CI policy gate

**Audience:** GitHub Actions / GitLab CI (agents may generate this YAML)

Validate policy files on every pull request; apply only on merge to main.

## GitHub Actions

```yaml
jobs:
  policy-validate:
    runs-on: ubuntu-latest
    env:
      SP_API_URL: https://sp-staging.example.com
      SP_TOKEN: ${{ secrets.SP_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - name: Install sp
        run: |
          curl -fsSL -o sp "${SP_DOWNLOAD_URL:-https://install.softprobe.ai/artifacts/sp/latest}/sp-linux-amd64"
          chmod +x sp && sudo mv sp /usr/local/bin/
      - name: Validate recording policy
        run: sp policy recording validate -f policies/recording.yaml --json
      - name: Validate mock policy
        run: sp policy mock validate -f policies/mock.yaml --json
```

Exit code non-zero fails the job.

## Apply on deploy (main only)

```yaml
      - name: Apply policies
        if: github.ref == 'refs/heads/main'
        run: |
          sp policy recording apply -f policies/recording.yaml --json
          sp policy mock apply -f policies/mock.yaml --json
```

## Agent-generated pipelines

When an agent edits policy YAML in a repo, it should:

1. Run `validate` locally against staging `SP_API_URL`
2. Commit YAML + open PR
3. Not embed `SP_TOKEN` in repo files

## Related

* [policy](/en/testing/commands/policy)
* [GitOps policies](./gitops-policies)

---

---
url: /zh/testing/examples/ci-policy-gate.md
---
# Example: CI policy gate

**Audience:** GitHub Actions / GitLab CI (agents may generate this YAML)

Validate policy files on every pull request; apply only on merge to main.

## GitHub Actions

```yaml
jobs:
  policy-validate:
    runs-on: ubuntu-latest
    env:
      SP_API_URL: https://sp-staging.example.com
      SP_TOKEN: ${{ secrets.SP_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - name: Install sp
        run: |
          curl -fsSL -o sp "${SP_DOWNLOAD_URL:-https://install.softprobe.ai/artifacts/sp/latest}/sp-linux-amd64"
          chmod +x sp && sudo mv sp /usr/local/bin/
      - name: Validate recording policy
        run: sp policy recording validate -f policies/recording.yaml --json
      - name: Validate mock policy
        run: sp policy mock validate -f policies/mock.yaml --json
```

Exit code non-zero fails the job.

## Apply on deploy (main only)

```yaml
      - name: Apply policies
        if: github.ref == 'refs/heads/main'
        run: |
          sp policy recording apply -f policies/recording.yaml --json
          sp policy mock apply -f policies/mock.yaml --json
```

## Agent-generated pipelines

When an agent edits policy YAML in a repo, it should:

1. Run `validate` locally against staging `SP_API_URL`
2. Commit YAML + open PR
3. Not embed `SP_TOKEN` in repo files

## Related

* [policy](/zh/testing/commands/policy)
* [GitOps policies](./gitops-policies)

---

---
url: /en/testing/examples/agent-diagnose-replay.md
---
# Example: Diagnose a failed replay

**Audience:** AI agent skill (OpenCode, Claude Code, Codex)

**Goal:** Given a failing replay plan, find failed cases, fetch diff artifact, triage unified logs by `trace_id`.

## Prerequisites

```bash
export SP_API_URL=http://127.0.0.1:8090
export SP_TOKEN=<jwt>
```

## One-shot (preferred)

```bash
sp diagnose replay plan-xyz --failed-only --out-dir .sp-work --json
```

Read `data.artifacts[]` paths locally. See [diagnose](/en/testing/commands/diagnose).

## Manual steps

### 1. Confirm backend and list apps

```bash
sp health --json
sp app list --json
```

Select `appId` from `data.items[].appId`.

### 2. Find the plan

If the user supplied `planId`, skip. Otherwise:

```bash
sp app replays my-app --limit 5 --json
```

### 3. List failed cases

```bash
sp replay case list --plan plan-xyz --failed --page 1 --limit 20 --json
```

From each item collect `diffId`, `replayId`, `planItemId`, and **`traceId`** (log lookup key).

### 4. Fetch diff (artifact)

```bash
sp replay diff get diff-abc --out-dir .sp-work --json
```

Parse `data.artifact` and read the JSON file in a follow-up tool call. Inspect `baseMsg` vs `testMsg`.

### 5. Optional: correlated runtime logs

Pull agent, application, and sp-backend logs for a failed case with `sp logs`, keyed by the failed case's `traceId`. Start with **backend replay send markers** — they show whether schedule reached your app:

```bash
sp logs --trace-id <traceId> --since <start> --until <end> --json
```

Filter `backend` rows for `Replay send start`, `Replay send done`, and `Replay send failed`. See [Replay send log markers](/en/testing/reference/replay-send-log-markers).

When failure may be recording-time agent behavior:

```bash
TRACE_ID="<traceId from failed case>"
SINCE="2026-06-27T10:00:00Z"
UNTIL="2026-06-27T10:05:00Z"

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${SINCE}&until=${UNTIL}" \
  -H "Accept: application/json" -o .sp-work/unified-logs.json

jq '.rows | length' .sp-work/unified-logs.json
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' .sp-work/unified-logs.json
jq '.warnings' .sp-work/unified-logs.json
jq -r '.rows[] | select(.source=="backend" and .severity=="ERROR") | .body' .sp-work/unified-logs.json | head -20
```

The `curl` above hits the unified-log API directly; the `sp logs` CLI wraps the same endpoint:

```bash
sp logs --trace-id "$TRACE_ID" --since "$SINCE" --until "$UNTIL" --json > .sp-work/unified-logs.json
```

**Triage:** empty rows + `warnings` → backend/schema skew; empty + no warnings → wrong window or ingest lag; all three `source` values → pipeline OK, focus on diff + log bodies.

See [Log correlation IDs](/en/testing/reference/log-correlation-ids) and [sp logs](/en/testing/commands/logs).

Do **not** use `sp recorder logs --replay-id`, `sp record logs`, `sp replay logs`, or legacy `/api/record-logs/*` / `/api/replay-logs/*`.

### 6. Optional: metadata for full-link

```bash
sp replay metadata <replayId> --json
```

Use `data.fullLink` to decide if `sp replay compare` needs `traceId`.

## Skill prompt snippet

```markdown
When diagnosing SoftProbe replay failures:
1. Run `sp replay case list --plan <id> --failed --json`
2. For each diffId: `sp replay diff get <id> --out-dir .sp-work --json`
3. Read the artifact file path from JSON; do not parse multi-MB stdout
4. For logs: copy `traceId` from the failed case; curl `GET /api/recorder/logs?trace_id=…&since=…&until=…`; jq row count and group by `source`; read backend then agent then app ERROR lines
5. On pytest failure: read Softprobe correlation (`trace_id`) and Unified logs summary in output
6. If traceId unknown, ask user for business ID and run `sp trace find`
```

## Related

* [Replay send log markers](/en/testing/reference/replay-send-log-markers)
* [replay-diff](/en/testing/commands/replay-diff)
* [sp logs](/en/testing/commands/logs)
* [Log correlation IDs](/en/testing/reference/log-correlation-ids)

---

---
url: /zh/testing/examples/agent-diagnose-replay.md
---
# Example: Diagnose a failed replay

**Audience:** AI agent skill (OpenCode, Claude Code, Codex)

**Goal:** Given a failing replay plan, find failed cases, fetch diff artifact, optionally inspect record logs.

## Prerequisites

```bash
export SP_API_URL=http://127.0.0.1:8090
export SP_TOKEN=<jwt>
```

## One-shot (preferred)

```bash
sp diagnose replay plan-xyz --failed-only --out-dir .sp-work --json
```

Read `data.artifacts[]` paths locally. See [diagnose](/zh/testing/commands/diagnose).

## Manual steps

### 1. Confirm backend and list apps

```bash
sp health --json
sp app list --json
```

Select `appId` from `data.items[].appId`.

### 2. Find the plan

If the user supplied `planId`, skip. Otherwise:

```bash
sp app replays my-app --limit 5 --json
```

### 3. List failed cases

```bash
sp replay case list --plan plan-xyz --failed --page 1 --limit 20 --json
```

From each item collect `diffId`, `replayId`, `planItemId`, `traceId`.

### 4. Fetch diff (artifact)

```bash
sp replay diff get diff-abc --out-dir .sp-work --json
```

Parse `data.artifact` and read the JSON file in a follow-up tool call. Inspect `baseMsg` vs `testMsg`.

### 5. Optional: correlated runtime logs

When failure may be agent-side (no recording, incomplete trace), pull the unified logs for the failed case's `traceId`:

```bash
sp logs --trace-id <traceId> --since <start> --until <end> --json
```

Filter `backend` rows for `Replay send start` / `done` / `failed`. See [Replay send log markers](/zh/testing/reference/replay-send-log-markers).

### 6. Optional: metadata for full-link

```bash
sp replay metadata <replayId> --json
```

Use `data.fullLink` to decide if `sp replay compare` needs `traceId`.

## Skill prompt snippet

```markdown
When diagnosing SoftProbe replay failures:
1. Run `sp replay case list --plan <id> --failed --json`
2. For each diffId: `sp replay diff get <id> --out-dir .sp-work --json`
3. Read the artifact file path from JSON; do not parse multi-MB stdout
4. If traceId unknown, ask user for business ID and run `sp trace find`
```

## Related

* [replay-diff](/zh/testing/commands/replay-diff)

---

---
url: /en/testing/examples/gitops-policies.md
---
# Example: GitOps policies

**Audience:** Platform engineers and agents maintaining policy repos

## Repository layout

```text
policies/
├── recording-staging.yaml
├── recording-prod.yaml
├── mock-staging.yaml
└── compare-global.yaml
```

## Export from server (bootstrap)

```bash
sp policy recording list --json
sp policy recording export <policy-id> -o policies/recording-prod.yaml
```

## Drift check (agent or CI)

```bash
sp policy recording validate -f policies/recording-prod.yaml --json
# optional v2:
# sp policy recording diff -f policies/recording-prod.yaml --against prod-id
```

## Apply after review

```bash
sp policy recording apply -f policies/recording-prod.yaml --json
```

## Multi-environment profiles

```jsonc
// ~/.config/softprobe/config.jsonc
{
  "profiles": {
    "staging": {
      "api_url": "https://sp-staging.example.com"
    },
    "prod": {
      "api_url": "https://sp.example.com"
    }
  }
}
```

```bash
sp --profile staging policy recording apply -f policies/recording-staging.yaml --json
sp --profile prod policy recording apply -f policies/recording-prod.yaml --json
```

## Related

* [Policy YAML](/en/testing/policies)
* [CI policy gate](./ci-policy-gate)

---

---
url: /zh/testing/examples/gitops-policies.md
---
# Example: GitOps policies

**Audience:** Platform engineers and agents maintaining policy repos

## Repository layout

```text
policies/
├── recording-staging.yaml
├── recording-prod.yaml
├── mock-staging.yaml
└── compare-global.yaml
```

## Export from server (bootstrap)

```bash
sp policy recording list --json
sp policy recording export <policy-id> -o policies/recording-prod.yaml
```

## Drift check (agent or CI)

```bash
sp policy recording validate -f policies/recording-prod.yaml --json
# optional v2:
# sp policy recording diff -f policies/recording-prod.yaml --against prod-id
```

## Apply after review

```bash
sp policy recording apply -f policies/recording-prod.yaml --json
```

## Multi-environment profiles

```jsonc
// ~/.config/softprobe/config.jsonc
{
  "profiles": {
    "staging": {
      "api_url": "https://sp-staging.example.com"
    },
    "prod": {
      "api_url": "https://sp.example.com"
    }
  }
}
```

```bash
sp --profile staging policy recording apply -f policies/recording-staging.yaml --json
sp --profile prod policy recording apply -f policies/recording-prod.yaml --json
```

## Related

* [Policy YAML](/zh/testing/policies)
* [CI policy gate](./ci-policy-gate)

---

---
url: /zh/testing/reference/exit-codes.md
---
# Exit codes

Contract for agents and CI scripts invoking `sp`.

| Code | Name | Meaning |
|------|------|---------|
| `0` | Success | Parsed JSON on stdout when `--json` |
| `1` | API\_ERROR | Backend returned error or HTTP non-success (includes `NO_RECORDED_CASES` on `replay run`) |
| `2` | USAGE / PROFILE\_NOT\_FOUND / CONFIG\_\* | Invalid flags, missing config, parse errors, unknown profile |
| `3` | AUTH\_REQUIRED | No token available in non-interactive mode |

## stderr with `--json`

On exit `1` or `2` or `3`, stderr contains a JSON error object:

```json
{
  "ok": false,
  "command": "app list",
  "error": {
    "code": "AUTH_REQUIRED",
    "message": "Set SP_TOKEN or run sp auth login"
  }
}
```

## Config error codes (exit `2`)

| Code | Cause |
|------|-------|
| `CONFIG_MISSING` | No `config.jsonc` / `sp.jsonc`; run `sp config init` |
| `CONFIG_PARSE_ERROR` | Invalid JSONC or schema validation failure |
| `CONFIG_WRITE_ERROR` | Could not write config file |
| `PROFILE_NOT_FOUND` | Selected profile missing from merged config |

## Agent retry guidance

| Code | Retry? |
|------|--------|
| 0 | No |
| 1 | Sometimes — after fixing data or if transient 5xx |
| 2 | No — fix invocation |
| 3 | After refresh/login |

---

---
url: /zh/testing/agents/overview.md
---
# For AI agents

This page is the primary entry point for authors of **OpenCode / spcode**, **Claude Code**, **Codex**, **Cursor**, and other agent hosts that invoke SoftProbe through shell tools.

## 把文档喂给你的 Agent

整套文档以 [llmstxt.org](https://llmstxt.org/) 格式发布，Agent 无需抓取 HTML 即可直接消费：

| URL | 内容 |
|-----|------|
| [`/llms.txt`](/llms.txt) | 带描述的全站页面索引 —— 体积小的入口 |
| [`/llms-full.txt`](/llms-full.txt) | 整套文档正文合并为单个纯文本文件 |
| `‹任意页面›.md` | 单个页面的 Markdown 源（在 URL 后追加 `.md`） |

先让 Agent 读 `/llms.txt`，它会按需跟进到全文或单页 `.md`。

## Design goals

1. **One process, one job** — each tool call runs a single `sp` command with explicit flags. Do not rely on shell aliases or interactive prompts.
2. **Always pass `--json`** for machine parsing unless you are showing output to a human in chat.
3. **Use artifacts for large payloads** — diff bodies, log downloads, and record queries write files under `--out-dir`; stdout returns paths and summaries only.
4. **Backend data only** — use API commands (`sp app`, `sp replay`, `sp record`, …) for cluster state.

## Recommended tool shape

Wrap `sp` as a shell tool with a fixed argv prefix:

```bash
sp --json --profile "${SP_PROFILE:-default}" <subcommand> ...
```

Environment variables agents should set in the host config:

| Variable | Purpose |
|----------|---------|
| `SP_API_URL` | Backend base URL (e.g. `http://127.0.0.1:8090`) |
| `SP_TOKEN` | JWT from `sp auth login` or CI secret |
| `SP_PROFILE` | Named profile in `${XDG_CONFIG_HOME}/softprobe/config.jsonc` or `sp.jsonc` |
| `SP_CONFIG` | Extra config file path loaded after `sp.jsonc` and before `--config` |

`sp` also reads `${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc` for
shared SoftProbe settings and `${XDG_CONFIG_HOME:-~/.config}/softprobe/sp.jsonc`
for CLI-specific overrides. Agents should prefer `SP_API_URL` and `SP_TOKEN`
when running in ephemeral CI containers.

## Lifecycle command order

Use these job-oriented commands before low-level building blocks:

1. `sp doctor --json`
2. `sp app create <name> --json` → save `data.appId`
3. `sp policy recording apply -f … --json`
4. 从 `install.softprobe.ai` 下载 `sp-agent.jar`，然后运行 `sp agent command --app <id> --agent-jar ./sp-agent.jar --json`
5. Start the app with `data.startCommand`; send traffic
6. `sp record case list --app <id> --since -1h --json`
7. `sp policy mock apply` / `sp policy compare apply`
8. `sp replay run --app <id> --env <service-base-url> --json`
9. `sp replay status <planId> --json` or `sp diagnose replay <planId> --json`

## Typical skill flows

### Diagnose a failed replay

```mermaid
sequenceDiagram
  participant Agent
  participant SP as sp CLI
  participant API as sp-backend

  Agent->>SP: app list --json
  SP->>API: GET /api/applications/list
  Agent->>SP: replay case list --plan X --failed --json
  SP->>API: report/storage APIs
  Agent->>SP: replay diff get diffId --out-dir .sp-work --json
  SP->>API: GET /api/report/queryDiffMsgById/{id}
  Agent->>Agent: Read artifact file locally
```

Commands: see [Diagnose replay failure](/zh/testing/examples/agent-diagnose-replay).

### Change policy and re-run

1. `sp policy recording validate -f policy.yaml --json`
2. `sp policy recording apply -f policy.yaml --json`
3. `sp replay run --app … --env <service-base-url> --json`
4. `sp replay status <planId> --watch --json`

## Migrating from `sp_api` (spcode plugin)

The OpenCode `@spcode/plugin` tool `sp_api` uses named endpoints (`diff_detail`, `query_replay_case`, …). The `sp` CLI exposes the same operations as **stable subcommands**.

| Old | New |
|-----|-----|
| `sp_api endpoint=list_applications` | `sp app list --json` |
| `sp_api endpoint=diff_detail diffId=…` | `sp replay diff get … --out-dir … --json` |
| `sp_api endpoint=find_traces_by_attr …` | `sp trace find … --json` |

Skills should be updated to call `sp` so behavior is consistent outside OpenCode.

## Error handling for agents

| Exit code | Meaning | Agent action |
|-----------|---------|--------------|
| `0` | Success | Parse stdout JSON |
| `1` | API or business error | Read stderr JSON `error` field; may retry or ask user |
| `2` | Usage / missing config / auth | Fix argv or run `sp auth login` / set `SP_TOKEN` |
| `3` | Auth required | Refresh token |

See [Output contract](./output-contract) and [Exit codes](/zh/testing/reference/exit-codes).

## What not to call

* Agent write APIs (`POST /api/storage/record/save`, batch saves) — instrumentation only
* OTLP ingestion (`/v1/traces`) — use collectors
* Mutating `sp ops` without `--confirm` — blocked by design

## Related pages

* [Output contract](./output-contract)
* [Versioning](./versioning)
* [Commands](/zh/testing/commands/)

---

---
url: /zh/platform/deployment/GKE-Autopilot-Istio-Installation-Guide.md
---
# GKE Autopilot + Istio 安装指南

本文介绍在 **Google Kubernetes Engine (GKE) Autopilot** 模式下安装 **Istio**，并部署 **Softprobe** 的完整流程及注意事项。

## 前提条件

* 已开通 GCP 账户并创建 GKE Autopilot 集群
* 已安装 `gcloud` 与 `kubectl`
* 拥有集群管理员权限

## 步骤概览

1. 安装 Istio
2. 启用自动 Sidecar 注入
3. 部署 Softprobe 组件
4. 验证与排错

## 1. 安装 Istio

使用官方安装方式或 Operator 管理：

* 参见 [Istio 官方安装文档](https://istio.io/latest/docs/setup/install/)

## 2. 启用自动 Sidecar 注入

在目标命名空间配置标签：

```bash
kubectl label namespace <your-namespace> istio-injection=enabled
```

## 3. 部署 Softprobe

在启用 Sidecar 的命名空间部署 Softprobe 相关组件，并配置公钥认证与采样策略。

## 4. 验证

* 访问服务并观察请求链路
* 在 Softprobe 仪表盘查看服务依赖图与追踪样例
* 如无数据，检查 Sidecar 注入、WASM 插件与网格流量策略

## 常见问题

* Autopilot 下资源配额限制较严格，建议按需调优采样与队列
* 需确保 Istio 版本与 SP-Istio Agent 兼容

***

更多部署细节：

* 若你未使用 GKE，请参考 [生产安装指南](./installation)

## 在 Autopilot 环境进行应用程序插桩（不使用 Operator）

在部分 GKE Autopilot 场景下，集群级 Operator（如 OpenTelemetry Operator）可能受安全策略、私有集群防火墙或 Webhook 要求限制。若无法（或不希望）使用 Operator，可在应用侧手动挂载/启用各语言的 OpenTelemetry Agent，并将数据导出到 OTLP 端点（Collector 或 Softprobe 接入端点）。

### 通用配置

* 选择 OTLP 端点（Collector 服务或外部接入地址）
* 通过环境变量设置服务元数据（service.name、namespace、version 等）
* 确保工作负载到 OTLP 端点的 egress（HTTP 或 gRPC）连通性
* 遵循 Autopilot 规范：尽量使用非 root 容器，并为所有容器设置合理的资源请求/限制

常用环境变量（按需替换为你的端点）：

```bash
# 禁用导出（仅采集，不上报）
export OTEL_TRACES_EXPORTER=none
export OTEL_METRICS_EXPORTER=none
export OTEL_LOGS_EXPORTER=none

# 如需开启上报，请改为：
# export OTEL_TRACES_EXPORTER=otlp
# export OTEL_METRICS_EXPORTER=otlp
# export OTEL_LOGS_EXPORTER=otlp
# 并配置 OTLP 端点与协议：
# export OTEL_EXPORTER_OTLP_ENDPOINT="https://otel.example.com"
# export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"  # 或 "grpc"
# export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_TOKEN"
# export OTEL_SERVICE_NAME="your-service"
# export OTEL_RESOURCE_ATTRIBUTES="service.namespace=production,service.version=1.0.0"
```

可以在 Kubernetes Deployment 的 `env:` 中直接设置以上变量。

***

### Java（JVM）

通过 `-javaagent` 挂载 OpenTelemetry Java Agent 并设置环境变量：

```dockerfile
# 建议将 Agent 直接打包进镜像
ADD opentelemetry-javaagent.jar /otel/javaagent.jar
```

```yaml
# Deployment 片段（默认仅采集，不上报）
spec:
  template:
    spec:
      containers:
        - name: app
          image: your-registry/your-java-app:latest
          env:
            - name: OTEL_TRACES_EXPORTER
              value: "none"
            - name: OTEL_METRICS_EXPORTER
              value: "none"
            - name: OTEL_LOGS_EXPORTER
              value: "none"
            - name: OTEL_RESOURCE_ATTRIBUTES
              value: "service.namespace=production,service.version=1.0.0"
            - name: OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_REQUEST_HEADERS
              value: "x-request-id,authorization"
            - name: OTEL_INSTRUMENTATION_HTTP_SERVER_CAPTURE_RESPONSE_HEADERS
              value: "content-type,content-length"
            - name: JAVA_TOOL_OPTIONS
              value: "-javaagent:/otel/javaagent.jar"
          # 如需开启上报，请将上面 three exporter 改为 otlp，并补充以下 OTLP 配置：
          # - name: OTEL_EXPORTER_OTLP_ENDPOINT
          #   value: "https://otel.example.com"
          # - name: OTEL_EXPORTER_OTLP_PROTOCOL
          #   value: "http/protobuf"  # 或 "grpc"
          # - name: OTEL_EXPORTER_OTLP_HEADERS
          #   value: "Authorization=Bearer YOUR_TOKEN"
          # - name: OTEL_SERVICE_NAME
          #   value: "your-service"
```

***

### Node.js

使用 NodeSDK + auto-instrumentations：

```bash
npm install @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node @opentelemetry/exporter-trace-otlp-http
```

创建启动前置文件（例如 `otel.js`）：

```js
// otel.js
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');

const exporter = new OTLPTraceExporter({
  // 对 HTTP 导出器，URL 会自动追加 /v1/traces
  url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
  headers: process.env.OTEL_EXPORTER_OTLP_HEADERS
    ? Object.fromEntries(process.env.OTEL_EXPORTER_OTLP_HEADERS.split(',').map(h => h.split('=')))
    : undefined,
});

const sdk = new NodeSDK({
  traceExporter: exporter,
  instrumentations: [getNodeAutoInstrumentations()],
});

sdk.start();
```

以预加载方式启动：

```bash
# 方式一：require 启动前置
node -r ./otel.js app.js
# 方式二：通过 NODE_OPTIONS
export NODE_OPTIONS="--require ./otel.js" && node app.js
```

将通用环境变量配置到 Deployment 即可。

***

### Python

使用 Python Distro 与 CLI 插桩：

```bash
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap --action=install
```

以 CLI 插桩运行：

```bash
# 先按“通用配置”设置环境变量，然后：
opentelemetry-instrument python app.py
```

也可以在代码中直接初始化 SDK 并配置 OTLP 导出。

***

### .NET

可以使用 SDK 方式进行插桩（更易落地），或启用自动插桩（Native Profiler）。下面示例为 SDK 方式：

```csharp
// Program.cs（示例）
using OpenTelemetry;
using OpenTelemetry.Trace;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddOpenTelemetry().WithTracing(tracerProviderBuilder =>
{
    tracerProviderBuilder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri(Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_ENDPOINT") ?? "https://otel.example.com");
            // 若使用 HTTP/protobuf，请确保协议匹配；需要时设置 headers
        });
});

var app = builder.Build();
app.MapGet("/", () => "Hello World!");
app.Run();
```

如需 .NET 自动插桩，请挂载自动插桩文件并在 Deployment 中设置相关 Profiler 环境变量（`CORECLR_ENABLE_PROFILING`、`CORECLR_PROFILER`、`CORECLR_PROFILER_PATH` 以及对应的 `OTEL_*` 变量）。

***

### Go

Go 通常在代码中使用 SDK 初始化：

```go
// 示意代码
import (
  "go.opentelemetry.io/otel"
  "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
  "go.opentelemetry.io/otel/sdk/resource"
  "go.opentelemetry.io/otel/sdk/trace"
)

func initTracer() (*trace.TracerProvider, error) {
  exporter, err := otlptracehttp.New(context.Background(), otlptracehttp.WithEndpoint(os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT")))
  if err != nil { return nil, err }

  tp := trace.NewTracerProvider(
    trace.WithBatcher(exporter),
    trace.WithResource(resource.Default()),
  )
  otel.SetTracerProvider(tp)
  return tp, nil
}
```

若需要无代码方式采集 HTTP（eBPF），可考虑单独的 DaemonSet（如 Beyla）。在 Autopilot 下需确保其符合非特权限制。

***

### Autopilot 场景排错建议

* 验证到 OTLP 端点的 egress 连通性以及 TLS/证书要求
* 为所有容器定义资源请求/限制（resource requests/limits）
* 避免使用特权标志与仅 root 可写路径
* 若策略限制 initContainer，尽量将 Agent 直接打包进镜像
* 同时查看应用与后端/Collector 的日志，确认是否成功导出

### 禁用导出（仅采集，不上报）

在某些场景（例如 Autopilot 冒烟测试）中，可以保留插桩但暂时关闭导出。这样会创建本地的 span/metric/log，但不会向后端发送。

* 跨语言（环境变量）：
  ```bash
  export OTEL_TRACES_EXPORTER=none
  export OTEL_METRICS_EXPORTER=none
  export OTEL_LOGS_EXPORTER=none
  ```
  如上将禁用所有导出器，插桩仍会创建数据，但不会对外发送。

* Java（OpenTelemetry Java Agent）示例：
  ```bash
  JAVA_TOOL_OPTIONS="-javaagent:/otel/opentelemetry-javaagent.jar \
    -Dotel.traces.exporter=none \
    -Dotel.metrics.exporter=none \
    -Dotel.logs.exporter=none \
    -Dotel.resource.attributes=service.name=sp-storage \
    -Dotel.instrumentation.http.server.capture-request-headers=tracestate \
    -Dotel.instrumentation.http.server.capture-response-headers=tracestate"
  ```
  或直接在 JVM 启动命令追加系统属性：
  ```bash
  java -javaagent:/otel/opentelemetry-javaagent.jar \
    -Dotel.traces.exporter=none \
    -Dotel.metrics.exporter=none \
    -Dotel.logs.exporter=none \
    -Dotel.resource.attributes=service.name=sp-storage \
    -Dotel.instrumentation.http.server.capture-request-headers=tracestate \
    -Dotel.instrumentation.http.server.capture-response-headers=tracestate \
    -jar app.jar
  ```

注意：

* 关闭导出后可减少外部流量，便于验证；但插桩的开销仍然存在（会创建数据）。生产环境请恢复所需导出（例如设置 `OTEL_TRACES_EXPORTER=otlp`）。
* 建议始终配置好服务资源属性（service.name、namespace、version 等），以便后续随时开启导出，无需改动应用清单。

---

---
url: /zh/testing/agents/introduction.md
---
# Introduction

SoftProbe is a record-and-replay testing system for Java services. It attaches to an application as a `-javaagent`, observes real traffic in the background, records request and dependency data, then replays recorded cases later with automatic mocking and automatic comparison.

The SoftProbe CLI (`sp`) is the command-line interface to the SoftProbe backend (**sp-backend**). It is specified **AI-agent first**: stable JSON output, predictable exit codes, and artifact files for large diagnosis payloads.

## How SoftProbe Works

SoftProbe has four moving parts:

| Part | Role |
|------|------|
| **Instrumented application** | Your Java service, started with `-javaagent:/path/to/sp-agent.jar`. |
| **SoftProbe Java agent** | Weaves bytecode at runtime, similar operationally to an OpenTelemetry Java agent. It records HTTP, DB, cache, RPC, config, auth, dynamic class, and other dependency data without application code changes. |
| **SoftProbe backend** | Stores recorded cases, policies, replay plans, replay logs, diff results, extraction rules, app config, and agent heartbeats. |
| **`sp` CLI** | Registers apps, configures policies/rules, checks agent status, queries recorded data, starts replay plans, and diagnoses replay failures. |

The normal lifecycle is:

1. **Setup** the CLI and confirm the backend is reachable.
2. **Create an app** and get an `appId`.
3. **List the app** and confirm it is visible to the current user or agent token.
4. **Set recording policy** so the agent knows what traffic to capture.
5. **Install the Java agent** (`sp-agent.jar`) on the service host or image.
6. **Run the app with the agent in recording mode**, pinning `sp.app.id` to the registered `appId`, then send real or test traffic through the app.
7. **List recorded cases** and verify the expected requests were captured.
8. **Set replay policy**: mock policy and compare policy decide what is mocked and how differences are judged.
9. **Create a replay plan** from the recorded cases and run replay.
10. **Check replay results** with case, diff, log, trace, and mock-tree commands.

Replay is therefore not the first action in a fresh system. A replay plan requires recorded cases from an app that was actually run with the SoftProbe agent.

## Agent Startup Example

The agent is attached with a JVM `-javaagent` flag. Pin the app id explicitly in production; do not rely on inferred service names.

```bash
java \
  -javaagent:/opt/softprobe/sp-agent.jar \
  -Dsp.app.id=a1b2c3d4e5f67890 \
  -Dsp.api.url=http://127.0.0.1:8090 \
  -jar order-service.jar
```

`sp.app.id` is the identity SoftProbe uses to isolate recordings, pull config, and match replay data. It is separate from `OTEL_SERVICE_NAME`, which only describes telemetry resource attributes.

## Single public surface

All documented `sp` commands call **sp-backend** over HTTP (`:8090` by default). The Java agent also talks to backend services to pull config, send heartbeats, and upload recording/replay observability data.

## Who uses `sp`?

1. **AI coding agents** — replay failure diagnosis, trace lookup by business ID, policy validation, automated re-runs
2. **CI/CD** — policy gates on pull requests, scheduled replay smoke tests
3. **Platform engineers** — same commands as agents, often without `--json`

The web UI remains available for visual inspection and manual review; the CLI covers **scriptable** and **agent** workflows focused on record-and-replay.

## Relationship to other tools

| Tool | Role |
|------|------|
| **Web UI** | Visual diff, trace inspection, and console review |
| **Java agent** | In-process instrumentation, recording, replay-time mocking, and replay observability |
| **`spcode` CLI** | Local autonomous AI engine and interactive terminal workspace. Shared SSO configuration. |
| **[策略 YAML 指南](/zh/testing/policy-yaml-guide)** | `sp policy` 输入的 schema 与示例 |

## Documentation map

* [Softprobe Testing](/zh/testing/) — product overview (Java agent, record/replay, policies)
* [For AI agents](./overview) — start here if you write skills or tools
* [spcode CLI](/zh/testing/installation/code) — local autonomous AI engine and interactive terminal workspace
* [Commands](/zh/testing/commands/) — platform, investigation, and administration

---

---
url: /zh/platform/deployment/installation.md
---
# Istio WASM 插件安装

将 SP-Istio Agent 部署到您的生产 Istio 服务网格中。

::: tip 查找用户安装？
Softprobe 服务端安装（Helm sp-backend）、客户端安装（`sp setup`、Spcode Service）、`sp code`、`sp doctor` 与 `sp upgrade` 位于 [测试安装](/zh/testing/installation/)。本平台页面仅面向 Istio 网格 / SESSIFY 部署。
:::

::: tip 下一步：在 Dashboard 查看 Context View
部署 SP‑Istio Agent 并初始化 SESSIFY 后，先在应用中产生一些流量，然后：

1. 打开 Softprobe 仪表盘 → Context View
2. 选择与部署相匹配的时间范围与环境（env）
3. 如有需要可按 serviceName 过滤；也可用 userId/sessionId/request\_body\_hash 搜索定位会话
4. 点击会话查看端到端图谱、span、客户端指标与交互事件

无需改动服务端代码即可获得完整上下文可视化。
:::

## 先决条件

在生产环境中安装 SP-Istio Agent 之前，请确保您拥有：

* 一个正在运行的 Kubernetes 集群
* 已安装并配置好 Istio
* 具有适当权限的 kubectl 访问权限
* 到 Softprobe 端点的网络连接

## 安装

使用您在 [账户设置](../getting-started/account-setup) 阶段下载的个性化 `minimal.yaml` 文件安装 SP-Istio Agent。此文件包含您唯一的 API 密钥和预配置设置。

```bash
# 确保您使用的是从 Softprobe 控制台下载的 minimal.yaml 文件
kubectl apply -f minimal.yaml
```

这将在您的 Istio 服务网格中全局部署 WasmPlugin。

::: tip 前端 SESSIFY（可选但推荐）
若要在浏览器侧采集性能与用户行为数据，请在应用前端安装 Softprobe SESSIFY。详见 [前端插件安装](/zh/platform/sessify)。
:::

## 验证安装

检查 WasmPlugin 是否已成功创建：

```bash
kubectl get wasmplugin -A
```

您应该会看到列出的 SP-Istio Agent 插件。

## 配置

默认配置会捕获网格中所有服务的 HTTP 流量。您可以通过修改 WasmPlugin 资源来自定义行为。

## 前端可观测性与会话关联（sessionId）

若要实现从浏览器到后端的全链路追踪，请在应用前端集成 Softprobe SESSIFY。SDK 会在每个浏览器标签页初始化时自动生成并复用唯一的 sessionId：

* 同一标签页内的页面跳转与交互会复用同一个 sessionId，使一次用户访问能够完整串联。
* 打开新的标签页或窗口会生成新的 sessionId；关闭标签页或重新初始化后，会话也会随之重置。
* 所有上报的前端事件、性能指标与网络请求都会携带该 sessionId，便于与后端链路数据进行关联，形成端到端视图。

安装并在应用入口初始化：

```bash
npm install @softprobe/sessify
```

```jsx
// app/layout.tsx 或你的应用入口文件
'use client'
import { useEffect } from 'react';
import { initSessify } from '@softprobe/sessify';

export default function RootLayout({ children }) {
  useEffect(() => {
    // 使用默认配置初始化会话管理
    initSessify({});
  }, []);

  return (
    <html lang="zh">
      <body>{children}</body>
    </html>
  );
}
```

框架专属示例（React/Vue/Next.js）与进阶用法见完整的 [SESSIFY 集成](/zh/platform/sessify)。

最佳实践：

* 在前端将 sessionId 通过请求头（例如 `X-Session-Id`）或 Tracing Context 传递到后端；
* 在后端采集与日志中记录该标识，或在可观测性系统中进行关联，以提升排障与定位效率。

参考文档：

* [前端插件安装](/zh/platform/sessify)
* [配置指南](/zh/platform/configuration/config)

### 范围化部署

要仅将代理部署到特定的命名空间或工作负载，您可以创建一个范围化的 WasmPlugin 配置。请参阅 [配置指南](/zh/platform/configuration/config) 以获取详细的配置选项。

## 使用 Bookinfo 演示进行测试

要使用 Istio 的 Bookinfo 演示应用程序验证安装：

```bash
# 为默认命名空间启用 Istio 注入
kubectl label namespace default istio-injection=enabled --overwrite

# 部署 Bookinfo 应用程序
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.22/samples/bookinfo/platform/kube/bookinfo.yaml

# 部署 Bookinfo 网关
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.22/samples/bookinfo/networking/bookinfo-gateway.yaml

# 应用范围化的测试配置
kubectl apply -f https://raw.githubusercontent.com/softprobe/softprobe/main/deploy/test-bookinfo.yaml
```

### 生成测试流量

```bash
# 获取入口网关 URL
export GATEWAY_URL=$(kubectl -n istio-system get svc istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

# 生成一些流量
curl -sf "http://${GATEWAY_URL}/productpage" >/dev/null

# 验证插件是否正常工作
kubectl get wasmplugin -A
```

## 卸载

要从您的集群中移除 SP-Istio Agent：

```bash
kubectl delete wasmplugin -n istio-system sp-istio-agent
```

---

---
url: /zh/testing/java-agent.md
---

# Softprobe Java Agent

Softprobe Java Agent（`sp-agent.jar`）通过 `-javaagent` 挂载到 JVM。它在字节码层织入各类框架（*部署方式*上类似 OpenTelemetry Java Agent），但目的是**测试数据采集与回放时 Mock**，而非通用分布式追踪。

::: warning 不是 Istio/Envoy Agent
网格采集见[平台 Agent 架构](/zh/platform/advanced-guides/agent-architecture)。本节仅介绍 **JVM** Agent。
:::

## 前置条件

* 可通过 JVM 参数重启的 Java 服务
* Agent 主机可访问 **sp-backend**（本地默认 `http://127.0.0.1:8090`）
* 已注册 **`appId`** — 使用 `sp app create` 创建，并在所有实例上固定同一 id

## 启动命令

使用 `-javaagent` 及下列 JVM 参数挂载 Agent：

```bash
java \
  -javaagent:sp-agent.jar \
  -Dsp.app.id=<appId> \
  -Dsp.api.url=http://127.0.0.1:8090 \
  -jar your-service.jar
```

| 参数 | 指向 | 含义 |
|------|------|------|
| `-Dsp.app.id` | — | 注册应用 id（`sp app create` 返回的 16 位十六进制）。**请在共享录制的各环境固定此值。** |
| `-Dsp.api.url` | **sp-backend**（如 `:8090`） | **必填** — sp-backend 根 URL（须含 `http://` 或 `https://`）。环境变量回退：`SP_API_URL`。录制、回放、Mock、对比，**以及关联日志导出**（`{sp.api.url}/v1/logs`）。 |

当 `sp.api.url` 已设置且服务端 [统一日志管道](./installation/server.md#unified-log-pipeline) 已启用时，日志由 sp-backend 内部代理到 Vector — Agent **无需**单独配置 Vector URL。

### 可选：直连 Vector

高级场景（绕过 backend 代理）可设置：

```bash
-Dsp.otel.exporter.otlp.log.endpoint=http://<vector-host>:4320/v1/logs
```

该 JVM 属性优先于 `{sp.api.url}/v1/logs`。

未设置 `sp.api.url`（且未设置上述覆盖）时，录制与回放仍可用，但应用日志不会导出，该 trace 的 `sp logs` 将为空。

Agent 也可能从 jar 名或环境自动解析 app id；显式设置 `-Dsp.app.id` 可避免录制与回放 id 不一致。旧文档中的 **`sp.service.name`** 在部分部署中仍作别名；新环境请优先使用 **`sp.app.id`**。

## 环境标签

为录制流量打标签，便于筛选与限定回放范围：

```bash
-Dsp.mocker.tags=env=staging
```

录制数据会带上 `env=<值>`，从而只回放特定环境的用例。策略里用 `selector.envTags` 匹配同一标签——见 [策略 YAML 指南 · 通用字段](/zh/testing/policy-yaml-guide#common-fields)。

## 其他部署方式

### `sp.agent.conf` 配置文件

```properties title="META-INF/sp/sp.agent.conf（打包进 agent JAR）"
sp.api.url=http://127.0.0.1:8090
```

一体化与 Helm 部署会在打包时烘焙该配置；运维通常只需 `-javaagent` 与 `-Dsp.app.id`。指向其他后端时用 `SP_API_URL` 或 `-Dsp.api.url` 覆盖。

### Tomcat / `JAVA_OPTS`

在 `catalina.sh` 或 `JAVA_TOOL_OPTIONS` 中设置 Agent 参数，使每个工作 JVM 启动时自动加载。

### 与 OpenTelemetry 共存

若与其他 `-javaagent`（如 OpenTelemetry）冲突，可添加忽略前缀：

```bash
-Dsp.ignore.type.prefixes=io.opentelemetry
-Dsp.ignore.classloader.prefixes=io.opentelemetry
```

多个前缀用英文逗号分隔。

### 调试日志

```bash
-Dsp.enable.debug=true
```

## Agent 状态

`sp app status <appId>` 根据实例心跳返回 **`online`**、**`offline`** 或 **`never`**（默认约 60 秒无心跳视为 offline）。状态反映 Agent 是否在运行，而非仅是否注册了应用。

录制时，旧版界面曾用 **WORKING** / **SLEEPING** / **UNSTART** 表示实例状态；含义相同：必须注入 Agent 且开启录制才会产生用例。

## 完整用例应包含什么

健康的录制用例通常包括：

* **Servlet**（或其他入口类型）— 主 API 请求/响应
* **Database**、**Redis**、**HttpClient** 等 — 按调用顺序的依赖 mocker
* **DynamicClass**（可选）— 已配置的缓存/时间/加解密方法

有流量后列出用例：`sp record case list --app <appId> --json`。

## 生产环境保护

为降低对线上流量的影响，Agent 在过载或存储异常时会**背压**。

### 队列溢出

1. 录制任务先进入内存队列（默认容量 **1024**）。
2. 队列满则立即停止录制。
3. 约 30 秒后健康任务将采样率降低约 20% 并重试。
4. 若约 5 分钟后仍满，继续降频直至最低（约每小时一次）。
5. 队列恢复后（约 10 分钟）恢复正常录制。

### 存储健康

1. 调用 sp-storage 失败或超时时立即停止录制。
2. 约 10 秒后恢复录制并采样服务健康度。
3. 若约 3 分钟内仍不健康，按与队列溢出类似的方式逐步降频，直至存储恢复。

配合[录制策略](/zh/testing/policies)中的采样与脱敏，可将生产风险控制在可接受范围。

## 回放侧 Agent

接收回放流量的实例也必须挂载**同一** Agent JAR。专用回放机上请将录制设为关闭或极低，避免在回放过程中误录大量新流量。

## 下一步

Agent 挂上、`sp app status` 显示 online 之后，接入就完成了 → 进入核心流程 **[录制流量](/zh/testing/recording)**。

相关：[下载 Java Agent](/zh/testing/download-java-agent) · [支持的框架](/zh/testing/supported-frameworks) · [快速开始](/zh/testing/getting-started)

---

---
url: /zh/testing/reference/json-types.md
---
# JSON types

CLI-level envelope: see [Output contract](/zh/testing/agents/output-contract).

## Common `data` shapes

### ApplicationListItem

Used in `sp app list` → `data.items[]`.

```json
{
  "appId": "string",
  "appName": "string",
  "name": "string",
  "agentStatus": "online | offline | never",
  "lastSeenAt": 0,
  "agentVersion": "string",
  "env": "string",
  "tags": ["string"],
  "worktreeDirectory": "string"
}
```

### ApplicationCreateResult

Used in `sp app create` → `data`.

```json
{
  "success": true,
  "appId": "string",
  "msg": "string"
}
```

### AgentStatus

Used in `sp app status` → `data`.

```json
{
  "appId": "string",
  "status": "online | offline | never",
  "instanceCount": 0,
  "lastSeenAt": 0,
  "agentVersion": "string"
}
```

### PolicyValidateResult

```json
{
  "valid": true,
  "errors": [{ "path": "spec.sampling.rate", "message": "..." }],
  "warnings": []
}
```

### ReplayPlanCreated

```json
{
  "planId": "string",
  "result": 1,
  "desc": "string"
}
```

### ReplayProgress

```json
{
  "planId": "string",
  "status": "RUNNING | FINISHED | FAILED | CANCELLED",
  "percent": 0,
  "finished": false
}
```

### TraceSummary

```json
{
  "traceId": "string",
  "endpoint": "string",
  "status": "string",
  "durationMs": 0,
  "startedAt": "ISO-8601",
  "attrs": { "orderId": "ORD-123" }
}
```

### ArtifactResult

```json
{
  "artifact": "relative/path.json",
  "summary": {}
}
```

### PaginatedList

```json
{
  "items": [],
  "page": 1,
  "pageSize": 20,
  "total": 0,
  "hasMore": false
}
```

## Backend passthrough

When `--include-backend` (optional debug flag) is set, implementers may add `data._backend` with the raw `Response.body` or `CommonResponse.data`.

Agents should not depend on `_backend` in production skills.

## schemaVersion

Future CLI releases may add top-level `"schemaVersion": 1` in the envelope. Agents must ignore unknown top-level fields.

---

---
url: /en/testing/commands/log-query-fields.md
---
# Log query fields

**When agents use this:** Interpret rows returned by [`sp logs`](./logs) or `GET /api/recorder/logs` — field names, meanings, and when correlation ids may be missing.

This reference describes **CLI and API query output** only. It does not document Parquet file paths, partition layout, or how to query storage directly. Use [`sp logs`](./logs) or the canonical HTTP API for all lookups.

**Lookup key:** v1 accepts **`trace_id` only**. Optional correlation labels may appear on rows when ingested — they are not filter keys.

**Naming:** API and Parquet rows use **unprefixed** column names (`source`, `replay_id`, …). OTLP on the wire may use `sp.source`, `sp.replay_id`, etc. before Vector maps them into storage.

***

## Where rows appear

Each successful lookup returns one **chronological stream** of rows in `data.rows` (CLI `--json`) or the API `rows` array. Rows are ordered by event `timestamp` ascending.

Human-readable CLI output prints the same logical fields as API JSON.

See [sp logs](./logs) for the `--trace-id` lookup key, triage workflow, and required time bounds.

***

## Field reference

Every row includes the core fields below. Correlation fields are included **when the emitting runtime knew them at log time** — they may be absent on contextless lines (see [Absent correlation fields](#absent-correlation-fields)).

| Field | Always present | Description |
|-------|----------------|-------------|
| `timestamp` | yes | Event time of the log line. ISO-8601 UTC in JSON (for example `2026-06-27T10:00:10.123Z`). Used for chronological ordering and for caller `[since, until)` filtering. |
| `severity` | yes | Normalized severity text from the emitting logger (for example `DEBUG`, `INFO`, `WARN`, `ERROR`). Each component controls which severities it emits through **its own native logging configuration** — Softprobe does not impose a product-wide severity filter. |
| `body` | yes | Full log message text. v1 returns the complete `body`; query results do not truncate message content. |
| `service_name` | yes | Runtime service identity for the line (for example `travel-ota`, `sp-backend`). Identifies which process produced the row. |
| `source` | yes | Which v1 pipeline source produced the row. Fixed values: `agent`, `app`, or `backend` (see [source values](#source-values)). |
| `trace_id` | when known | W3C OpenTelemetry trace id for the request or work unit that was active when the line was emitted. **The only v1 lookup key.** |
| `span_id` | when known | OpenTelemetry span id for the active span when the line was emitted. |
| `replay_id` | when known | One replay **attempt** id when replay context was active at emit time. Optional label — not a query key. |
| `plan_id` | when known | Replay **plan** id when plan context was active at emit time. Optional label — not a query key. |
| `plan_item_id` | when known | One case or operation inside a replay plan. Optional label — not a query key. |

**Not in v1 query results:** `session_id` / `sp.session_id`.

***

## `source` values

| Value | Meaning | Typical `service_name` examples |
|-------|---------|--------------------------------|
| `agent` | Java agent self-diagnostics (instrumentation, export, internal agent logging) | Application service name under instrumentation |
| `app` | Application-under-test logs captured by the agent (Logback, Log4j2, JUL) | `travel-ota`, customer app id |
| `backend` | sp-backend diagnostic logs exported through OpenTelemetry | `sp-backend` |

Filter or scan by `source` when you only want application lines versus agent diagnostics versus backend lines in the same trace lookup.

```bash
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/sp-logs.json
jq -r '.rows[] | select(.source=="backend") | .body' /tmp/sp-logs.json | head -20
```

***

## Absent correlation fields

Correlation fields (`trace_id`, `span_id`, `replay_id`, `plan_id`, `plan_item_id`) are **omitted or empty when the runtime genuinely had no request or replay context** at emit time. This is expected — not a query defect.

Common cases:

| Situation | Typical absent fields | Why |
|-----------|----------------------|-----|
| Process **startup** or **shutdown** | Some or all correlation fields | No active HTTP/RPC request or replay dispatch yet, or context already cleared |
| **Background / housekeeping** lines | `trace_id`, `span_id`, replay/plan ids | Thread or timer work outside record/replay traffic |
| **Agent or backend idle** diagnostics | `replay_id`, `plan_id`, `plan_item_id` | Diagnostic line with trace context only, or no inbound W3C context |
| **Plan context not set** on a line | `plan_id`, `plan_item_id` | Line emitted outside a plan item dispatch even during replay |

When diagnosing a failed replay, query by the **`trace_id`** from the replay case or pytest correlation block. Filter rows by optional `replay_id` in local output when you need replay-scoped lines.

**Case-scoped diagnosis:** when the case row includes **`recordTime`** (API `requestDateTime`) and **`replayTime`**, use two ±2 minute windows (one per anchor) instead of one span from record to replay. The replay window usually has the relevant lines; the record window is often empty. See [sp logs — Case-scoped lookup](./logs#case-scoped-lookup-dual-windows).

***

## Per-component logging ownership

| `source` | Who controls `severity` and what gets emitted |
|----------|-----------------------------------------------|
| `agent` | Agent / JVM logging configuration (`sp.log.path`, `sp.log.console`, `sp.enable.debug`, etc.) |
| `app` | Application Logback, Log4j2, or JUL settings |
| `backend` | sp-backend logging and OpenTelemetry log export configuration |

Softprobe attaches correlation ids and forwards lines each logger already emits. It does not change application log levels or filter severities product-wide.

***

## Example row (JSON)

From [`sp logs --json`](./logs) or `GET /api/recorder/logs`:

```json
{
  "timestamp": "2026-06-27T10:00:10.123Z",
  "severity": "WARN",
  "body": "Replay comparison mismatch",
  "service_name": "sp-backend",
  "source": "backend",
  "trace_id": "2057ad46a7ce03d3955385f2a4142d29",
  "span_id": "8d10c94a2a6f4e11",
  "replay_id": "6891fd300c676b31",
  "plan_id": "6a3f2aad59f0c4655b0f99da",
  "plan_item_id": "6a3f2aad59f0c4655b0f99da:1"
}
```

A startup line from the same service might omit correlation fields entirely:

```json
{
  "timestamp": "2026-06-27T09:59:55.000Z",
  "severity": "INFO",
  "body": "Started SpBootApplication in 4.2 seconds",
  "service_name": "sp-backend",
  "source": "backend"
}
```

***

## Out of scope (v1)

This reference covers unified pipeline **query output** only. Not part of v1 unless separately specified:

* Record trace tables, metrics tables, replay read migration, historical backfill
* Non-replay-path service logs (dashboard, auth, and other Helm/workspace services)
* Direct Parquet file access, object-store credentials, or standalone query tools

***

## Related

* [sp logs](./logs) — command reference, flags, triage, and API mapping
* [Log correlation IDs — find and use ids](/en/testing/reference/log-correlation-ids)
* [Diagnose replay failure example](/en/testing/examples/agent-diagnose-replay)

---

---
url: /zh/testing/agents/output-contract.md
---
# Output contract

All public `sp` commands follow this contract when `--json` is set.

## CLI envelope (stdout on success)

```json
{
  "ok": true,
  "command": "app list",
  "data": { }
}
```

| Field | Type | Description |
|-------|------|-------------|
| `ok` | boolean | Always `true` on exit 0 |
| `command` | string | Normalized command name for logging |
| `data` | object | Command-specific payload (see [JSON types](/zh/testing/reference/json-types)) |

## CLI envelope (stderr on failure, exit 1)

```json
{
  "ok": false,
  "command": "replay run",
  "error": {
    "code": "API_ERROR",
    "message": "Human-readable summary",
    "httpStatus": 500,
    "backend": { }
  }
}
```

`backend` optionally contains the raw SoftProbe `Response` or schedule `CommonResponse` body for debugging.

## Backend response shapes

### sp-tr-api (`Response`)

Most console APIs return:

```json
{
  "responseStatusType": { "responseCode": 0, "responseDesc": "success" },
  "body": { }
}
```

The CLI maps `responseCode !== 0` to exit code `1`.

### sp-replay-schedule (`CommonResponse`)

Replay control (`createPlan`, `progress`, …) uses:

```json
{
  "result": 1,
  "desc": "success",
  "data": { }
}
```

The CLI maps `result !== 1` (per server convention) to exit code `1`.

## Artifacts (large output)

When response bodies exceed an internal threshold (recommended: 64 KiB) or when the command always produces files (diff detail, log download):

**stdout:**

```json
{
  "ok": true,
  "command": "replay diff get",
  "data": {
    "artifact": ".sp-work/diff-abc123.json",
    "summary": {
      "diffId": "abc123",
      "diffResultCode": 1,
      "categoryName": "..."
    }
  }
}
```

Agents should **read the artifact file** with their file tool, not expect full payloads on stdout.

Default `--out-dir`: `.sp-work/` in the current working directory (override per invocation).

## Pagination

List commands accept:

| Flag | Default | Description |
|------|---------|-------------|
| `--page` | `1` | 1-based page index |
| `--limit` | `20` | Page size (max 100 unless documented) |
| `--fields` | all | Comma-separated field filter |

Paginated JSON:

```json
{
  "ok": true,
  "command": "replay case list",
  "data": {
    "items": [],
    "page": 1,
    "pageSize": 20,
    "total": 142,
    "hasMore": true
  }
}
```

## Human-readable mode (no `--json`)

* Tables on stdout
* Errors on stderr as plain text
* Colors only when stdout is a TTY and `--pretty` is set (optional; not required for v1 implementers)

## Non-interactive rule

If `--json` is set and authentication is missing, the CLI must **not** prompt. Exit `3` with:

```json
{
  "ok": false,
  "error": { "code": "AUTH_REQUIRED", "message": "Set SP_TOKEN or run sp auth login" }
}
```

---

---
url: /en/platform.md
---

# Softprobe Platform

**Zero code changes · Full-context visibility · Cost optimization**

::: info
Softprobe captures every user journey as a session graph—making interactions analyzable, automation-ready, and economical to retain.
:::

## Quick links

* [Quick Start](/en/platform/getting-started/quick-start)
* [Production Installation](/en/platform/deployment/installation)
* [SESSIFY](/en/platform/sessify)
* [Dashboard User Guide](/en/platform/production/dashboard-user-guide)

## Record and replay (Java)

For Java traffic capture, replay, and diff, see [Softprobe Testing](/en/testing/getting-started). Automate with the [CLI quickstart](/en/testing/getting-started).

## How it works

* **Server-side:** Wasm plugin in Istio Envoy sidecar — [SP-Istio Agent on GitHub](https://github.com/softprobe/softprobe)
* **Client-side:** [SESSIFY](/en/platform/sessify) enriches sessions with routes, metrics, and interaction events

![Softprobe Architecture](/img/docs/how-it-work.png)

::: tip Currently available

* Data collection: SESSIFY and Istio/Envoy OpenTelemetry traces
* Visualization: Context View (session graph)
  :::

![Session Graph](/img/docs/context-view.png)

---

---
url: /en/testing/reference.md
---

# Reference

Reference material for Testing automation:

* API mapping
* JSON output envelopes
* Exit codes
* Replay send log markers

---

---
url: /zh/platform/sessify.md
---
# SESSIFY 集成

**轻量 • 零依赖 • 符合 W3C 标准的分布式追踪**

SESSIFY 是 Softprobe 的会话生命周期管理与分布式追踪 SDK，专为前端应用的会话管理与请求追踪设计。它自动处理会话的创建、校验、保活与过期逻辑，并向 HTTP 请求注入符合 W3C 标准的 trace 上下文，实现端到端的会话关联。

核心收益包括：自动化会话管理、跨微服务的分布式追踪、基于 Web Crypto API 的安全性，以及零依赖带来的最小打包体积。

::: info
快速开始环境已预装并启用了 SESSIFY（`@softprobe/sessify`）。本文档用于将 SDK 集成进你自己的前端应用（React / Vue / Next.js 等）。
:::

## 前置条件

* 一个现代 Web 应用（React、Vue、Next.js 或原生 JavaScript）
* Node.js 16+ 及打包工具（如 Vite、Webpack）

## 安装

```bash
# npm
npm install @softprobe/sessify

# yarn
yarn add @softprobe/sessify

# pnpm
pnpm add @softprobe/sessify
```

## 初始化

```ts
// React / Next.js 示例
import { useEffect } from 'react';
import { initSessify } from '@softprobe/sessify';

useEffect(() => {
  // 使用默认配置初始化
  initSessify({
  });
}, []);

// 原生 JS 示例
import { initSessify } from '@softprobe/sessify';

initSessify({
});
```

## 核心功能

## 核心用法

初始化后，你可以在应用的任意位置管理会话。

```javascript
import { getSessionId, startSession, endSession, isSessionActive } from '@softprobe/sessify';

// 1. 获取当前会话
// 若已过期或不存在，会自动创建一个新会话。
const sessionId = getSessionId();
console.log('Active Session:', sessionId);

// 2. 检查状态
if (isSessionActive()) {
  console.log('User is currently active');
}

// 3. 强制刷新（如登录时）
// 立即作废旧会话并开启一个全新会话。
const newSessionId = startSession();

// 4. 登出 / 清理
// 清空存储并作废会话。
endSession();
```

## 会话与上下文传播

* SDK 会在标签页首次初始化时自动生成一个唯一的 sessionId，无需你手动创建。
* 在同一浏览器标签页内进行的页面跳转与交互，都会复用同一个 sessionId，使一次用户访问在时间维度上能够完整串联。
* 打开新的标签页或窗口，会生成新的 sessionId；关闭标签页或重新初始化后，会话也会随之重置。
* 所有上报的数据都会携带该 sessionId，使后端能够将同一次会话中的前端事件与后端服务侧的链路数据进行关联，实现端到端的可观测性与排障效率提升。

## 配置

`initSessify` 函数接受一个配置对象，用于按需定制行为。

| 选项 | 类型 | 默认值 | 说明 |
|--------|------|---------|-------------|
| customTraceState | object | {} | 要包含进 tracestate 头的自定义键值对。 |
| sessionStorageType | session/local | 'session' | 控制持久化。`'local'` 在浏览器重启后仍保留；`'session'` 在标签页关闭时清除。 |

### 自定义 Trace State 示例

通过加入环境、版本等上下文来增强请求追踪：

```javascript
initSessify({
  sessionStorageType: 'local',
  customTraceState: {
    'x-sp-env': 'production',
    'x-sp-ver': '1.0.0',
    'x-sp-tier': 'premium'
  }
});
```

生成的头部：`tracestate: x-sp-session-id=...,x-sp-env=production,x-sp-ver=1.0.0...`

## 技术细节

### 会话生命周期策略

* **创建**：使用 base36 时间戳结合 Web Crypto API 随机串，生成约 16 字符的唯一 ID。

* **校验**：每次访问自动检查是否超时。一旦超过超时时间，旧会话被丢弃并无缝生成新会话。

### HTTP 拦截

@softprobe/sessify 会（安全地）对全局 fetch API 打补丁，以便：

* 检查会话是否活跃。
* 注入符合 W3C 标准的 tracestate 头。
* 附加你的 siteName 或 customTraceState 以及当前 session\_id。

---

---
url: /en.md
---

::: info
Two product areas, one site — **Testing** (Java record/replay plus `sp` automation) and **Business Observability** (Istio/SESSIFY). Pick the path that matches your role above.
:::

---

---
url: /zh.md
---

::: info
同一站点包含 **测试**（Java 录制/回放和 `sp` 自动化）与 **业务观测**（Istio/SESSIFY）两类产品区域，请按上方角色选择路径。
:::

---

---
url: /zh/platform/production/dashboard-user-guide.md
---
# Softprobe 仪表盘使用指南

欢迎使用 Softprobe 仪表盘。本指南帮助你快速熟悉界面、完成租户与成员管理，并稳定地使用核心功能。

## 🚀 快速开始

### 1. 注册与登录

* 访问首页，点击右上角“Sign Up”
* 输入邮箱与密码，完成邮件验证
* 登录并开始使用系统

::: tip
账号设置与公钥管理可参见 [账号设置指南](/zh/platform/getting-started/account-setup)。
:::

### 2. 界面总览

登录后，你将看到：

* **左侧导航**：模块快速入口
* **顶部栏**：用户信息与租户切换
* **主工作区**：当前模块的操作界面

## 📊 核心功能

### 1. 仪表盘首页

实时监控系统核心指标

#### 数据概览卡片

* **数据库使用量**：当前存储用量（GB）、使用比例与颜色提示
* **数据写入统计**：总记录数、时间范围筛选与历史对比
* **会话统计**：活跃会话、所选时间段总会话、趋势图
* **性能指标**：平均响应时间、P95 指标、系统健康状况

::: tip
如需实现端到端用户旅程关联，请在前端安装 [SESSIFY](/zh/platform/sessify) 并通过服务网格传播 sessionId。
:::

#### 操作与更新

* 页面进入时加载一次数据（不自动刷新）
* 更新方式：切换时间范围、切换租户或手动刷新页面

### 2. 租户管理

面向多租户环境的资源隔离与管理

#### 租户切换

* 使用顶部栏租户选择器（支持搜索与筛选）
* 点击目标租户完成切换
* 通过“+”创建新租户并填写基本信息

#### 租户设置

* 基本信息：名称、描述、图标
* 成员管理：添加/移除、角色与权限
* API 密钥：创建/管理密钥与有效期
* 危险操作区：删除租户、导出备份、清理数据

### 3. 团队成员管理

权限控制与协同管理

#### 添加成员

* 进入 租户设置 → 成员管理
* 点击“Add Member”，输入邮箱并选择角色
* 角色类型：**管理员**、**编辑者**、**查看者**

#### 角色权限

* **管理员**：拥有完整控制权限；可管理成员/设置、API 密钥与危险操作
* **编辑者**：可进行数据与监控配置的操作；不可管理成员与全局设置
* **查看者**：只读访问；适合报表查看与审计

## ⚠️ 常见限制与最佳实践

* 数据刷新：仪表盘首页不自动刷新；请切换时间范围或租户，或手动刷新更新数据。
* 角色与安全：遵循最小权限原则；避免对所有成员授予“管理员”；定期审查成员列表。
* API 密钥：完整密钥仅在创建时展示；请妥善保存并定期轮换；切勿提交到代码仓库。
* 速率限制：每个 API 密钥有每小时上限；客户端应实现重试/退避，并在仪表盘监控用量计数。
* PII 处理：避免在属性中发送敏感个人信息；使用哈希后的请求体进行关联。
* 多环境隔离：为生产/预发/开发分别创建租户与公钥，提升隔离性与审计便利性。
* 性能建议：结合时间过滤与 request\_body\_hash 使用；数据集增大时配合分区与缓存策略。

::: tip 快速提醒
若需要更高配额或自定义保留周期，请携带租户 ID 与预期负载邮件联系 support@softprobe.ai。
:::

## ❓ 常见问题

参阅 [FAQ](/zh/platform/support/faq)，包含数据与安全、移动端访问与导出等常见问题。

## ➡️ 下一步

* 学习核心概念：[理解核心概念](/zh/platform/advanced-guides/concepts)
* 部署生产环境：[安装指南](/zh/platform/deployment/installation)
* 配置高级规则：[配置参考](/zh/platform/configuration/config)

***

::: info 提示
部署完成并产生流量后，打开 Context View 即可查看端到端会话图谱、性能指标与交互事件。
:::

***

***

最后更新：2024 年 12 月

---

---
url: /zh/testing.md
---

# Softprobe 测试

**录制真实流量。回放时自动 Mock。无需手写用例即可对比。**

::: tip 准备开始？
🚀 如果您是首次接触 Softprobe，建议直接阅读 **[快速开始](/zh/testing/getting-started)** 指南！在 5 分钟内通过预构建的 JAR 快速体验，并无缝接入您自己的应用程序。
:::

Softprobe 测试面向 **Java** 服务的**录制回放**回归。以 `-javaagent` 挂载 Softprobe Java Agent，在后台采集入口 API 流量与对外依赖调用，随后在测试环境回放已录制用例：依赖由存储数据 Mock，结果自动对比。

::: info 本站产品区域
| 区域 | 适用场景 |
|------|----------|
| **[平台](/zh/platform/)** | 需要 Istio/Envoy 网格采集、SESSIFY 会话上下文或可观测性仪表盘 |
| **测试**（本节） | 需要 Java 录制回放、JVM Agent、策略、回放语义、安装、命令与自动化 |
:::

## 为什么选择录制回放

传统集成测试需要维护环境、造数与手写用例。Softprobe 测试则：

* **零业务代码改动** — 通过 `sp-agent.jar` 做字节码织入
* **真实流量带来高覆盖** — 生产或预发请求直接成为回放用例
* **回放环境隔离** — 数据库、HTTP 客户端、Redis、RPC、缓存等由录制数据 Mock，测试环境无需真实下游
* **WRITE 路径更安全** — 回放依赖行为不会污染共享库表
* **降低对比噪声** — 对比规则、时间 Mock、忽略节点处理时间戳、随机 ID 与环境差异字段

## 组件分工

| 组件 | 作用 |
|------|------|
| **你的 Java 服务** | 被测应用，以 `-javaagent:…/sp-agent.jar` 启动 |
| **Softprobe Java Agent** | 运行时录制与回放；回放阶段 Mock 依赖 |
| **Softprobe 后端**（`:8090`） | 通过 Helm 部署。存储用例（MongoDB）、下发策略、执行回放计划、计算差异 |
| **`sp` 命令**（可选） | 注册应用、应用策略、发起回放、排查失败 — 见 [命令](/zh/testing/commands/) |
| **仪表盘 / 工作台**（可选） | 可视化差异与链路查看 |

```mermaid
flowchart LR
  App[JVM 被测应用]
  Agent[Java Agent]
  Backend[sp-backend]
  App --> Agent
  Agent --> Backend
```

## 录制 → 回放 → 对比（简述）

1. **录制** — Agent 在应用处理真实请求时采集入口流量（如 HTTP `Servlet`）与依赖调用（`HttpClient`、`Database`、`Redis` 等）。
2. **存储** — 后端持久化每次交互；回放热路径使用 Redis Mock 缓存。
3. **回放** — 调度服务将录制的入口请求发到**测试实例**（`targetEnv` URL）；Agent 返回录制的依赖响应，不访问真实下游。
4. **对比** — 引擎对比录制与回放流量；策略定义忽略项与匹配严格度。

详情：[录制流量](/zh/testing/recording) · [工作原理](/zh/testing/how-it-works) · [回放与对比](/zh/testing/replay-and-diff)

## 平台 Agent ≠ Java Agent

[平台](/zh/platform/advanced-guides/agent-architecture) 的 **SP-Istio Agent** 运行在 Envoy 侧车，采集网格 HTTP 流量。**Softprobe 测试**使用以 `-javaagent` 挂载的 **JVM Agent**。二者解决的问题不同；在 SaaS 部署中可同时接入同一后端做关联分析。

## 适合阅读本节的人

* 首次接触录制回放的 **Java 开发者**
* 无需完整下游栈即可做回归的 **QA / 发布工程师**
* 在 K8s 或镜像中配置 `sp-backend` 与 Agent 启动的 **平台工程师**
* **自动化作者** — 先理解概念，再看 [命令](/zh/testing/commands/) 中的 `sp` 自动化约定

## 快速链接

* [快速开始](/zh/testing/getting-started) — 5分钟内体验 [Travel OTA](https://github.com/softprobe/demo-ota) 演示并快速接入您自己的应用。
* [安装 Softprobe](/zh/testing/installation/) — 使用 `sp` 安装、设置、启动编码、诊断与升级。
* [录制流量](/zh/testing/recording) — 核心流程第 1 步：产生用例。
* [Java Agent](/zh/testing/java-agent) — 挂载、JVM 参数与生产安全。
* [策略概览](/zh/testing/policies) — 按阶段配置 YAML。
* [回放与对比](/zh/testing/replay-and-diff) — 核心流程第 2 步：回归运行。
* [Webhook 与 CI/CD](/zh/testing/webhook-and-ci) — 部署后触发回放与流水线门禁。
* [支持的框架](/zh/testing/supported-frameworks)

---

---
url: /en/testing/commands/agent.md
---
# sp agent

**When agents use this:** Install a backend-compatible `sp-agent.jar` and get copy-paste JVM flags after `sp app create`.

## Synopsis

| Subcommand | Description |
|------------|-------------|
| `command` | Emit `-javaagent` and `sp.*` system properties for record mode |

## `agent command`

On **Softprobe Cloud**, run `sp tenant key ensure` once (or let this command auto-create the key). The output includes `-Dsp.api.token=` for the Java agent.

```bash
sp tenant key ensure --json   # SaaS: once per tenant
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
sp agent command --app a1b2c3d4e5f67890 --json
sp agent command --app a1b2c3d4e5f67890 --agent-jar ./sp-agent.jar --app-jar target/app.jar --json
```

| Flag | Description |
|------|-------------|
| `--app` | Required. Registered `appId` from `sp app create` |
| `--agent-jar` | Optional. Default: `$SP_AGENT_JAR`, then `${XDG_DATA_HOME}/softprobe/agent/sp-agent.jar` |
| `--app-jar` | Optional. Trailing `-jar …` in `startCommand` |
| `--format` | `json` (default), `shell`, `docker`, `maven` |

Download `sp-agent.jar` from [Download Java agent](/en/testing/download-java-agent). Available immutable versions are published under `https://install.softprobe.ai/artifacts/agent/<version>/sp-agent.jar`.

`apiUrl` in the JSON output comes from the resolved CLI profile (`api_url` / `SP_API_URL`). Override with `sp config set-url`, `SP_API_URL`, or the global `--api-url` flag before `agent command`.

Precedence for the agent at runtime (not the CLI): JVM `-Dsp.api.url` → env `SP_API_URL` → bundled `META-INF/sp/sp.agent.conf` in the agent JAR.

When the default jar is missing:

```json
{
  "ok": false,
  "command": "agent command",
  "error": {
    "code": "USAGE",
    "message": "sp-agent.jar not found",
    "backend": {
      "nextActions": ["Download sp-agent.jar from https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar"]
    }
  }
}
```

Example success `data` (abbreviated):

```json
{
  "appId": "a1b2c3d4e5f67890",
  "agentJar": "/home/user/.local/share/softprobe/agent/sp-agent.jar",
  "agentJarExists": true,
  "apiUrl": "http://127.0.0.1:8090",
  "jvmArgs": [
    "-javaagent:/home/user/.local/share/softprobe/agent/sp-agent.jar",
    "-Dsp.app.id=a1b2c3d4e5f67890",
    "-Dsp.api.url=http://127.0.0.1:8090",
    "-Dsp.api.token=…"
  ],
  "startCommand": "java -javaagent:… -Dsp.app.id=… …",
  "startCommandMultiline": "java \\\n  -javaagent:… \\\n  …",
  "dockerJavaToolOptions": "-javaagent:… …",
  "mavenArgLine": "-javaagent:… …",
  "nextActions": [
    "Run startCommand (or paste startCommandMultiline into a run script)",
    "Send traffic, then: sp record case list --app a1b2c3d4e5f67890 --since -1h --json"
  ]
}
```

| `--format` | stdout |
|------------|--------|
| `json` | Full envelope on stdout |
| `shell` | `startCommandMultiline` only |
| `docker` | `ENV JAVA_TOOL_OPTIONS='…'` |
| `maven` | `<argLine>…</argLine>` |

## Related

* [Doctor](/en/testing/installation/doctor) — `sp doctor`
* [app](./app) — create app and check heartbeat
* [record](./record) — list recorded cases
* [Concepts: Java agent](/en/testing/agents/concepts)

---

---
url: /zh/testing/commands/agent.md
---
# sp agent

**When agents use this:** Install a backend-compatible `sp-agent.jar` and get copy-paste JVM flags after `sp app create`.

## Synopsis

| Subcommand | Description |
|------------|-------------|
| `command` | Emit `-javaagent` and `sp.*` system properties for record mode |

## `agent command`

在 **Softprobe Cloud** 上，先运行一次 `sp tenant key ensure`（或让本命令自动创建 key）。输出中会包含供 Java agent 使用的 `-Dsp.api.token=`。

```bash
sp tenant key ensure --json   # SaaS：每个租户一次
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
sp agent command --app a1b2c3d4e5f67890 --json
sp agent command --app a1b2c3d4e5f67890 --agent-jar ./sp-agent.jar --app-jar target/app.jar --json
```

| Flag | Description |
|------|-------------|
| `--app` | Required. Registered `appId` from `sp app create` |
| `--agent-jar` | Optional. Default: `$SP_AGENT_JAR`, then `${XDG_DATA_HOME}/softprobe/agent/sp-agent.jar` |
| `--app-jar` | Optional. Trailing `-jar …` in `startCommand` |
| `--format` | `json` (default), `shell`, `docker`, `maven` |

请从 [下载 Java Agent](/zh/testing/download-java-agent) 下载 `sp-agent.jar`。不可变版本发布在 `https://install.softprobe.ai/artifacts/agent/<version>/sp-agent.jar`。

JSON 中的 `apiUrl` 来自 CLI 配置（`api_url` / `SP_API_URL`）。可通过 `sp config set-url`、`SP_API_URL` 或全局 `--api-url` 覆盖。

Agent 运行时解析顺序：JVM `-Dsp.api.url` → 环境变量 `SP_API_URL` → JAR 内嵌 `META-INF/sp/sp.agent.conf`。

When the default jar is missing:

```json
{
  "ok": false,
  "command": "agent command",
  "error": {
    "code": "USAGE",
    "message": "sp-agent.jar not found",
    "backend": {
      "nextActions": ["从 https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar 下载 sp-agent.jar"]
    }
  }
}
```

Example success `data` (abbreviated):

```json
{
  "appId": "a1b2c3d4e5f67890",
  "agentJar": "/home/user/.local/share/softprobe/agent/sp-agent.jar",
  "agentJarExists": true,
  "apiUrl": "http://127.0.0.1:8090",
  "jvmArgs": [
    "-javaagent:/home/user/.local/share/softprobe/agent/sp-agent.jar",
    "-Dsp.app.id=a1b2c3d4e5f67890",
    "-Dsp.api.url=http://127.0.0.1:8090",
    "-Dsp.api.token=…"
  ],
  "startCommand": "java -javaagent:… -Dsp.app.id=… …",
  "startCommandMultiline": "java \\\n  -javaagent:… \\\n  …",
  "dockerJavaToolOptions": "-javaagent:… …",
  "mavenArgLine": "-javaagent:… …",
  "nextActions": [
    "Run startCommand (or paste startCommandMultiline into a run script)",
    "Send traffic, then: sp record case list --app a1b2c3d4e5f67890 --since -1h --json"
  ]
}
```

| `--format` | stdout |
|------------|--------|
| `json` | Full envelope on stdout |
| `shell` | `startCommandMultiline` only |
| `docker` | `ENV JAVA_TOOL_OPTIONS='…'` |
| `maven` | `<argLine>…</argLine>` |

## Related

* [Doctor](/zh/testing/installation/doctor) — `sp doctor`
* [app](./app) — create app and check heartbeat
* [record](./record) — list recorded cases
* [Concepts: Java agent](/zh/testing/agents/concepts)

---

---
url: /en/testing/commands/app.md
---
# sp app

**When agents use this:** Resolve `appId`, check agent connectivity, register a service, or list recent replay plans for an app.

## Synopsis

List, create, and inspect SoftProbe **applications**. An application is a registered service; recording, replay, and policies are scoped by `appId`. See [Concepts — Application](/en/testing/agents/concepts#application-appid).

Use `--json` on all subcommands. See [Output contract](/en/testing/agents/output-contract).

## Subcommands

| Subcommand | Args | Description |
|------------|------|-------------|
| `list` | — | Applications visible to the caller, with agent status |
| `create` | `<appName>` | Register a new application (server assigns `appId`) |
| `status` | `<appId>` | Agent connectivity for one application |
| `replays` | `<appId>` | Recent replay plans (`--limit`) |

## `list`

```bash
sp app list --json
```

**REST:** `GET /api/applications/list`

Requires a token when `--json` is set (CLI exits with an auth error if `SP_TOKEN` is unset). Results are filtered by user group, ownership, and cross-group grants; see [Authentication](/en/testing/agents/authentication).

### JSON output

`data.items` is an array of list rows. Each row merges app config with live agent status (heartbeat threshold defaults to 60s → `offline`).

| Field | Description |
|-------|-------------|
| `appId` | Stable id for agent config and other `sp` commands |
| `appName`, `name` | Display name |
| `agentStatus` | `online`, `offline`, or `never` |
| `lastSeenAt` | Unix ms, freshest instance heartbeat |
| `agentVersion` | Agent build string |
| `env` | Primary env tag, or `production` |
| `tags` | Flattened tag values |
| `worktreeDirectory` | Optional workspace path |

```json
{
  "ok": true,
  "command": "app list",
  "data": {
    "items": [
      {
        "appId": "a1b2c3d4e5f67890",
        "appName": "order-service",
        "agentStatus": "online",
        "lastSeenAt": 1747564800000
      }
    ]
  }
}
```

## `create`

```bash
sp app create order-service-staging --json
```

**REST:** `POST /api/applications/create`

`appName` must be unique. The CLI sends `{ "appName": "<arg>" }`; the server may set `owners` from the JWT when omitted.

### Request body (REST)

| Field | Required | Description |
|-------|----------|-------------|
| `appName` | Yes | Unique name (CLI positional) |
| `owners` | No | Owner user names; defaults to current user |
| `visibilityLevel` | No | `0` public, `1` private |
| `groupId`, `groupName` | No | User group assignment |

### JSON output

| Field | Description |
|-------|-------------|
| `success` | Whether creation succeeded |
| `appId` | Generated id — use in agent and `sp replay run --app` |
| `msg` | Detail when `success` is false |

```json
{
  "ok": true,
  "command": "app create",
  "data": {
    "success": true,
    "appId": "f3e2d1c0b9a87654"
  }
}
```

## `status`

```bash
sp app status f3e2d1c0b9a87654 --json
```

**REST:** `GET /api/applications/{appId}/agent-status`

Aggregates JVM instance heartbeats for the app. `status` reflects the **freshest** instance:

| Value | Meaning |
|-------|---------|
| `never` | No instances reported |
| `online` | Latest heartbeat within threshold |
| `offline` | Instances exist but heartbeat is stale |

| Field | Description |
|-------|-------------|
| `appId` | Application id |
| `status` | `never`, `online`, or `offline` |
| `instanceCount` | Registered instances |
| `lastSeenAt` | Unix ms |
| `agentVersion` | From freshest instance |

```json
{
  "ok": true,
  "command": "app status",
  "data": {
    "appId": "f3e2d1c0b9a87654",
    "status": "online",
    "instanceCount": 2,
    "lastSeenAt": 1747564800000
  }
}
```

## `replays`

```bash
sp app replays f3e2d1c0b9a87654 --limit 10 --json
```

**REST:** `GET /api/applications/{appId}/replays/recent?limit=N`

| Flag | Default | Description |
|------|---------|-------------|
| `--limit` | `5` | Max plans (CLI rejects < 1; values above 100 are clamped to 100; server also clamps 1–100) |

`data` is an array of plan summaries (`planId`, `planName`, `status`, case counts, `createTime`, `triggeredBy`, …). Use `planId` with [sp replay](replay.md) and [replay case](/en/testing/commands/replay-case).

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `list` | GET | `/api/applications/list` |
| `create` | POST | `/api/applications/create` |
| `status` | GET | `/api/applications/{appId}/agent-status` |
| `replays` | GET | `/api/applications/{appId}/replays/recent` |

Header: `access-token: <JWT>`.

## Replaces `sp_api`

| sp\_api endpoint | sp command |
|-----------------|------------|
| `list_applications` | `sp app list` |
| `agent_status` | `sp app status` |
| `recent_replays` | `sp app replays` |

## Related

* [Concepts — Application](/en/testing/agents/concepts#application-appid)
* [Quickstart](/en/testing/getting-started)
* [sp replay](replay.md)
* [Diagnose replay failure](/en/testing/examples/agent-diagnose-replay)
* [JSON types — ApplicationListItem](/en/testing/reference/json-types#applicationlistitem)
* [API mapping](/en/testing/reference/api-mapping)

---

---
url: /zh/testing/commands/app.md
---
# sp app

**When agents use this:** Resolve `appId`, check agent connectivity, register a service, or list recent replay plans for an app.

## Synopsis

List, create, and inspect SoftProbe **applications**. An application is a registered service; recording, replay, and policies are scoped by `appId`. See [Concepts — Application](/zh/testing/agents/concepts#application-appid).

Use `--json` on all subcommands. See [Output contract](/zh/testing/agents/output-contract).

## Subcommands

| Subcommand | Args | Description |
|------------|------|-------------|
| `list` | — | Applications visible to the caller, with agent status |
| `create` | `<appName>` | Register a new application (server assigns `appId`) |
| `status` | `<appId>` | Agent connectivity for one application |
| `replays` | `<appId>` | Recent replay plans (`--limit`) |

## `list`

```bash
sp app list --json
```

**REST:** `GET /api/applications/list`

Requires a token when `--json` is set (CLI exits with an auth error if `SP_TOKEN` is unset). Results are filtered by user group, ownership, and cross-group grants; see [Authentication](/zh/testing/agents/authentication).

### JSON output

`data.items` is an array of list rows. Each row merges app config with live agent status (heartbeat threshold defaults to 60s → `offline`).

| Field | Description |
|-------|-------------|
| `appId` | Stable id for agent config and other `sp` commands |
| `appName`, `name` | Display name |
| `agentStatus` | `online`, `offline`, or `never` |
| `lastSeenAt` | Unix ms, freshest instance heartbeat |
| `agentVersion` | Agent build string |
| `env` | Primary env tag, or `production` |
| `tags` | Flattened tag values |
| `worktreeDirectory` | Optional workspace path |

```json
{
  "ok": true,
  "command": "app list",
  "data": {
    "items": [
      {
        "appId": "a1b2c3d4e5f67890",
        "appName": "order-service",
        "agentStatus": "online",
        "lastSeenAt": 1747564800000
      }
    ]
  }
}
```

## `create`

```bash
sp app create order-service-staging --json
```

**REST:** `POST /api/applications/create`

`appName` must be unique. The CLI sends `{ "appName": "<arg>" }`; the server may set `owners` from the JWT when omitted.

### Request body (REST)

| Field | Required | Description |
|-------|----------|-------------|
| `appName` | Yes | Unique name (CLI positional) |
| `owners` | No | Owner user names; defaults to current user |
| `visibilityLevel` | No | `0` public, `1` private |
| `groupId`, `groupName` | No | User group assignment |

### JSON output

| Field | Description |
|-------|-------------|
| `success` | Whether creation succeeded |
| `appId` | Generated id — use in agent and `sp replay run --app` |
| `msg` | Detail when `success` is false |

```json
{
  "ok": true,
  "command": "app create",
  "data": {
    "success": true,
    "appId": "f3e2d1c0b9a87654"
  }
}
```

## `status`

```bash
sp app status f3e2d1c0b9a87654 --json
```

**REST:** `GET /api/applications/{appId}/agent-status`

Aggregates JVM instance heartbeats for the app. `status` reflects the **freshest** instance:

| Value | Meaning |
|-------|---------|
| `never` | No instances reported |
| `online` | Latest heartbeat within threshold |
| `offline` | Instances exist but heartbeat is stale |

| Field | Description |
|-------|-------------|
| `appId` | Application id |
| `status` | `never`, `online`, or `offline` |
| `instanceCount` | Registered instances |
| `lastSeenAt` | Unix ms |
| `agentVersion` | From freshest instance |

```json
{
  "ok": true,
  "command": "app status",
  "data": {
    "appId": "f3e2d1c0b9a87654",
    "status": "online",
    "instanceCount": 2,
    "lastSeenAt": 1747564800000
  }
}
```

## `replays`

```bash
sp app replays f3e2d1c0b9a87654 --limit 10 --json
```

**REST:** `GET /api/applications/{appId}/replays/recent?limit=N`

| Flag | Default | Description |
|------|---------|-------------|
| `--limit` | `5` | Max plans (CLI rejects < 1; values above 100 are clamped to 100; server also clamps 1–100) |

`data` is an array of plan summaries (`planId`, `planName`, `status`, case counts, `createTime`, `triggeredBy`, …). Use `planId` with [sp replay](replay.md) and [replay case](/zh/testing/commands/replay-case).

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `list` | GET | `/api/applications/list` |
| `create` | POST | `/api/applications/create` |
| `status` | GET | `/api/applications/{appId}/agent-status` |
| `replays` | GET | `/api/applications/{appId}/replays/recent` |

Header: `access-token: <JWT>`.

## Replaces `sp_api`

| sp\_api endpoint | sp command |
|-----------------|------------|
| `list_applications` | `sp app list` |
| `agent_status` | `sp app status` |
| `recent_replays` | `sp app replays` |

## Related

* [Concepts — Application](/zh/testing/agents/concepts#application-appid)
* [Quickstart](/zh/testing/getting-started)
* [sp replay](replay.md)
* [Diagnose replay failure](/zh/testing/examples/agent-diagnose-replay)
* [JSON types — ApplicationListItem](/zh/testing/reference/json-types#applicationlistitem)
* [API mapping](/zh/testing/reference/api-mapping)

---

---
url: /en/testing/commands/auth.md
---
# sp auth

**When agents use this:** When `SP_TOKEN` is unset or expired.

## Synopsis

Authenticate and inspect identity.

## Subcommands

| Subcommand | Flags | Description |
|------------|-------|-------------|
| `login` | `--email`, `--code`, `--guest`, `--no-save` | Obtain JWT |
| `whoami` | — | Current user from token |
| `refresh` | `--user`, `--no-save` | Refresh token |

## Examples

```bash
sp auth login --email ops@corp.com --code 848291 --json
sp auth login --email ops@corp.com --code 848291 --no-save --json
sp auth whoami --json
export SP_TOKEN="$(sp auth login ... --no-save --json | jq -r '.data.token')"
```

By default, `login` and `refresh` save the token to the shared configuration file in
`${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc`. With `--no-save`, the token

is returned in JSON only and no config, cache, data, or state file is modified.

### JSON output (`login`)

```json
{
  "ok": true,
  "command": "auth login",
  "data": {
    "token": "eyJ...",
    "userName": "ops@corp.com",
    "expiresAt": "2026-05-19T12:00:00Z"
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `login` (email) | POST | `/api/login/verify` |
| `login` (guest) | POST | `/api/login/loginAsGuest` |
| `refresh` | GET | `/api/login/refresh/{userName}` |
| `whoami` | — | JWT decode locally, or future profile API |

Request body for verify: `VerifyRequestType` (`userName`, `verifyCode`, …).

## Replaces

None (new). CI uses `SP_TOKEN` directly.

## Related

* [Authentication guide](/en/testing/agents/authentication)

---

---
url: /zh/testing/commands/auth.md
---
# sp auth

**When agents use this:** When `SP_TOKEN` is unset or expired.

## Synopsis

Authenticate and inspect identity.

## Subcommands

| Subcommand | Flags | Description |
|------------|-------|-------------|
| `login` | `--email`, `--code`, `--guest`, `--no-save` | Obtain JWT |
| `whoami` | — | Current user from token |
| `refresh` | `--user`, `--no-save` | Refresh token |

## Examples

```bash
sp auth login --email ops@corp.com --code 848291 --json
sp auth login --email ops@corp.com --code 848291 --no-save --json
sp auth whoami --json
export SP_TOKEN="$(sp auth login ... --no-save --json | jq -r '.data.token')"
```

By default, `login` and `refresh` save the token to the shared configuration file in
`${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc`. With `--no-save`, the token

is returned in JSON only and no config, cache, data, or state file is modified.

### JSON output (`login`)

```json
{
  "ok": true,
  "command": "auth login",
  "data": {
    "token": "eyJ...",
    "userName": "ops@corp.com",
    "expiresAt": "2026-05-19T12:00:00Z"
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `login` (email) | POST | `/api/login/verify` |
| `login` (guest) | POST | `/api/login/loginAsGuest` |
| `refresh` | GET | `/api/login/refresh/{userName}` |
| `whoami` | — | JWT decode locally, or future profile API |

Request body for verify: `VerifyRequestType` (`userName`, `verifyCode`, …).

## Replaces

None (new). CI uses `SP_TOKEN` directly.

## Related

* [Authentication guide](/zh/testing/agents/authentication)

---

---
url: /en/testing/commands/config.md
---
# sp config

**When agents use this:** Once per session to verify connectivity; CI uses env vars instead of `init`.

## Synopsis

Manage XDG-backed SoftProbe config, profiles, and backend URL.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `init` | Create `${XDG_CONFIG_HOME}/softprobe/config.jsonc` and `sp.jsonc` |
| `show` | Print resolved config sources, active profile, URL, and masked token |
| `set-url <url>` | Set URL for the active `sp` profile |
| `set-profile <name>` | Switch the active `sp` profile |

## Examples

```bash
sp config init
sp config show --json
sp config set-url http://127.0.0.1:8090
sp config set-profile staging
```

### File layout

`sp` uses the global XDG namespace `softprobe`:

```text
${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc  # shared Softprobe connectivity config
${XDG_CONFIG_HOME:-~/.config}/softprobe/sp.jsonc      # sp CLI-specific config
${XDG_CONFIG_HOME:-~/.config}/softprobe/spcode.jsonc  # spcode AI assistant-specific config
.softprobe/                                           # local project config directory
${XDG_CACHE_HOME:-~/.cache}/softprobe/                # caches
${XDG_DATA_HOME:-~/.local/share}/softprobe/           # durable data, agent jars
${XDG_STATE_HOME:-~/.local/state}/softprobe/          # logs and state
```

`config.jsonc` is shared with other Softprobe tools. `sp.jsonc` is only for this CLI and overrides shared values. `spcode.jsonc` (global and inside project-level `.softprobe/`) is parsed separately by the `spcode` AI assistant engine.

### JSON output (`show`)

```json
{
  "ok": true,
  "command": "config show",
  "data": {
    "profile": "default",
    "url": "http://127.0.0.1:8090",
    "tokenConfigured": true,
    "sources": [
      "~/.config/softprobe/config.jsonc",
      "~/.config/softprobe/sp.jsonc"
    ]
  }
}
```

## Precedence

Later sources override earlier sources:

1. Defaults.
2. `${XDG_CONFIG_HOME}/softprobe/config.jsonc`.
3. `${XDG_CONFIG_HOME}/softprobe/sp.jsonc`.
4. Extra config from `SP_CONFIG`, when set.
5. Extra config from `--config`, when set.
6. Selected profile. Profile selection priority is `--profile`, `SP_PROFILE`,
   merged `profile`, then `default`.
7. Scalar env overrides: `SP_API_URL`, `SP_TOKEN`, `SP_AGENT_JAR`.
8. Scalar CLI flags: `--api-url`, `--token`, `--agent-jar`.

Explicit unknown profiles fail closed with `PROFILE_NOT_FOUND`; they never
silently fall back to `default`.

## REST mapping

Config subcommands are **local only** (no HTTP), except `config agent load` in [config legacy](./config-legacy).

## Errors

| Code | Exit | Cause |
|------|------|-------|
| `CONFIG_MISSING` | 2 | No config file; run `init` |
| `PROFILE_NOT_FOUND` | 2 | Unknown profile name |
| `CONFIG_PARSE_ERROR` | 2 | Invalid JSONC |
| `CONFIG_WRITE_ERROR` | 2 | Config file could not be written |

## Related

* [Configuration guide](/en/testing/installation/configuration)
* [auth](./auth)

---

---
url: /zh/testing/commands/config.md
---
# sp config

**When agents use this:** Once per session to verify connectivity; CI uses env vars instead of `init`.

## Synopsis

Manage XDG-backed SoftProbe config, profiles, and backend URL.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `init` | Create `${XDG_CONFIG_HOME}/softprobe/config.jsonc` and `sp.jsonc` |
| `show` | Print resolved config sources, active profile, URL, and masked token |
| `set-url <url>` | Set URL for the active `sp` profile |
| `set-profile <name>` | Switch the active `sp` profile |

## Examples

```bash
sp config init
sp config show --json
sp config set-url http://127.0.0.1:8090
sp config set-profile staging
```

### File layout

`sp` uses the global XDG namespace `softprobe`:

```text
${XDG_CONFIG_HOME:-~/.config}/softprobe/config.jsonc  # shared Softprobe connectivity config
${XDG_CONFIG_HOME:-~/.config}/softprobe/sp.jsonc      # sp CLI-specific config
${XDG_CONFIG_HOME:-~/.config}/softprobe/spcode.jsonc  # spcode AI assistant-specific config
.softprobe/                                           # local project config directory
${XDG_CACHE_HOME:-~/.cache}/softprobe/                # caches
${XDG_DATA_HOME:-~/.local/share}/softprobe/           # durable data, agent jars
${XDG_STATE_HOME:-~/.local/state}/softprobe/          # logs and state
```

`config.jsonc` is shared with other Softprobe tools. `sp.jsonc` is only for this CLI and overrides shared values. `spcode.jsonc` (global and inside project-level `.softprobe/`) is parsed separately by the `spcode` AI assistant engine.

### JSON output (`show`)

```json
{
  "ok": true,
  "command": "config show",
  "data": {
    "profile": "default",
    "url": "http://127.0.0.1:8090",
    "tokenConfigured": true,
    "sources": [
      "~/.config/softprobe/config.jsonc",
      "~/.config/softprobe/sp.jsonc"
    ]
  }
}
```

## Precedence

Later sources override earlier sources:

1. Defaults.
2. `${XDG_CONFIG_HOME}/softprobe/config.jsonc`.
3. `${XDG_CONFIG_HOME}/softprobe/sp.jsonc`.
4. Extra config from `SP_CONFIG`, when set.
5. Extra config from `--config`, when set.
6. Selected profile. Profile selection priority is `--profile`, `SP_PROFILE`,
   merged `profile`, then `default`.
7. Scalar env overrides: `SP_API_URL`, `SP_TOKEN`, `SP_AGENT_JAR`.
8. Scalar CLI flags: `--api-url`, `--token`, `--agent-jar`.

Explicit unknown profiles fail closed with `PROFILE_NOT_FOUND`; they never
silently fall back to `default`.

## REST mapping

Config subcommands are **local only** (no HTTP), except `config agent load` in [config legacy](./config-legacy).

## Errors

| Code | Exit | Cause |
|------|------|-------|
| `CONFIG_MISSING` | 2 | No config file; run `init` |
| `PROFILE_NOT_FOUND` | 2 | Unknown profile name |
| `CONFIG_PARSE_ERROR` | 2 | Invalid JSONC |
| `CONFIG_WRITE_ERROR` | 2 | Config file could not be written |

## Related

* [Configuration guide](/zh/testing/installation/configuration)
* [auth](./auth)

---

---
url: /en/testing/commands/config-legacy.md
---
# sp config legacy

**When agents use this:** Only when migrating old Mongo-backed config — prefer `sp policy` for recording/mock/compare.

## Synopsis

CRUD wrappers for `/api/config/*` abstract configurable controllers.

## Subcommand groups

| Group | Base path |
|-------|-----------|
| `legacy application` | `/api/config/application` |
| `legacy schedule` | `/api/config/schedule` |
| `legacy operation` | `/api/config/applicationOperation` |
| `legacy dynamic-class` | `/api/config/dynamicClass` |
| `legacy comparison` | `/api/config/comparison/*` |
| `legacy multiservice` | `/api/config/multiservice` |
| `legacy yaml-template` | `/api/config/yamlTemplate` |

Common pattern per resource:

```text
GET  …/useResult/appId/{appId}
GET  …/useResultAsList/appId/{appId}
POST …/modify/{modifyType}
POST …/batchModify/{modifyType}
```

## Agent guidance

* **Do not** use legacy schedule operation include/exclude as source of truth — recording policy overlay applies at read time.
* Use `sp policy recording|mock|compare` for declarative config.

## `sp config agent`

| Subcommand | Method | Path |
|------------|--------|------|
| `agent load --app <id>` | POST | `/api/config/agent/load` |
| `agent status` | POST | `/api/config/agent/agentStatus` |

Replaces `sp_api` `app_config`.

## Related

* [policy](/en/testing/commands/policy)
* [concepts](/en/testing/agents/concepts)

---

---
url: /zh/testing/commands/config-legacy.md
---
# sp config legacy

**When agents use this:** Only when migrating old Mongo-backed config — prefer `sp policy` for recording/mock/compare.

## Synopsis

CRUD wrappers for `/api/config/*` abstract configurable controllers.

## Subcommand groups

| Group | Base path |
|-------|-----------|
| `legacy application` | `/api/config/application` |
| `legacy schedule` | `/api/config/schedule` |
| `legacy operation` | `/api/config/applicationOperation` |
| `legacy dynamic-class` | `/api/config/dynamicClass` |
| `legacy comparison` | `/api/config/comparison/*` |
| `legacy multiservice` | `/api/config/multiservice` |
| `legacy yaml-template` | `/api/config/yamlTemplate` |

Common pattern per resource:

```text
GET  …/useResult/appId/{appId}
GET  …/useResultAsList/appId/{appId}
POST …/modify/{modifyType}
POST …/batchModify/{modifyType}
```

## Agent guidance

* **Do not** use legacy schedule operation include/exclude as source of truth — recording policy overlay applies at read time.
* Use `sp policy recording|mock|compare` for declarative config.

## `sp config agent`

| Subcommand | Method | Path |
|------------|--------|------|
| `agent load --app <id>` | POST | `/api/config/agent/load` |
| `agent status` | POST | `/api/config/agent/agentStatus` |

Replaces `sp_api` `app_config`.

## Related

* [policy](/zh/testing/commands/policy)
* [concepts](/zh/testing/agents/concepts)

---

---
url: /en/testing/commands/diagnose.md
---
# sp diagnose

**When agents use this:** One-shot workflows that combine progress, report, storage, and trace APIs — fewer manual steps than chaining low-level commands.

## Synopsis

| Subcommand | Description |
|------------|-------------|
| `replay <planId>` | Plan progress, failed cases, diff artifacts on disk |
| `trace <traceId>` | Record trace, completeness, summary artifacts |

## `diagnose replay`

Replaces the manual sequence in [Diagnose replay failure](/en/testing/examples/agent-diagnose-replay):

```bash
sp diagnose replay plan-abc123 --failed-only --out-dir .sp-work --json
```

| Flag | Default | Description |
|------|---------|-------------|
| `--failed-only` | `true` | Filter to cases with compare failures |
| `--out-dir` | `.sp-work` | Write `{planId}/{planItemId}-diff.json` files |
| `--page` / `--limit` | global | Pagination for case query |

Steps performed:

1. `GET /api/progress?planId=…`
2. `POST /api/report/queryReplayCase` with `diffResultCode=1` when `--failed-only`
3. For each failed case with `diffId`: `GET /api/report/queryDiffMsgById/{id}` → artifact file

Example `data` shape:

```json
{
  "planId": "plan-abc123",
  "status": "FINISHED",
  "classification": "invalid_target",
  "message": "Connection refused: travel-ota:9999",
  "failedCaseCount": 0,
  "invalidCaseCount": 12,
  "artifacts": [
    ".sp-work/plan-abc123/item-1-diff.json"
  ]
}
```

`classification` is one of: `empty_window`, `invalid_target`, `assertion_failure`, `mixed`, `other`. `message` comes from backend `errorMessage` or case send errors when available — not fabricated client copy.

**Note:** `nextActions` was removed from `diagnose replay --json` output (feature 007). Use `classification` + `message` for automation.

## `diagnose trace`

```bash
sp diagnose trace 4bf92f3577b34da6a3ce929d0e0e4736 --out-dir .sp-work --json
```

Fetches:

* `GET /api/storage/record/trace/{traceId}`
* `GET /api/storage/record/completeness?traceId=…`
* Trace summary (when available)

Writes JSON under `{outDir}/trace-{traceId}/` and returns a summary plus `nextActions` (e.g. `sp record query --trace-id …`).

## After diagnose: unified logs

`diagnose replay` returns diff artifacts but not runtime log lines. For each failed case:

1. Copy **`traceId`** from `sp replay case list --plan <planId> --failed --json` (v1 log lookup key — not `replayId`).
2. Query unified logs:

```bash
curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${SINCE}&until=${UNTIL}" \
  -H "Accept: application/json" -o .sp-work/unified-logs.json

jq '.rows | length' .sp-work/unified-logs.json
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' .sp-work/unified-logs.json
```

3. Triage: empty rows + `warnings` → reader/schema issue; empty + no warnings → wrong window or ingest lag; rows from `agent`, `app`, and `backend` → pipeline OK, read `body` and diff artifacts together.

On **`make e2e`** failures, pytest prints **Softprobe correlation** (`trace_id`) and **Unified logs** summaries — use those before widening the investigation.

See [Log correlation IDs](/en/testing/reference/log-correlation-ids) and [sp logs — troubleshooting](./logs#troubleshooting-failed-replays).

## Related

* [Log correlation IDs](/en/testing/reference/log-correlation-ids)
* [replay](./replay)
* [replay diff](./replay-diff)
* [record](./record)
* [trace](./trace)

---

---
url: /zh/testing/commands/diagnose.md
---
# sp diagnose

**When agents use this:** One-shot workflows that combine progress, report, storage, and trace APIs — fewer manual steps than chaining low-level commands.

## Synopsis

| Subcommand | Description |
|------------|-------------|
| `replay <planId>` | Plan progress, failed cases, diff artifacts on disk |
| `trace <traceId>` | Record trace, completeness, summary artifacts |

## `diagnose replay`

Replaces the manual sequence in [Diagnose replay failure](/zh/testing/examples/agent-diagnose-replay):

```bash
sp diagnose replay plan-abc123 --failed-only --out-dir .sp-work --json
```

| Flag | Default | Description |
|------|---------|-------------|
| `--failed-only` | `true` | Filter to cases with compare failures |
| `--out-dir` | `.sp-work` | Write `{planId}/{planItemId}-diff.json` files |
| `--page` / `--limit` | global | Pagination for case query |

Steps performed:

1. `GET /api/progress?planId=…`
2. `POST /api/report/queryReplayCase` with `diffResultCode=1` when `--failed-only`
3. For each failed case with `diffId`: `GET /api/report/queryDiffMsgById/{id}` → artifact file

Example `data` shape:

```json
{
  "planId": "plan-abc123",
  "status": "FINISHED",
  "classification": "invalid_target",
  "message": "Connection refused: travel-ota:9999",
  "failedCaseCount": 0,
  "invalidCaseCount": 12,
  "artifacts": [
    ".sp-work/plan-abc123/item-1-diff.json"
  ]
}
```

`classification` 取值之一：`empty_window`、`invalid_target`、`assertion_failure`、`mixed`、`other`。`message` 在可用时来自后端 `errorMessage` 或 case 发送错误——不是客户端伪造的固定文案。

**注意：** `nextActions` 已从 `diagnose replay --json` 输出中移除（feature 007）。请改用 `classification` + `message` 做自动化。

## `diagnose trace`

```bash
sp diagnose trace 4bf92f3577b34da6a3ce929d0e0e4736 --out-dir .sp-work --json
```

Fetches:

* `GET /api/storage/record/trace/{traceId}`
* `GET /api/storage/record/completeness?traceId=…`
* Trace summary (when available)

Writes JSON under `{outDir}/trace-{traceId}/` and returns a summary plus `nextActions` (e.g. `sp record query --trace-id …`).

## diagnose 之后：统一日志

`diagnose replay` 返回 diff 产物，但不包含运行时日志行。对每个失败的 case：

1. 从 `sp replay case list --plan <planId> --failed --json` 复制 **`traceId`**（v1 日志查询键——不是 `replayId`）。
2. 查询统一日志：

```bash
curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${SINCE}&until=${UNTIL}" \
  -H "Accept: application/json" -o .sp-work/unified-logs.json

jq '.rows | length' .sp-work/unified-logs.json
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' .sp-work/unified-logs.json
```

3. 分诊：rows 为空 + 有 `warnings` → reader/schema 问题；为空 + 无 warnings → 时间窗错误或摄入延迟；来自 `agent`、`app`、`backend` 的 rows → 管道正常，结合 `body` 与 diff 产物一起阅读。

在 **`make e2e`** 失败时，pytest 会打印 **Softprobe correlation**（`trace_id`）和 **Unified logs** 摘要——先看这些，再扩大排查范围。

参见 [日志关联 ID](/zh/testing/reference/log-correlation-ids) 和 [sp logs — 排查回放失败](./logs#troubleshooting-failed-replays)。

## Related

* [日志关联 ID](/zh/testing/reference/log-correlation-ids)
* [replay](./replay)
* [replay diff](./replay-diff)
* [record](./record)
* [trace](./trace)

---

---
url: /en/testing/commands/extraction-rule.md
---
# sp extraction-rule

**When agents use this:** Configure or preview business-attribute extraction for `sp trace find`.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `list --app <id>` | App extraction rules |
| `apply -f rules.yaml` | PUT app rules |
| `preview -f rules.yaml` | POST preview |

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `list` | GET | `/api/applications/{appId}/extraction-rules` |
| `apply` | PUT | `/api/applications/{appId}/extraction-rules` |
| `preview` | POST | `/api/extraction-rules/preview` |
| agent pull | GET | `/api/agent/extraction-rules` (agent use; CLI rarely) |

## Related

* [trace](./trace)

---

---
url: /zh/testing/commands/extraction-rule.md
---
# sp extraction-rule

**When agents use this:** Configure or preview business-attribute extraction for `sp trace find`.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `list --app <id>` | App extraction rules |
| `apply -f rules.yaml` | PUT app rules |
| `preview -f rules.yaml` | POST preview |

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `list` | GET | `/api/applications/{appId}/extraction-rules` |
| `apply` | PUT | `/api/applications/{appId}/extraction-rules` |
| `preview` | POST | `/api/extraction-rules/preview` |
| agent pull | GET | `/api/agent/extraction-rules` (agent use; CLI rarely) |

## Related

* [trace](./trace)

---

---
url: /en/testing/commands/group.md
---
# sp group & grant

**When agents use this:** Admin automation for access control (not typical diagnosis flows).

## Subcommands

### `sp group`

| Subcommand | Path prefix |
|------------|-------------|
| `list` | `/api/userGroup/list` |
| `my` | `/api/userGroup/my` |

### `sp grant`

| Subcommand | Path |
|------------|------|
| `list` | `GET /api/appGrant/list` |

`sp grant list` requires `--app <appId>`.

All commands require `access-token`.

## Examples

```bash
sp group list --json
sp group my --json
sp grant list --app my-app --json
```

## Related

* [app](/en/testing/commands/app)

---

---
url: /zh/testing/commands/group.md
---
# sp group & grant

**When agents use this:** Admin automation for access control (not typical diagnosis flows).

## Subcommands

### `sp group`

| Subcommand | Path prefix |
|------------|-------------|
| `list` | `/api/userGroup/list` |
| `my` | `/api/userGroup/my` |

### `sp grant`

| Subcommand | Path |
|------------|------|
| `list` | `GET /api/appGrant/list` |

`sp grant list` requires `--app <appId>`.

All commands require `access-token`.

## Examples

```bash
sp group list --json
sp group my --json
sp grant list --app my-app --json
```

## Related

* [app](/zh/testing/commands/app)

---

---
url: /en/testing/commands/health.md
---
# sp health

**When agents use this:** Preflight before a long diagnosis session; CI smoke check.

## Synopsis

Check sp-backend availability.

## Usage

```bash
sp health --json
sp health --api-url https://sp.example.com --json
```

## JSON output

```json
{
  "ok": true,
  "command": "health",
  "data": {
    "status": "UP",
    "url": "http://127.0.0.1:8090"
  }
}
```

## REST mapping

| Method | Path |
|--------|------|
| GET | `/vi/health` |

No authentication required in default deployments (verify for your environment).

## Errors

| Situation | Exit |
|-----------|------|
| Connection refused | 1 |
| Non-200 HTTP | 1 |

## Related

* [Installation](/en/testing/installation/)

---

---
url: /zh/testing/commands/health.md
---
# sp health

**When agents use this:** Preflight before a long diagnosis session; CI smoke check.

## Synopsis

Check sp-backend availability.

## Usage

```bash
sp health --json
sp health --api-url https://sp.example.com --json
```

## JSON output

```json
{
  "ok": true,
  "command": "health",
  "data": {
    "status": "UP",
    "url": "http://127.0.0.1:8090"
  }
}
```

## REST mapping

| Method | Path |
|--------|------|
| GET | `/vi/health` |

No authentication required in default deployments (verify for your environment).

## Errors

| Situation | Exit |
|-----------|------|
| Connection refused | 1 |
| Non-200 HTTP | 1 |

## Related

* [Installation](/zh/testing/installation/)

---

---
url: /en/testing/commands/logs.md
---
# sp logs

**When agents use this:** Retrieve correlated application, agent, and sp-backend logs for a W3C trace within caller-provided time bounds — without direct access to Parquet files or storage credentials.

**Prerequisite:** Unified log pipeline enabled (Vector ingest + Parquet storage + query wiring). See [Install sp-backend (server) — unified log pipeline](/en/testing/installation/server#unified-log-pipeline) and [Log correlation IDs](/en/testing/reference/log-correlation-ids).

v1 is **trace-id-only, canned lookup** — no SQL, no ad hoc query language, no `sp logs status` health command, and no replay/plan lookup keys.

**API:** `GET /api/recorder/logs?trace_id=…&since=…&until=…` on sp-backend. Top-level **`sp logs`** uses the same contract.

***

## Synopsis

Query unified log rows by **`trace_id`** within a caller-provided time window.

```bash
sp logs --trace-id <id> --since <time> --until <time> [--json]
```

## Flags

| Flag | Required | Description |
|------|----------|-------------|
| `--trace-id` | Yes | W3C trace id — **the only v1 lookup key** |
| `--since` | Yes | Inclusive lower bound — ISO-8601 UTC (e.g. `2026-06-27T10:00:00Z`) |
| `--until` | Yes | Exclusive upper bound — ISO-8601 UTC |
| `--json` | No | Stable JSON envelope for automation and Agent Skills |

Rules:

* **`--trace-id`**, **`--since`**, and **`--until`** are required for every lookup. Time range is half-open: `[since, until)`.
* v1 does not expose `--limit` or row truncation — narrow the window or filter locally (`grep`, `tail`, redirect to a file).
* Unsupported lookup keys (`--replay-id`, `--plan-id`, `--plan-item-id`, `--include-recording-log`) fail validation before reading Parquet.
* No authentication is required for v1 log lookups when you can reach the deployment endpoint.
* When the log pipeline is disabled or query dependencies are unavailable, the command fails fast with a clear error. It does not return an empty success result and does not fall back to legacy log storage.

***

## Examples

```bash
# Trace-scoped lookup after a failed replay (obtain trace_id from replay API or pytest output)
sp logs \
  --trace-id 2057ad46a7ce03d3955385f2a4142d29 \
  --since 2026-06-27T10:00:00Z \
  --until 2026-06-27T10:05:00Z \
  --json

# Human-readable output
sp logs \
  --trace-id 2057ad46a7ce03d3955385f2a4142d29 \
  --since 2026-06-27T10:00:00Z \
  --until 2026-06-27T10:05:00Z

# Large result — redirect or pipe (no --limit in v1)
sp logs --trace-id 2057ad46a7ce03d3955385f2a4142d29 --since … --until … > /tmp/trace.log
grep ERROR /tmp/trace.log | head -20
```

Agent Skills workflow (CLI or API):

```bash
# Canonical CLI
sp logs --trace-id "$TRACE_ID" --since "$SINCE" --until "$UNTIL" > .spcode/unified-logs-"$TRACE_ID".log
grep ERROR .spcode/unified-logs-"$TRACE_ID".log | head -20

# HTTP API (same contract)
curl -s "$SP_API_URL/api/recorder/logs?trace_id=$TRACE_ID&since=$SINCE&until=$UNTIL" > .spcode/unified-logs-"$TRACE_ID".json
```

***

## Output

**Human (default):** Chronological log stream — one line per row with timestamp, severity, `source`, `service_name`, and body.

**`--json`:** Same logical data in the standard CLI envelope (`ok`, `command`, `data`). Top-level `data` fields:

| Field | Meaning |
|-------|---------|
| `lookup` | Lookup type (`trace`), value, and caller `[since, until)` bounds |
| `rows` | Log lines — see [Log query fields](./log-query-fields) |
| `warnings` | Non-fatal schema-skip or similar notices (may be empty) |

v1 responses do **not** include `source_summary` or per-source row-count bucketing.

Rows do **not** include pytest labels, suite names, or test node ids.

Optional Softprobe labels (`replay_id`, `plan_id`, `plan_item_id`, …) may appear on individual rows when the emitter had that context — they are not filter keys.

***

## Case-scoped lookup (dual windows)

When diagnosing a **replay case**, you often have two timestamps:

* **`recordTime`** — when the case was originally recorded (API field `requestDateTime`)
* **`replayTime`** — when the replay run executed

**Do not** query from `recordTime` through `replayTime` in one request. That spans every minute partition in between and can scan hundreds of Parquet files.

Instead, run **two** narrow lookups (±2 minutes around each anchor) and merge rows client-side:

```bash
export SP_API_URL="${SP_API_URL:-http://127.0.0.1:18090}"
TRACE_ID="<32-hex from replay case traceId>"
RECORD_TIME_MS=1714000000000   # requestDateTime from case row
REPLAY_TIME_MS=1714046100000   # replayTime from case row
PADDING_MS=$((2 * 60 * 1000))

# Window 1: recording
RECORD_SINCE=$(date -u -d "@$(( (RECORD_TIME_MS - PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)
RECORD_UNTIL=$(date -u -d "@$(( (RECORD_TIME_MS + PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${RECORD_SINCE}&until=${RECORD_UNTIL}" \
  -H "Accept: application/json" -o /tmp/sp-logs-record.json

# Window 2: replay
REPLAY_SINCE=$(date -u -d "@$(( (REPLAY_TIME_MS - PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)
REPLAY_UNTIL=$(date -u -d "@$(( (REPLAY_TIME_MS + PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${REPLAY_SINCE}&until=${REPLAY_UNTIL}" \
  -H "Accept: application/json" -o /tmp/sp-logs-replay.json

# Merge and sort by timestamp (example with jq)
jq -s '[.[].rows[]] | sort_by(.timestamp)' /tmp/sp-logs-record.json /tmp/sp-logs-replay.json
```

The SoftProbe workbench **View case logs** action uses the same dual-window pattern automatically. The replay window usually contains the lines you need; the record window is often empty but cheap to query.

See [Log query fields](./log-query-fields) and [Log correlation IDs](/en/testing/reference/log-correlation-ids).

***

## Troubleshooting failed replays

Use this after `sp diagnose replay` or a pytest failure. See [Log correlation IDs](/en/testing/reference/log-correlation-ids) for id sources.

```bash
export SP_API_URL="${SP_API_URL:-http://127.0.0.1:18090}"
TRACE_ID="<32-hex from replay case traceId or pytest correlation block>"
SINCE="2026-06-27T10:00:00Z"
UNTIL="2026-06-27T10:05:00Z"

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${SINCE}&until=${UNTIL}" \
  -H "Accept: application/json" -o /tmp/sp-logs.json

jq '.rows | length' /tmp/sp-logs.json
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/sp-logs.json
jq '.warnings' /tmp/sp-logs.json
jq -r '.rows[] | select(.source=="backend" and .severity=="ERROR") | .body' /tmp/sp-logs.json | head -20
```

| Symptom | Likely cause |
|---------|----------------|
| 0 rows + non-empty `warnings` | Backend Parquet reader out of sync with schema — rebuild sp-backend image |
| 0 rows, empty `warnings` | Wrong `trace_id`, time window, or ingest not flushed yet |
| Rows from `agent`, `app`, and `backend` | Pipeline OK — inspect diff artifacts and log `body` for compare/mock timing |

**Pytest:** read **Softprobe correlation** (`trace_id`) and **Unified logs** (row/source summary) in failure output.

**Agent Skills:** shell first (`curl`, `jq`, `grep`) — do not implement Parquet readers in plugin code.

***

### JSON output

```json
{
  "ok": true,
  "command": "logs",
  "data": {
    "lookup": {
      "type": "trace",
      "value": "2057ad46a7ce03d3955385f2a4142d29",
      "windows": [
        {
          "since": "2026-06-27T10:00:00Z",
          "until": "2026-06-27T10:02:00Z"
        }
      ]
    },
    "rows": [
      {
        "timestamp": "2026-06-27T10:00:10.123Z",
        "severity": "WARN",
        "body": "Replay comparison mismatch",
        "service_name": "sp-backend",
        "source": "backend",
        "trace_id": "2057ad46a7ce03d3955385f2a4142d29",
        "span_id": "8d10c94a2a6f4e11",
        "replay_id": "6891fd300c676b31"
      }
    ],
    "warnings": []
  }
}
```

### JSON errors

Validation and API failures use the standard CLI stderr envelope:

```json
{
  "ok": false,
  "command": "logs",
  "error": {
    "code": "API_ERROR",
    "message": "API error 1: trace_id is required",
    "httpStatus": 200,
    "backend": {
      "responseCode": 1,
      "responseDesc": "trace_id is required"
    }
  }
}
```

Example validation messages: `trace_id is required`, `unsupported logs query parameter: replay_id`, `since is required`, `until is required`, `since must be before until`, `since and until must be ISO-8601 UTC timestamps`, `unsupported logs query parameter: <name>`, `log pipeline is disabled`, `log pipeline is unavailable`.

***

## REST mapping

| CLI | Method | Path |
|-----|--------|------|
| `--trace-id` | GET | `/api/recorder/logs?trace_id=<id>&since=<ts>&until=<ts>` |

Hosted on the same sp-backend base URL as other `sp` commands. v1 log lookups do not require authentication when you can reach the deployment endpoint.

***

## Retired commands (v1)

These pre-unified paths are removed, not shimmed:

| Retired | Replacement |
|---------|-------------|
| `sp record logs overview` | `sp logs --trace-id <id> --since … --until …` |
| `sp record logs download` | `sp logs --trace-id <id> …` (redirect to file) or `--json` with `jq` |
| `sp replay logs` (including `--overview`) | `sp logs --trace-id <id> …` (redirect to file) or `--json` with `jq` |
| `sp logs --replay-id`, `--plan-id`, `--plan-item-id` | **Rejected** — use `--trace-id` only |
| `--include-recording-log` | **Removed** — no record-link query |
| `GET /api/record-logs/*` | `GET /api/recorder/logs?trace_id=…` |
| `GET /api/replay-logs/*` | `GET /api/recorder/logs?trace_id=…` |

***

## Out of scope (v1)

* `sp logs status` / pipeline health status commands
* Replay-id, plan-id, or plan-item-id lookup keys
* Direct Parquet paths, catalog URLs, object-store credentials, or SQL
* `sp.session_id` in query results
* `--limit` / row truncation — narrow time bounds or filter locally instead
* Authentication for log lookups
* Record trace tables, metrics tables, replay read migration, historical backfill, and non-replay-path service logs (dashboard, auth, etc.) — replay **data** stays on the legacy replay-compatible storage path

***

## Related

* [Log query fields](./log-query-fields) — row field reference (FR-042)
* [Log correlation IDs — find and use ids](/en/testing/reference/log-correlation-ids)
* [sp replay case](./replay-case)
* [sp diagnose replay](./diagnose)
* [Diagnose replay failure example](/en/testing/examples/agent-diagnose-replay)

---

---
url: /zh/testing/commands/logs.md
---
> 本页翻译可能滞后于英文版，如有出入以[英文版](/en/testing/commands/logs)为准。

# sp logs

**Agent 何时使用：** 无需直接访问 Parquet 文件或存储凭据，即可在调用方提供的时间范围内，检索某个 W3C trace 的应用、agent 和 sp-backend 关联日志。

**前置条件：** 已启用统一日志管道（Vector 采集 + Parquet 存储 + 查询接线）。参见[安装 sp-backend（服务端）— 统一日志管道](/zh/testing/installation/server#unified-log-pipeline)和[日志关联 ID](/zh/testing/reference/log-correlation-ids)。

v1 **仅支持 trace-id、固定查询**——没有 SQL、没有即席查询语言、没有 `sp logs status` 健康检查命令，也没有 replay/plan 查询键。

**API：** sp-backend 上的 `GET /api/recorder/logs?trace_id=…&since=…&until=…`。顶层命令 **`sp logs`** 使用相同的契约。

***

## 概要

在调用方提供的时间窗口内，按 **`trace_id`** 查询统一日志行。

```bash
sp logs --trace-id <id> --since <time> --until <time> [--json]
```

## 标志

| Flag | 是否必填 | 说明 |
|------|----------|-------------|
| `--trace-id` | 是 | W3C trace id——**v1 唯一的查询键** |
| `--since` | 是 | 包含式下界——ISO-8601 UTC（例如 `2026-06-27T10:00:00Z`） |
| `--until` | 是 | 排除式上界——ISO-8601 UTC |
| `--json` | 否 | 用于自动化和 Agent Skills 的稳定 JSON 封装 |

规则：

* 每次查询都必须提供 **`--trace-id`**、**`--since`** 和 **`--until`**。时间范围是半开区间：`[since, until)`。
* v1 不提供 `--limit` 或行截断——请缩小时间窗口，或在本地过滤（`grep`、`tail`、重定向到文件）。
* 不支持的查询键（`--replay-id`、`--plan-id`、`--plan-item-id`、`--include-recording-log`）会在读取 Parquet 之前校验失败。
* 只要能访问到部署端点，v1 日志查询无需身份认证。
* 当日志管道被禁用或查询依赖不可用时，命令会快速失败并给出清晰的错误。它不会返回空的成功结果，也不会回退到旧版日志存储。

***

## 示例

```bash
# 回放失败后按 trace 范围查询（从 replay API 或 pytest 输出中获取 trace_id）
sp logs \
  --trace-id 2057ad46a7ce03d3955385f2a4142d29 \
  --since 2026-06-27T10:00:00Z \
  --until 2026-06-27T10:05:00Z \
  --json

# 人类可读输出
sp logs \
  --trace-id 2057ad46a7ce03d3955385f2a4142d29 \
  --since 2026-06-27T10:00:00Z \
  --until 2026-06-27T10:05:00Z

# 结果较大——重定向或用管道（v1 没有 --limit）
sp logs --trace-id 2057ad46a7ce03d3955385f2a4142d29 --since … --until … > /tmp/trace.log
grep ERROR /tmp/trace.log | head -20
```

Agent Skills 工作流（CLI 或 API）：

```bash
# 标准 CLI
sp logs --trace-id "$TRACE_ID" --since "$SINCE" --until "$UNTIL" > .spcode/unified-logs-"$TRACE_ID".log
grep ERROR .spcode/unified-logs-"$TRACE_ID".log | head -20

# HTTP API（相同契约）
curl -s "$SP_API_URL/api/recorder/logs?trace_id=$TRACE_ID&since=$SINCE&until=$UNTIL" > .spcode/unified-logs-"$TRACE_ID".json
```

***

## 输出

**人类可读（默认）：** 按时间顺序的日志流——每行一条记录，包含时间戳、严重级别、`source`、`service_name` 和正文。

**`--json`：** 相同的逻辑数据，采用标准 CLI 封装（`ok`、`command`、`data`）。顶层 `data` 字段：

| Field | 含义 |
|-------|---------|
| `lookup` | 查询类型（`trace`）、查询值，以及调用方的 `[since, until)` 边界 |
| `rows` | 日志行——参见[日志查询字段](./log-query-fields) |
| `warnings` | 非致命的 schema 跳过或类似提示（可能为空） |

v1 响应**不**包含 `source_summary`，也不做按来源的行数分桶统计。

各行**不**包含 pytest 标签、套件名称或测试节点 id。

当发射端具备相应上下文时，可选的 Softprobe 标签（`replay_id`、`plan_id`、`plan_item_id`……）可能出现在个别行上——它们不是过滤键。

***

## 按 case 范围查询（双窗口） {#case-scoped-lookup-dual-windows}

在诊断某个**回放 case** 时，你通常会有两个时间戳：

* **`recordTime`**——case 最初被录制的时间（API 字段 `requestDateTime`）
* **`replayTime`**——回放运行执行的时间

**不要**在一次请求中从 `recordTime` 一直查询到 `replayTime`。那会跨越其间的每一个分钟分区，可能扫描数百个 Parquet 文件。

正确做法是运行**两次**窄范围查询（在每个锚点前后各 ±2 分钟），并在客户端合并各行：

```bash
export SP_API_URL="${SP_API_URL:-http://127.0.0.1:18090}"
TRACE_ID="<32-hex from replay case traceId>"
RECORD_TIME_MS=1714000000000   # requestDateTime from case row
REPLAY_TIME_MS=1714046100000   # replayTime from case row
PADDING_MS=$((2 * 60 * 1000))

# Window 1: recording
RECORD_SINCE=$(date -u -d "@$(( (RECORD_TIME_MS - PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)
RECORD_UNTIL=$(date -u -d "@$(( (RECORD_TIME_MS + PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${RECORD_SINCE}&until=${RECORD_UNTIL}" \
  -H "Accept: application/json" -o /tmp/sp-logs-record.json

# Window 2: replay
REPLAY_SINCE=$(date -u -d "@$(( (REPLAY_TIME_MS - PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)
REPLAY_UNTIL=$(date -u -d "@$(( (REPLAY_TIME_MS + PADDING_MS) / 1000 ))" +%Y-%m-%dT%H:%M:%SZ)

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${REPLAY_SINCE}&until=${REPLAY_UNTIL}" \
  -H "Accept: application/json" -o /tmp/sp-logs-replay.json

# Merge and sort by timestamp (example with jq)
jq -s '[.[].rows[]] | sort_by(.timestamp)' /tmp/sp-logs-record.json /tmp/sp-logs-replay.json
```

SoftProbe 工作台的 **View case logs** 操作会自动使用相同的双窗口模式。回放窗口通常包含你需要的日志行；录制窗口往往为空，但查询成本很低。

参见[日志查询字段](./log-query-fields)和[日志关联 ID](/zh/testing/reference/log-correlation-ids)。

***

## 排查回放失败 {#troubleshooting-failed-replays}

在 `sp diagnose replay` 或 pytest 失败之后使用。id 来源参见[日志关联 ID](/zh/testing/reference/log-correlation-ids)。

```bash
export SP_API_URL="${SP_API_URL:-http://127.0.0.1:18090}"
TRACE_ID="<32-hex from replay case traceId or pytest correlation block>"
SINCE="2026-06-27T10:00:00Z"
UNTIL="2026-06-27T10:05:00Z"

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${SINCE}&until=${UNTIL}" \
  -H "Accept: application/json" -o /tmp/sp-logs.json

jq '.rows | length' /tmp/sp-logs.json
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/sp-logs.json
jq '.warnings' /tmp/sp-logs.json
jq -r '.rows[] | select(.source=="backend" and .severity=="ERROR") | .body' /tmp/sp-logs.json | head -20
```

| 现象 | 可能原因 |
|---------|----------------|
| 0 行 + `warnings` 非空 | Backend Parquet 读取器与 schema 不同步——重新构建 sp-backend 镜像 |
| 0 行，`warnings` 为空 | `trace_id`、时间窗口有误，或采集尚未刷盘 |
| 来自 `agent`、`app` 和 `backend` 的日志行 | 管道正常——检查 diff 制品，并查看日志 `body` 了解 compare/mock 时序 |

**Pytest：** 在失败输出中查看 **Softprobe correlation**（`trace_id`）和 **Unified logs**（行数/来源汇总）。

**Agent Skills：** 优先用 shell（`curl`、`jq`、`grep`）——不要在插件代码中实现 Parquet 读取器。

***

### JSON 输出

```json
{
  "ok": true,
  "command": "logs",
  "data": {
    "lookup": {
      "type": "trace",
      "value": "2057ad46a7ce03d3955385f2a4142d29",
      "windows": [
        {
          "since": "2026-06-27T10:00:00Z",
          "until": "2026-06-27T10:02:00Z"
        }
      ]
    },
    "rows": [
      {
        "timestamp": "2026-06-27T10:00:10.123Z",
        "severity": "WARN",
        "body": "Replay comparison mismatch",
        "service_name": "sp-backend",
        "source": "backend",
        "trace_id": "2057ad46a7ce03d3955385f2a4142d29",
        "span_id": "8d10c94a2a6f4e11",
        "replay_id": "6891fd300c676b31"
      }
    ],
    "warnings": []
  }
}
```

### JSON 错误

校验和 API 失败使用标准 CLI stderr 封装：

```json
{
  "ok": false,
  "command": "logs",
  "error": {
    "code": "API_ERROR",
    "message": "API error 1: trace_id is required",
    "httpStatus": 200,
    "backend": {
      "responseCode": 1,
      "responseDesc": "trace_id is required"
    }
  }
}
```

校验消息示例：`trace_id is required`、`unsupported logs query parameter: replay_id`、`since is required`、`until is required`、`since must be before until`、`since and until must be ISO-8601 UTC timestamps`、`unsupported logs query parameter: <name>`、`log pipeline is disabled`、`log pipeline is unavailable`。

***

## REST 映射

| CLI | Method | Path |
|-----|--------|------|
| `--trace-id` | GET | `/api/recorder/logs?trace_id=<id>&since=<ts>&until=<ts>` |

与其他 `sp` 命令托管在同一个 sp-backend 基础 URL 上。只要能访问到部署端点，v1 日志查询无需身份认证。

***

## 已废弃命令（v1）

以下统一化之前的路径已被移除，且没有做兼容垫片：

| 已废弃 | 替代方案 |
|---------|-------------|
| `sp record logs overview` | `sp logs --trace-id <id> --since … --until …` |
| `sp record logs download` | `sp logs --trace-id <id> …`（重定向到文件）或 `--json` 配合 `jq` |
| `sp replay logs`（包括 `--overview`） | `sp logs --trace-id <id> …`（重定向到文件）或 `--json` 配合 `jq` |
| `sp logs --replay-id`、`--plan-id`、`--plan-item-id` | **拒绝**——仅使用 `--trace-id` |
| `--include-recording-log` | **已移除**——不支持 record-link 查询 |
| `GET /api/record-logs/*` | `GET /api/recorder/logs?trace_id=…` |
| `GET /api/replay-logs/*` | `GET /api/recorder/logs?trace_id=…` |

***

## 不在范围内（v1）

* `sp logs status` / 管道健康状态命令
* Replay-id、plan-id 或 plan-item-id 查询键
* 直接的 Parquet 路径、catalog URL、对象存储凭据或 SQL
* 查询结果中的 `sp.session_id`
* `--limit` / 行截断——请改为缩小时间边界或在本地过滤
* 日志查询的身份认证
* Record trace 表、metrics 表、回放读取迁移、历史回填，以及非回放路径的服务日志（dashboard、auth 等）——回放**数据**仍保留在旧版兼容回放的存储路径上

***

## 相关

* [日志查询字段](./log-query-fields)——行字段参考（FR-042）
* [日志关联 ID——查找并使用 id](/zh/testing/reference/log-correlation-ids)
* [sp replay case](./replay-case)
* [sp diagnose replay](./diagnose)
* [诊断回放失败示例](/zh/testing/examples/agent-diagnose-replay)

---

---
url: /en/testing/commands/ops.md
---
# sp ops

**When agents use this:** SRE-style diagnostics for storage and schedule health.

## Synopsis

Operational endpoints under `/vi/storage` and `/vi/schedule`.

## Subcommands

| Subcommand | Method | Path |
|------------|--------|------|
| `storage overview` | GET | `/vi/storage/overview` |
| `storage diagnostics` | GET | `/vi/storage/diagnostics` |
| `storage monitor` | GET | `/vi/storage/monitor` |
| `schedule monitor` | GET | `/vi/schedule/monitor` |

## Examples

```bash
sp ops storage overview --json
sp ops schedule monitor --json
```

---

---
url: /zh/testing/commands/ops.md
---
# sp ops

**When agents use this:** SRE-style diagnostics for storage and schedule health.

## Synopsis

Operational endpoints under `/vi/storage` and `/vi/schedule`.

## Subcommands

| Subcommand | Method | Path |
|------------|--------|------|
| `storage overview` | GET | `/vi/storage/overview` |
| `storage diagnostics` | GET | `/vi/storage/diagnostics` |
| `storage monitor` | GET | `/vi/storage/monitor` |
| `schedule monitor` | GET | `/vi/schedule/monitor` |

## Examples

```bash
sp ops storage overview --json
sp ops schedule monitor --json
```

---

---
url: /en/testing/commands/policy.md
---
# sp policy

**When agents use this:** Validate and apply YAML policy changes; CI gates on `validate`.

## Synopsis

Manage declarative policies: **recording**, **mock**, **compare**.

## Structure

```text
sp policy <kind> <action>
```

`<kind>`: `recording` | `mock` | `compare`

| Action | Description |
|--------|-------------|
| `list` | List policy documents |
| `get <id>` | Get policy by id |
| `apply` | Create or update (`-f` file or stdin) |
| `delete <id>` | Remove policy |
| `validate` | Validate without save (`-f`) |
| `export <id>` | Write YAML to stdout or `-o` |
| `import` | Import YAML file (`-f`) |
| `diff` | Diff local file vs server export (`-f`, `--against <id>`) |
| `gate` | Validate all YAML files in a directory (CI) |

## Examples

```bash
sp policy recording list --json
sp policy recording validate -f policies/recording-prod.yaml --json
cat policies/recording-prod.yaml | sp policy recording validate -f - --json
sp policy recording apply -f policies/recording-prod.yaml --json
sp policy recording diff -f policies/recording-prod.yaml --against <policy-id> --json
sp policy gate --dir policies/ --json
sp policy gate --changed-only policies/recording-prod.yaml policies/mock-prod.yaml --json
sp policy recording delete <policy-id> --confirm --json
sp policy mock export policy-id-1 -o mock.yaml
sp policy compare get compare-global --json
```

### JSON output (`validate`)

```json
{
  "ok": true,
  "command": "policy recording validate",
  "data": {
    "valid": true,
    "errors": [],
    "warnings": []
  }
}
```

## REST mapping

### Recording (`/api/recording-policies`)

| Action | Method | Path |
|--------|--------|------|
| `list` | GET | `/policies` |
| `get` | GET | `/policies/{id}` |
| `apply` (JSON) | POST | `/policies` |
| `apply` (YAML) | POST | `/policies/yaml` (`Content-Type: text/yaml`) |
| `delete` | DELETE | `/policies/{id}` |
| `validate` | POST | `/policies/validate` |
| `export` | GET | `/policies/{id}/yaml` |

### Mock (`/api/mock-policies`)

Same path pattern under `/api/mock-policies`.

### Compare (`/api/compare-rules`)

| Action | Method | Path |
|--------|--------|------|
| `list` | GET | `/policies` |
| `get` | GET | `/policies/{id}` |
| `apply` | POST | `/policies` |
| `validate` | POST | `/policies/validate` |
| `export` | GET | `/policies/{id}/yaml` |
| templates | GET | `/templates`, `/functions` (v2 helpers) |

## Schema

See [CLI policies](/en/testing/policies) and [Policy YAML guide](/en/testing/policy-yaml-guide).

## Related

* [CI policy gate](/en/testing/examples/ci-policy-gate)
* [GitOps policies](/en/testing/examples/gitops-policies)

---

---
url: /zh/testing/commands/policy.md
---
# sp policy

**When agents use this:** Validate and apply YAML policy changes; CI gates on `validate`.

## Synopsis

Manage declarative policies: **recording**, **mock**, **compare**.

## Structure

```text
sp policy <kind> <action>
```

`<kind>`: `recording` | `mock` | `compare`

| Action | Description |
|--------|-------------|
| `list` | List policy documents |
| `get <id>` | Get policy by id |
| `apply` | Create or update (`-f` file or stdin) |
| `delete <id>` | Remove policy |
| `validate` | Validate without save (`-f`) |
| `export <id>` | Write YAML to stdout or `-o` |
| `import` | Import YAML file (`-f`) |
| `diff` | Diff local file vs server export (`-f`, `--against <id>`) |
| `gate` | Validate all YAML files in a directory (CI) |

## Examples

```bash
sp policy recording list --json
sp policy recording validate -f policies/recording-prod.yaml --json
cat policies/recording-prod.yaml | sp policy recording validate -f - --json
sp policy recording apply -f policies/recording-prod.yaml --json
sp policy recording diff -f policies/recording-prod.yaml --against <policy-id> --json
sp policy gate --dir policies/ --json
sp policy gate --changed-only policies/recording-prod.yaml policies/mock-prod.yaml --json
sp policy recording delete <policy-id> --confirm --json
sp policy mock export policy-id-1 -o mock.yaml
sp policy compare get compare-global --json
```

### JSON output (`validate`)

```json
{
  "ok": true,
  "command": "policy recording validate",
  "data": {
    "valid": true,
    "errors": [],
    "warnings": []
  }
}
```

## REST mapping

### Recording (`/api/recording-policies`)

| Action | Method | Path |
|--------|--------|------|
| `list` | GET | `/policies` |
| `get` | GET | `/policies/{id}` |
| `apply` (JSON) | POST | `/policies` |
| `apply` (YAML) | POST | `/policies/yaml` (`Content-Type: text/yaml`) |
| `delete` | DELETE | `/policies/{id}` |
| `validate` | POST | `/policies/validate` |
| `export` | GET | `/policies/{id}/yaml` |

### Mock (`/api/mock-policies`)

Same path pattern under `/api/mock-policies`.

### Compare (`/api/compare-rules`)

| Action | Method | Path |
|--------|--------|------|
| `list` | GET | `/policies` |
| `get` | GET | `/policies/{id}` |
| `apply` | POST | `/policies` |
| `validate` | POST | `/policies/validate` |
| `export` | GET | `/policies/{id}/yaml` |
| templates | GET | `/templates`, `/functions` (v2 helpers) |

## Schema

见 [CLI 策略索引](/zh/testing/policies) 与 [策略 YAML 指南](/zh/testing/policy-yaml-guide)。

## Related

* [CI policy gate](/zh/testing/examples/ci-policy-gate)
* [GitOps policies](/zh/testing/examples/gitops-policies)

---

---
url: /en/testing/commands/record.md
---
# sp record

**When agents use this:** Inspect what was recorded for a trace; verify agent behavior via logs.

## Synopsis

Read-only access to stored recordings (not agent write APIs).

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `case list` | List recorded entry cases for an app and time window |
| `query` | Query mocker/record payload by trace or replay id |
| `trace <traceId>` | Trace tree and children |
| `completeness <traceId>` | Full-link recording completeness |
| `view` | Visualization query/view |

> **Log lookup (v1):** Use top-level [`sp logs`](./logs) with `--trace-id` — not `sp record logs *` (removed in unified log pipeline).

## Examples

```bash
sp record case list --app a1b2c3d4e5f67890 --since -1h --json
sp record query --trace-id abc --out-dir .sp-work --json
sp record trace abc --json
sp record completeness abc --json
sp logs --trace-id abc --since 2026-06-27T10:00:00Z --until 2026-06-27T10:05:00Z --json
```

## `case list`

Use this immediately after running an instrumented app and sending traffic. It proves that replay has input data.

```bash
sp record case list --app a1b2c3d4e5f67890 --since -1h --limit 20 --json
```

Required behavior:

* `--app` is required and maps to the registered `appId`.
* `--since` / `--until` select the recording window. Durations such as `-1h` are resolved relative to now.
* `--page` / `--limit` paginate cases.
* Output includes case ids or trace ids, operation names, recorded time, and enough metadata to start trace or replay workflows.

Example JSON shape:

```json
{
  "ok": true,
  "command": "record case list",
  "data": {
    "items": [
      {
        "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
        "operationName": "GET /api/orders/{id}",
        "recordedAt": "2026-05-19T10:15:00Z"
      }
    ],
    "page": 1,
    "pageSize": 20,
    "total": 1
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `case list` | POST | `/api/storage/replay/query/replayCase` |
| `query` | POST | `/api/storage/record/query` |
| `trace` | GET | `/api/storage/record/trace/{traceId}` |
| `trace children` | GET | `/api/storage/record/trace/{traceId}/children` |
| `completeness` | GET | `/api/storage/record/completeness` |
| `view` | POST | `/api/storage/visualization/query` |

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `record_data` | `sp record query` |
| `record_log_overview` | `sp logs --trace-id …` (see [logs](./logs)) |
| `download_record_logs` | `sp logs --trace-id …` → redirect or `jq` |

## Non-goals

* `POST /api/storage/record/save`, `batchSave*` — agent instrumentation only

## Related

* [trace](./trace)
* [replay-diff](./replay-diff)

---

---
url: /zh/testing/commands/record.md
---
# sp record

**When agents use this:** Inspect what was recorded for a trace; verify agent behavior via logs.

## Synopsis

Read-only access to stored recordings (not agent write APIs).

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `case list` | List recorded entry cases for an app and time window |
| `query` | Query mocker/record payload by trace or replay id |
| `trace <traceId>` | Trace tree and children |
| `completeness <traceId>` | Full-link recording completeness |
| `view` | Visualization query/view |

> **日志查询（v1）：** 使用顶层 [`sp logs`](./logs) 加 `--trace-id` —— 不再使用 `sp record logs *`（已在统一日志管线中移除）。

## Examples

```bash
sp record case list --app a1b2c3d4e5f67890 --since -1h --json
sp record query --trace-id abc --out-dir .sp-work --json
sp record trace abc --json
sp record completeness abc --json
sp logs --trace-id abc --since 2026-06-27T10:00:00Z --until 2026-06-27T10:05:00Z --json
```

## `case list`

Use this immediately after running an instrumented app and sending traffic. It proves that replay has input data.

```bash
sp record case list --app a1b2c3d4e5f67890 --since -1h --limit 20 --json
```

Required behavior:

* `--app` is required and maps to the registered `appId`.
* `--since` / `--until` select the recording window. Durations such as `-1h` are resolved relative to now.
* `--page` / `--limit` paginate cases.
* Output includes case ids or trace ids, operation names, recorded time, and enough metadata to start trace or replay workflows.

Example JSON shape:

```json
{
  "ok": true,
  "command": "record case list",
  "data": {
    "items": [
      {
        "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
        "operationName": "GET /api/orders/{id}",
        "recordedAt": "2026-05-19T10:15:00Z"
      }
    ],
    "page": 1,
    "pageSize": 20,
    "total": 1
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `case list` | POST | `/api/storage/replay/query/replayCase` |
| `query` | POST | `/api/storage/record/query` |
| `trace` | GET | `/api/storage/record/trace/{traceId}` |
| `trace children` | GET | `/api/storage/record/trace/{traceId}/children` |
| `completeness` | GET | `/api/storage/record/completeness` |
| `view` | POST | `/api/storage/visualization/query` |

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `record_data` | `sp record query` |
| `record_log_overview` | `sp logs --trace-id …`（见 [logs](./logs)） |
| `download_record_logs` | `sp logs --trace-id …` → 重定向或用 `jq` |

## Non-goals

* `POST /api/storage/record/save`, `batchSave*` — agent instrumentation only

## Related

* [trace](./trace)
* [replay-diff](./replay-diff)

---

---
url: /en/testing/commands/recorder.md
---
# sp recorder (retired)

**This page is retired.** v1 unified log lookup is **trace-id-only** via [sp logs](./logs) and `GET /api/recorder/logs?trace_id=…`. `sp recorder logs`, `sp query`, and `sp recorder query` have been removed.

| Retired | Replacement |
|---------|-------------|
| `sp recorder logs --replay-id` | `sp logs --trace-id <traceId> …` or HTTP API with `trace_id` |
| `sp recorder logs --plan-id` / `--plan-item-id` | Resolve per-case **`traceId`**, then `sp logs --trace-id …` |
| `sp query` / `sp recorder query --sql` | Use `sp logs --trace-id …` (arbitrary SQL queries no longer supported) |
| `--include-recording-log` | **Removed** — record and replay share the same `trace_id` on replay |
| `source_summary` in responses | **Removed** — use `jq` to group rows by `source` |

Obtain **`traceId`** from replay case JSON, pytest **Softprobe correlation** output, or [Log correlation IDs](/en/testing/reference/log-correlation-ids).

## `sp recorder info` (still valid)

Show Recorder product health without exposing catalog or object-store credentials.

```bash
sp recorder info --json
```

Example JSON shape:

```json
{
  "ok": true,
  "command": "recorder info",
  "data": {
    "healthy": true,
    "profile": "onprem-helm",
    "helmEnabled": true,
    "vector": "ready",
    "ingest": "ready",
    "query": "ready",
    "catalog": "ready",
    "storage": "ready",
    "maintenance": "ready"
  }
}
```

## For log lookup, use `sp logs`

To correlate logs by trace, filter sp-backend `Replay send start` / `done` / `failed` markers, and for full flags, examples, triage workflow, and API mapping, see [sp logs](./logs) and [Replay send log markers](/en/testing/reference/replay-send-log-markers).

---

---
url: /zh/testing/commands/recorder.md
---
# sp recorder（已停用）

**本页已停用。** v1 统一日志查询**仅支持按 trace-id 查询**，通过 [sp logs](./logs) 及 `GET /api/recorder/logs?trace_id=…` 进行。`sp recorder logs`、`sp query`、`sp recorder query` 均已移除。

| 已停用 | 替代 |
|---------|-------------|
| `sp recorder logs --replay-id` | `sp logs --trace-id <traceId> …` 或使用带 `trace_id` 的 HTTP API |
| `sp recorder logs --plan-id` / `--plan-item-id` | 先解析出每个 case 对应的 **`traceId`**，再使用 `sp logs --trace-id …` |
| `sp query` / `sp recorder query --sql` | 使用 `sp logs --trace-id …`（不再支持任意 SQL 查询） |
| `--include-recording-log` | **已移除** —— 录制与回放在回放时共用同一个 `trace_id` |
| 响应中的 `source_summary` | **已移除** —— 请使用 `jq` 按 `source` 对各行分组 |

可从回放 case 的 JSON、pytest 的 **Softprobe correlation** 输出，或[日志关联 ID](/zh/testing/reference/log-correlation-ids)中获取 **`traceId`**。

## `sp recorder info`（仍有效）

在不暴露 catalog 或对象存储凭据的前提下查看 Recorder 产品健康状态。

```bash
sp recorder info --json
```

JSON 示例结构：

```json
{
  "ok": true,
  "command": "recorder info",
  "data": {
    "healthy": true,
    "profile": "onprem-helm",
    "helmEnabled": true,
    "vector": "ready",
    "ingest": "ready",
    "query": "ready",
    "catalog": "ready",
    "storage": "ready",
    "maintenance": "ready"
  }
}
```

## 日志查询请用 `sp logs`

按 trace 关联日志、筛选 sp-backend 的 `Replay send start` / `done` / `failed` 标记等，完整参数、示例、排查流程及 API 映射，请参见 [sp logs](./logs) 与[回放发送日志标记](/zh/testing/reference/replay-send-log-markers)。

---

---
url: /en/testing/commands/replay.md
---
# sp replay

**When agents use this:** Start and monitor replay plans. For failed cases and diffs, see [replay case](./replay-case) and [replay diff](./replay-diff).

## Synopsis

Create, monitor, stop, and rerun replay plans against recorded cases.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `run` | Create plan (`POST /api/createPlan`) |
| `status <planId>` | Poll progress (`GET /api/progress`) |
| `stop <planId>` | Stop plan |
| `rerun <planId>` | Re-run plan |

## Flags (`run`)

| Flag | API field | Description |
|------|-----------|-------------|
| `--app` | `appId` | Application id (required) |
| `--env` | `targetEnv` | **Replay target base URL** (required), not a symbolic name. Must include `http://` or `https://` and a host, e.g. `http://travel-ota:8080`. The CLI rejects values like `staging` or `dev`. |
| `--from` | `caseSourceFrom` | Start time (duration e.g. `-24h` or RFC3339) |
| `--to` | `caseSourceTo` | End time (default: now) |
| `--limit` | `caseCountLimit` | Max cases |
| `--name` | `planName` | Display name |
| `--operation` | `operationIds` | Repeatable; filter operations |
| `--enable-mock` | `enableMock` | Mock during replay (default true) |
| `--no-mock` | `enableMock` | Disable mock (`enableMock=false`; overrides `--enable-mock`) |
| `--allow-empty` | — | Create plan even when no recorded cases in window (default false) |
| `--watch` | — | On `run`: create plan then poll until terminal; on `status`: poll an existing plan |

## Examples

```bash
sp record case list --app my-app --since -24h --json   # verify cases exist first
sp replay run --app my-app --env http://travel-ota:8080 --from -24h --enable-mock --json
sp replay run --app my-app --env http://travel-ota:8080 --from -24h --no-mock --watch --json
sp replay status plan-xyz --watch --json
sp replay stop plan-xyz --json
sp replay rerun plan-xyz --json
```

### Preflight (`run`)

Before `POST /api/createPlan`, the CLI queries `POST /api/storage/replay/query/replayCase` for the same `--app`, `--from`, and `--to` window. If no entry cases exist and `--allow-empty` is not set:

```json
{
  "ok": false,
  "command": "replay run",
  "error": {
    "code": "NO_RECORDED_CASES",
    "message": "no recorded cases for app …; run the app with the agent and send traffic first, or use --allow-empty"
  }
}
```

### JSON output (`run`)

```json
{
  "ok": true,
  "command": "replay run",
  "data": {
    "planId": "plan-xyz",
    "result": 1,
    "desc": "success"
  }
}
```

### JSON output (`run --watch`)

With `--json`, stdout is newline-delimited envelopes:

1. One success envelope for plan creation (`command`: `replay run`).
2. One or more progress envelopes (`command`: `replay status`).
3. The final progress envelope includes `"finished": true` in `data`.

### JSON output (`status`)

```json
{
  "ok": true,
  "command": "replay status",
  "data": {
    "planId": "plan-xyz",
    "status": "RUNNING",
    "percent": 42,
    "finished": false
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `run` | POST | `/api/createPlan` |
| `run` (webhook style) | GET | `/api/createPlan?appId=…` (discouraged for CLI; use POST) |
| `status` | GET | `/api/progress?planId=` |
| `stop` | GET | `/api/stopPlan?planId=` |
| `rerun` | POST | `/api/reRunPlan` |

Body for `run`: `BuildReplayPlanRequest` (schedule module).

Response envelope: `CommonResponse` (`result`, `desc`, `data`).

## Related

* [replay case](./replay-case)
* [replay diff](./replay-diff)
* [Diagnose replay failure](/en/testing/examples/agent-diagnose-replay)

---

---
url: /zh/testing/commands/replay.md
---
# sp replay

**When agents use this:** Start and monitor replay plans. For failed cases and diffs, see [replay case](./replay-case) and [replay diff](./replay-diff).

## Synopsis

Create, monitor, stop, and rerun replay plans against recorded cases.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `run` | Create plan (`POST /api/createPlan`) |
| `status <planId>` | Poll progress (`GET /api/progress`) |
| `stop <planId>` | Stop plan |
| `rerun <planId>` | Re-run plan |

## Flags (`run`)

| Flag | API field | Description |
|------|-----------|-------------|
| `--app` | `appId` | Application id (required) |
| `--env` | `targetEnv` | **Replay target base URL** (required), not a symbolic name. Must include `http://` or `https://` and a host, e.g. `http://travel-ota:8080`. The CLI rejects values like `staging` or `dev`. |
| `--from` | `caseSourceFrom` | Start time (duration e.g. `-24h` or RFC3339) |
| `--to` | `caseSourceTo` | End time (default: now) |
| `--limit` | `caseCountLimit` | Max cases |
| `--name` | `planName` | Display name |
| `--operation` | `operationIds` | Repeatable; filter operations |
| `--enable-mock` | `enableMock` | Mock during replay (default true) |
| `--no-mock` | `enableMock` | Disable mock (`enableMock=false`; overrides `--enable-mock`) |
| `--allow-empty` | — | Create plan even when no recorded cases in window (default false) |
| `--watch` | — | On `run`: create plan then poll until terminal; on `status`: poll an existing plan |

## Examples

```bash
sp record case list --app my-app --since -24h --json   # verify cases exist first
sp replay run --app my-app --env http://travel-ota:8080 --from -24h --enable-mock --json
sp replay run --app my-app --env http://travel-ota:8080 --from -24h --no-mock --watch --json
sp replay status plan-xyz --watch --json
sp replay stop plan-xyz --json
sp replay rerun plan-xyz --json
```

### Preflight (`run`)

Before `POST /api/createPlan`, the CLI queries `POST /api/storage/replay/query/replayCase` for the same `--app`, `--from`, and `--to` window. If no entry cases exist and `--allow-empty` is not set:

```json
{
  "ok": false,
  "command": "replay run",
  "error": {
    "code": "NO_RECORDED_CASES",
    "message": "no recorded cases for app …; run the app with the agent and send traffic first, or use --allow-empty"
  }
}
```

### JSON output (`run`)

```json
{
  "ok": true,
  "command": "replay run",
  "data": {
    "planId": "plan-xyz",
    "result": 1,
    "desc": "success"
  }
}
```

### JSON output (`run --watch`)

With `--json`, stdout is newline-delimited envelopes:

1. One success envelope for plan creation (`command`: `replay run`).
2. One or more progress envelopes (`command`: `replay status`).
3. The final progress envelope includes `"finished": true` in `data`.

### JSON output (`status`)

```json
{
  "ok": true,
  "command": "replay status",
  "data": {
    "planId": "plan-xyz",
    "status": "RUNNING",
    "percent": 42,
    "finished": false
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `run` | POST | `/api/createPlan` |
| `run` (webhook style) | GET | `/api/createPlan?appId=…` (discouraged for CLI; use POST) |
| `status` | GET | `/api/progress?planId=` |
| `stop` | GET | `/api/stopPlan?planId=` |
| `rerun` | POST | `/api/reRunPlan` |

Body for `run`: `BuildReplayPlanRequest` (schedule module).

Response envelope: `CommonResponse` (`result`, `desc`, `data`).

## Related

* [replay case](./replay-case)
* [replay diff](./replay-diff)
* [Diagnose replay failure](/zh/testing/examples/agent-diagnose-replay)

---

---
url: /en/testing/commands/replay-case.md
---
# sp replay (data)

**When agents use this:** After `sp replay run` completes — list failed cases, fetch metadata, paginate cases.

## Synopsis

Query replay plans, cases, and metadata (read-only report/storage APIs).

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `metadata <replayId>` | Replay metadata (nodes, fullLink, traceId) |
| `case list` | Cases under plan or plan item |
| `case get <caseId>` | Single case detail (`--plan-item` required) |

## Flags (`case list`)

| Flag | Description |
|------|-------------|
| `--plan` | Plan id |
| `--plan-item` | Plan item / operation id |
| `--failed` | Only failed/error cases |
| `--diff-result-code` | Filter by diff code (1=diff, 2=error) |
| `--page` / `--limit` | Pagination |

## Examples

```bash
sp replay metadata replay-uuid --json
sp replay case list --plan plan-xyz --failed --page 1 --limit 20 --json
sp replay case list --plan-item item-abc --json
```

## REST mapping

| Subcommand | APIs |
|------------|------|
| `metadata` | Storage replay query / schedule metadata endpoints |
| `case list` | `/api/report/queryReplayCase`, storage `viewRecord`, schedule report |
| `case get` | Report query by planItemId |

Exact paths vary by deployment; see [API mapping](/en/testing/reference/api-mapping).

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `query_replay_metadata` | `sp replay metadata` |
| `query_plan_fail_cases` | `sp replay case list --plan … --failed` |
| `query_replay_case` | `sp replay case list` |

## Related

* [replay-diff](./replay-diff)
* [Diagnose replay failure](/en/testing/examples/agent-diagnose-replay)

---

---
url: /zh/testing/commands/replay-case.md
---
# sp replay (data)

**When agents use this:** After `sp replay run` completes — list failed cases, fetch metadata, paginate cases.

## Synopsis

Query replay plans, cases, and metadata (read-only report/storage APIs).

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `metadata <replayId>` | Replay metadata (nodes, fullLink, traceId) |
| `case list` | Cases under plan or plan item |
| `case get <caseId>` | Single case detail (`--plan-item` required) |

## Flags (`case list`)

| Flag | Description |
|------|-------------|
| `--plan` | Plan id |
| `--plan-item` | Plan item / operation id |
| `--failed` | Only failed/error cases |
| `--diff-result-code` | Filter by diff code (1=diff, 2=error) |
| `--page` / `--limit` | Pagination |

## Examples

```bash
sp replay metadata replay-uuid --json
sp replay case list --plan plan-xyz --failed --page 1 --limit 20 --json
sp replay case list --plan-item item-abc --json
```

## REST mapping

| Subcommand | APIs |
|------------|------|
| `metadata` | Storage replay query / schedule metadata endpoints |
| `case list` | `/api/report/queryReplayCase`, storage `viewRecord`, schedule report |
| `case get` | Report query by planItemId |

Exact paths vary by deployment; see [API mapping](/zh/testing/reference/api-mapping).

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `query_replay_metadata` | `sp replay metadata` |
| `query_plan_fail_cases` | `sp replay case list --plan … --failed` |
| `query_replay_case` | `sp replay case list` |

## Related

* [replay-diff](./replay-diff)
* [Diagnose replay failure](/zh/testing/examples/agent-diagnose-replay)

---

---
url: /en/testing/commands/replay-diff.md
---
# sp replay diff

**When agents use this:** Deep dive on a failed case — diff bodies, compare JSON. For correlated logs use [`sp logs`](./logs).

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `diff get <diffId>` | Base vs test messages (artifact) |
| `compare` | Full-link compare result |
| `mock-tree <replayId>` | Mock tree for replay |
| `noise query` | Query noise rules |
| `noise exclude` | Exclude noise (`-f` + `--confirm`) |
| `realtime …` | Real-time replay queue (see below) |

## Examples

```bash
sp replay diff get diff-abc --out-dir .sp-work --json
sp replay compare --trace-id t1 --replay-id r1 --out-dir .sp-work --json
sp logs --trace-id <trace-id> --since 2026-06-27T10:00:00Z --until 2026-06-27T10:05:00Z --json
sp replay mock-tree replay-uuid --json
```

### JSON output (`diff get`)

```json
{
  "ok": true,
  "command": "replay diff get",
  "data": {
    "artifact": ".sp-work/diff-abc.json",
    "summary": {
      "diffId": "abc",
      "diffResultCode": 1,
      "categoryName": "ResponseBody"
    }
  }
}
```

Artifact contains decoded `baseMsg` and `testMsg` (JSON when parseable).

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `diff get` | GET | `/api/report/queryDiffMsgById/{id}` |
| `compare` | POST | Schedule `/api/compareCase` + report APIs |
| `mock-tree` | GET | `/api/storage/replay-mock-tree/{replayId}` |
| `noise query` | GET | `/api/queryNoise` |
| `noise exclude` | POST | `/api/excludeNoise` |

### Real-time replay

| Subcommand | Path |
|------------|------|
| `realtime create` | `POST /api/createRealTimePlan` |
| `realtime queue pause <planId>` | `GET /api/queue/control/pause?planId=...` (`--confirm`) |
| `realtime queue resume <planId>` | `GET /api/queue/control/resume?planId=...` (`--confirm`) |
| `realtime log` | `POST /api/realtime/log/query` |

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `diff_detail` | `sp replay diff get` |
| `compare_result` | `sp replay compare` |
| `replay_log_overview` | `sp logs --trace-id …` |
| `download_replay_logs` | `sp logs --trace-id …` → file + `grep` |

## Related

* [Output contract](/en/testing/agents/output-contract)
* [Diagnose replay failure](/en/testing/examples/agent-diagnose-replay)

---

---
url: /zh/testing/commands/replay-diff.md
---
# sp replay diff

**When agents use this:** Deep dive on a failed case — diff bodies, compare JSON. For correlated logs use [`sp logs`](./logs).

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `diff get <diffId>` | Base vs test messages (artifact) |
| `compare` | Full-link compare result |
| `mock-tree <replayId>` | Mock tree for replay |
| `noise query` | Query noise rules |
| `noise exclude` | Exclude noise (`-f` + `--confirm`) |
| `realtime …` | Real-time replay queue (see below) |

## Examples

```bash
sp replay diff get diff-abc --out-dir .sp-work --json
sp replay compare --trace-id t1 --replay-id r1 --out-dir .sp-work --json
sp logs --trace-id <trace-id> --since 2026-06-27T10:00:00Z --until 2026-06-27T10:05:00Z --json
sp replay mock-tree replay-uuid --json
```

### JSON output (`diff get`)

```json
{
  "ok": true,
  "command": "replay diff get",
  "data": {
    "artifact": ".sp-work/diff-abc.json",
    "summary": {
      "diffId": "abc",
      "diffResultCode": 1,
      "categoryName": "ResponseBody"
    }
  }
}
```

Artifact contains decoded `baseMsg` and `testMsg` (JSON when parseable).

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `diff get` | GET | `/api/report/queryDiffMsgById/{id}` |
| `compare` | POST | Schedule `/api/compareCase` + report APIs |
| `mock-tree` | GET | `/api/storage/replay-mock-tree/{replayId}` |
| `noise query` | GET | `/api/queryNoise` |
| `noise exclude` | POST | `/api/excludeNoise` |

### Real-time replay

| Subcommand | Path |
|------------|------|
| `realtime create` | `POST /api/createRealTimePlan` |
| `realtime queue pause <planId>` | `GET /api/queue/control/pause?planId=...` (`--confirm`) |
| `realtime queue resume <planId>` | `GET /api/queue/control/resume?planId=...` (`--confirm`) |
| `realtime log` | `POST /api/realtime/log/query` |

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `diff_detail` | `sp replay diff get` |
| `compare_result` | `sp replay compare` |
| `replay_log_overview` | `sp logs --trace-id …` |
| `download_replay_logs` | `sp logs --trace-id …` → 落盘后 `grep` |

## Related

* [Output contract](/zh/testing/agents/output-contract)
* [Diagnose replay failure](/zh/testing/examples/agent-diagnose-replay)

---

---
url: /en/testing/commands/setup.md
---

# sp setup

Command reference. Install flow, Spcode Service, and systemd ops are documented in [Install Softprobe](/en/testing/installation/#spcode-service).

```bash
sp setup
sp setup --api-url http://127.0.0.1:8090
sp setup --install-spcode-service
sp setup --uninstall-spcode-service
```

| Flag | Description |
|------|-------------|
| `--api-url` | Softprobe API URL (backend base URL) |
| `--install-spcode-service` | Install Spcode Service (Linux only) |
| `--uninstall-spcode-service` | Remove Spcode Service |
| `--spcode-service-port` | Listen port (default 4096) |

With `--json`, responses may include `spcodeServiceStatus` and `spcodeServicePort`.

---

---
url: /zh/testing/commands/setup.md
---

# sp setup

命令参考。安装、Spcode Service 与运维说明见 [安装 Softprobe](/zh/testing/installation/#spcode-service)。

```bash
sp setup
sp setup --api-url http://127.0.0.1:8090
sp setup --install-spcode-service
sp setup --uninstall-spcode-service
```

| 参数 | 说明 |
|------|------|
| `--api-url` | 后端 URL |
| `--install-spcode-service` | 安装 Spcode Service（仅 Linux） |
| `--uninstall-spcode-service` | 卸载 Spcode Service |
| `--spcode-service-port` | 监听端口（默认 4096） |

使用 `--json` 时可能包含 `spcodeServiceStatus`、`spcodeServicePort`。

---

---
url: /en/testing/commands/system.md
---
# sp system

## sp system

System-wide key/value config.

| Subcommand | Method | Path |
|------------|--------|------|
| `list` | GET | `/api/system/config/list` |
| `get <key>` | GET | `/api/system/config/query/{key}` |
| `save` | POST | `/api/system/config/save` |
| `delete <key>` | DELETE | `/api/system/config/delete/{key}` |

---

---
url: /zh/testing/commands/system.md
---
# sp system

## sp system

System-wide key/value config.

| Subcommand | Method | Path |
|------------|--------|------|
| `list` | GET | `/api/system/config/list` |
| `get <key>` | GET | `/api/system/config/query/{key}` |
| `save` | POST | `/api/system/config/save` |
| `delete <key>` | DELETE | `/api/system/config/delete/{key}` |

---

---
url: /en/testing/commands/trace.md
---
# sp trace

**When agents use this:** User provides business ID (orderId, caseId) instead of traceId — **preferred discovery path**.

## Synopsis

Attribute-based trace lookup and lightweight trace summary.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `find` | Reverse lookup traces by extraction rule / attr |
| `get <traceId>` | Summary metadata and extracted attrs |
| `stats` | Attr rule statistics (optional) |

## Flags (`find`)

| Flag | Description |
|------|-------------|
| `--app` | Application id (required) |
| `--attr-name` | Rule name (e.g. `orderId`) |
| `--attr-value` | Business value (e.g. `ORD-1234`) |
| `--since` / `--until` | Epoch ms window (default: last 7 days) |
| `--limit` | Max traces (default 20) |

Modes:

1. `--attr-name` + `--attr-value` — precise lookup
2. `--attr-name` only — all traces hit by rule
3. Neither — recent traces hit by any extraction rule

## Examples

```bash
sp trace find --app my-app --attr-name orderId --attr-value ORD-1234 --json
sp trace get 4bf92f3577b34da6a3ce929d0e0e4736 --json
```

### JSON output (`find`)

```json
{
  "ok": true,
  "command": "trace find",
  "data": {
    "items": [
      {
        "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
        "endpoint": "POST /api/order",
        "status": "OK",
        "durationMs": 120,
        "startedAt": "2026-05-18T10:00:00Z"
      }
    ]
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `find` | POST | `/api/traces/find-by-attr` |
| `get` | GET | `/api/traces/{traceId}/summary` |
| `stats` | POST | `/api/traces/attr-rule-statistics` |

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `find_traces_by_attr` | `sp trace find` |
| `get_trace_summary` | `sp trace get` |

## Related

* [Attr → trace lookup](/en/testing/examples/agent-attr-trace-lookup)

---

---
url: /zh/testing/commands/trace.md
---
# sp trace

**When agents use this:** User provides business ID (orderId, caseId) instead of traceId — **preferred discovery path**.

## Synopsis

Attribute-based trace lookup and lightweight trace summary.

## Subcommands

| Subcommand | Description |
|------------|-------------|
| `find` | Reverse lookup traces by extraction rule / attr |
| `get <traceId>` | Summary metadata and extracted attrs |
| `stats` | Attr rule statistics (optional) |

## Flags (`find`)

| Flag | Description |
|------|-------------|
| `--app` | Application id (required) |
| `--attr-name` | Rule name (e.g. `orderId`) |
| `--attr-value` | Business value (e.g. `ORD-1234`) |
| `--since` / `--until` | Epoch ms window (default: last 7 days) |
| `--limit` | Max traces (default 20) |

Modes:

1. `--attr-name` + `--attr-value` — precise lookup
2. `--attr-name` only — all traces hit by rule
3. Neither — recent traces hit by any extraction rule

## Examples

```bash
sp trace find --app my-app --attr-name orderId --attr-value ORD-1234 --json
sp trace get 4bf92f3577b34da6a3ce929d0e0e4736 --json
```

### JSON output (`find`)

```json
{
  "ok": true,
  "command": "trace find",
  "data": {
    "items": [
      {
        "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
        "endpoint": "POST /api/order",
        "status": "OK",
        "durationMs": 120,
        "startedAt": "2026-05-18T10:00:00Z"
      }
    ]
  }
}
```

## REST mapping

| Subcommand | Method | Path |
|------------|--------|------|
| `find` | POST | `/api/traces/find-by-attr` |
| `get` | GET | `/api/traces/{traceId}/summary` |
| `stats` | POST | `/api/traces/attr-rule-statistics` |

## Replaces `sp_api`

| sp\_api | sp |
|--------|-----|
| `find_traces_by_attr` | `sp trace find` |
| `get_trace_summary` | `sp trace get` |

## Related

* [Attr → trace lookup](/zh/testing/examples/agent-attr-trace-lookup)

---

---
url: /zh/platform/advanced-guides/agent-architecture.md
---
# SP‑Istio Agent 架构

该页面介绍服务端采集组件 SP‑Istio Agent 的工作原理，包括如何注入、采集哪些数据、以及如何高效安全地传输遥测数据。

## 概览

* 部署：通过 WasmPlugin 注入到 Istio 的 Envoy sidecar
* 实现：Rust + WebAssembly (Wasm)
* 采集范围：HTTP 请求/响应、上下文头、状态码、延迟与自定义标签
* 传输：异步批量发送到 Softprobe 后端（带重试与队列）

## 工作流程

1. 请求进入 Envoy → Wasm 过滤器拦截并解析关键字段
2. `collectionRules` 决定是否采集事件
3. 构建事件并追加到内部缓冲/队列
4. 批量异步发送到后端，携带 `public_key` 进行认证
5. 后端存储与索引，仪表盘提供会话视图与跨服务调用树

## 配置要点

* `traffic_direction`：选择 `server`（入站）或 `client`（出站）
* `collectionRules`：使用正则匹配 `host`/`path` 精准控制采集范围
* 服务发现：通过 EnvoyFilter（Lua）自动注入 `x-sp-service-name`
* 外部通信：使用 ServiceEntry + DestinationRule 授权安全的 egress 到 Softprobe 后端

## 性能与可靠性

* 低开销：Wasm 在 Envoy 内运行，对业务代码无侵入
* 异步传输：最小化对请求的影响，支持背压与重试
* 资源控制：限速与队列配置，避免峰值压力

## 安全

* 传输加密：TLS，严格限制 egress 到允许的后端域名
* 认证：使用 public key 验证租户身份
* 隐私与合规：字段掩码/脱敏与可配置采集范围

## 与 OTLP 协作

* 可并行使用：Agent 提供零侵入 HTTP 采集；OTLP 接收来自 SDK/Collector 的数据
* 后端统一索引与关联，支持跨数据源的会话与追踪视图

---

---
url: /zh/testing/agents/versioning.md
---
# Versioning and stability

## Binary version

`sp version` prints the CLI build version (semver). Agents may log this on every session for support correlation.

## Command argv stability

| Change type | Policy |
|-------------|--------|
| New subcommand | Allowed anytime |
| New optional flag | Allowed anytime |
| Renaming subcommand or flag | Deprecate for one minor release; print stderr warning |
| Removing subcommand or flag | Major version bump only |

Document deprecations in release notes and in command `--help` text.

## JSON field stability

| Change type | Policy |
|-------------|--------|
| New fields in `data` | Allowed; agents must ignore unknown fields |
| Renaming or removing fields | Major version bump, or add `dataVersion` in envelope |
| Changing field types | Major version bump |

Recommended: include `"schemaVersion": 1` in the CLI envelope in a future release.

## API backend compatibility

The CLI targets **sp-backend** unified deployment. Backend URL is configurable; the CLI does not pin to separate storage/schedule hostnames (those are internal to sp-backend).

When backend APIs change, update [API mapping](/zh/testing/reference/api-mapping) in the same PR as the CLI implementation.

---

---
url: /zh/testing/webhook-and-ci.md
---

# Webhook 触发回放与 CI/CD 集成

在部署或合并后自动对**测试环境**中的服务发起录制回放，并根据对比结果决定流水线是否通过。本节说明两种触发方式、如何拿到结果，以及 GitHub Actions 与 Jenkins 的集成模式。

::: tip 先录制，再自动化回放
Webhook 与 CI 都不会替你产生用例。请先走通核心流程 [录制](/zh/testing/recording) 与 [回放](/zh/testing/replay-and-diff)（`appId`、策略、`targetEnv` 可达），再把它交给自动化。
:::

## 该用 `sp` CLI 还是 REST API？

| 场景 | 推荐 |
|------|------|
| **GitHub Actions、Jenkins、GitLab CI** 等完整流水线 | **`sp` CLI**（`--json`、稳定退出码、内置轮询） |
| **部署钩子**（Argo CD、K8s Job、脚本）只需「部署成功 → 立刻开跑」 | **GET Webhook**（`curl` 一行触发） |
| **自研编排**、已有 HTTP 客户端、需精细控制请求体 | **REST**（`POST /api/createPlan` + 报告类 API） |

**原则：** 流水线里负责「创建计划 → 等待结束 → 拉失败用例 → 判定通过/失败」时，优先用 **`sp replay run --watch`** 与 **`sp replay case list --failed`**，与 [CLI 输出约定](/zh/testing/agents/output-contract) 一致。Webhook GET 适合**只负责触发**；结果查询仍建议在同一流水线后续步骤用 `sp` 或带 `access-token` 的报告 API。

CLI 细节见 [replay 命令](/zh/testing/commands/replay)、[认证](/zh/testing/agents/authentication)。

## 架构（简述）

```mermaid
sequenceDiagram
  participant CI as CI/CD 或 Webhook
  participant API as sp-backend（调度 + 报告）
  participant SUT as 测试环境服务 targetEnv
  CI->>API: 创建回放计划 createPlan
  API->>SUT: 发送录制的入口 HTTP
  SUT-->>API: 回放流量与对比结果
  CI->>API: 轮询 progress / 查询失败用例
```

* **`SP_API_URL`**：sp-backend 地址（存储、调度、报告），**不是**被测服务 URL。
* **`targetEnv`**：被测服务基础 URL（如 `http://order-service.test.svc:8080`），与 `SP_API_URL` 不可混淆 — 见 [CLI 概念：targetEnv](/zh/testing/agents/concepts#replay-target-url-targetenv)。

## 方式一：Webhook（GET）触发回放

调度服务提供 **GET** `/api/createPlan`，供部署 Webhook、监控告警或简单 `curl` 调用。服务端将 `operator` 记为 `Webhook`，默认按 **应用维度** 选取最近 **24 小时**内录制的用例（可通过时间参数缩小范围）。

### 请求示例

```bash
curl -sS -G "${SP_API_URL}/api/createPlan" \
  -H "access-token: ${SP_TOKEN}" \
  --data-urlencode "appId=YOUR_APP_ID" \
  --data-urlencode "targetEnv=http://your-service.test:8080" \
  --data-urlencode "planName=post-deploy-smoke" \
  --data-urlencode "caseCountLimit=50" \
  --data-urlencode "enableMock=true"
```

### 常用查询参数

| 参数 | 必填 | 说明 |
|------|------|------|
| `appId` | 是 | 已注册应用 id |
| `targetEnv` | 是 | 测试实例基础 URL（含 `http://` 或 `https://`） |
| `caseSourceFrom` | 否 | 用例时间窗起点（Unix **毫秒**）；默认约 24 小时前 |
| `caseSourceTo` | 否 | 用例时间窗终点（毫秒）；默认当前时间 |
| `caseCountLimit` | 否 | 最多回放条数 |
| `planName` | 否 | 计划显示名 |
| `operationIds` | 否 | 可重复，仅回放指定操作 id |
| `enableMock` | 否 | `true` / `false`，默认按服务端策略 |
| `caseTags` | 否 | JSON 字符串，按标签过滤用例 |

### 响应

成功时 `result` 为 `1`，`data` 中含 **`replayPlanId`**（即后续轮询用的 `planId`）：

```json
{
  "result": 1,
  "desc": "success",
  "data": {
    "replayPlanId": "plan-abc123"
  }
}
```

::: warning
GET Webhook **只创建计划**，不会等待回放结束。请在 CI 后续步骤轮询进度或查询报告，见下文「获取结果」。
:::

::: info POST 与 GET
`POST /api/createPlan`（`sp replay run` 使用）支持完整 `BuildReplayPlanRequest`（时间窗、`operationCaseInfoList`、压测参数等）。复杂选例请用 POST 或 CLI，不要用 GET 勉强拼参数。
:::

## 方式二：CLI / REST 创建计划（推荐用于 CI）

与 [回放与对比](/zh/testing/replay-and-diff) 相同，CI 中常用：

```bash
export SP_API_URL=https://your-tenant.softprobe.ai   # 或内网 sp-backend :8090
export SP_TOKEN="${SP_TOKEN}"                         # 来自密钥库，勿写入仓库

sp replay run \
  --app YOUR_APP_ID \
  --env http://your-service.test:8080 \
  --from -24h \
  --name "ci-${GITHUB_SHA:-build}" \
  --watch \
  --json
```

`--watch` 会在创建计划后轮询 `GET /api/progress?planId=…`，直到进度结束。回放**对比失败**时 CLI 仍可能以 **0** 退出 — 流水线必须再检查失败用例（见下节）。

等价的 REST 创建（需 `access-token` 头）：

```http
POST /api/createPlan
Content-Type: application/json
access-token: <JWT>

{
  "appId": "YOUR_APP_ID",
  "targetEnv": "http://your-service.test:8080",
  "caseSourceFrom": 1717000000000,
  "caseSourceTo": 1717086400000,
  "enableMock": true,
  "planName": "ci-build-42"
}
```

## 获取回放结果

### 1. 轮询进度

```bash
sp replay status plan-abc123 --watch --json
```

REST：`GET /api/progress?planId=plan-abc123`（同一 `SP_API_URL`）。返回体含 `percent` 等字段；`percent` 达到 100 表示调度侧执行完毕。

### 2. 判定通过 / 失败（CI 门禁）

进度结束后，查询是否存在失败用例：

```bash
sp replay case list --plan plan-abc123 --failed --limit 1 --json
```

若 `data.items` 非空，或配合 `jq` 统计条数大于 0，则令流水线 **失败**。需要差异详情时：

```bash
sp replay diff get <diffId> --out-dir .sp-work --json
sp diagnose replay plan-abc123 --failed-only --out-dir .sp-work --json
```

参见 [示例：诊断回放失败](/zh/testing/examples/agent-diagnose-replay)。

### 3. 仪表盘（人工）

可在 Softprobe 工作台或仪表盘中打开同一 `planId` 查看差异树；自动化应以 CLI/`--json` 为准。

## GitHub Actions 示例

在**测试环境已部署且 Agent 已挂载**的 job 中运行（Secrets：`SP_API_URL`、`SP_TOKEN`）。

```yaml
name: Softprobe replay gate

on:
  workflow_dispatch:
  push:
    branches: [main]

jobs:
  replay-regression:
    runs-on: ubuntu-latest
    env:
      SP_API_URL: ${{ secrets.SP_API_URL }}
      SP_TOKEN: ${{ secrets.SP_TOKEN }}
      SP_APP_ID: ${{ vars.SP_APP_ID }}
      SP_TARGET_ENV: http://my-service.test:8080
    steps:
      - name: Install sp
        run: |
          curl -fsSL -o sp "${SP_DOWNLOAD_URL:-https://install.softprobe.ai/artifacts/sp/latest}/sp-linux-amd64"
          chmod +x sp && sudo mv sp /usr/local/bin/

      - name: Preflight
        run: |
          sp health --json
          sp record case list --app "$SP_APP_ID" --since -24h --json

      - name: Run replay and wait
        id: replay
        run: |
          sp replay run \
            --app "$SP_APP_ID" \
            --env "$SP_TARGET_ENV" \
            --from -24h \
            --name "gha-${GITHUB_SHA}" \
            --watch \
            --json | tee replay.ndjson
          PLAN_ID=$(grep '"command":"replay run"' replay.ndjson | tail -1 | jq -r '.data.planId // .data.replayPlanId')
          echo "plan_id=$PLAN_ID" >> "$GITHUB_OUTPUT"

      - name: Fail if any case failed
        run: |
          COUNT=$(sp replay case list --plan "${{ steps.replay.outputs.plan_id }}" --failed --json \
            | jq '[.data.items[]?] | length')
          if [ "$COUNT" -gt 0 ]; then
            echo "Replay had $COUNT failed case(s)"
            exit 1
          fi
```

可选：在 **deploy** workflow 末尾用 Webhook 仅触发（不等待），在独立 job 中做 `status --watch` 与门禁。

```yaml
      - name: Trigger replay webhook (fire-and-forget)
        run: |
          curl -fsS -G "$SP_API_URL/api/createPlan" \
            -H "access-token: $SP_TOKEN" \
            --data-urlencode "appId=$SP_APP_ID" \
            --data-urlencode "targetEnv=$SP_TARGET_ENV" \
            --data-urlencode "planName=deploy-${{ github.run_id }}"
```

策略校验可合并到 PR，见 [示例：CI 策略门禁](/zh/testing/examples/ci-policy-gate)。

## Jenkins 示例

使用 **Credentials** 绑定 `SP_TOKEN`，在 Declarative Pipeline 中：

```groovy
pipeline {
  agent any
  environment {
    SP_API_URL = credentials('sp-api-url')
    SP_TOKEN   = credentials('sp-token')
    SP_APP_ID  = 'YOUR_APP_ID'
    SP_TARGET_ENV = 'http://my-service.test:8080'
  }
  stages {
    stage('Replay') {
      steps {
        sh '''
          sp replay run \
            --app "$SP_APP_ID" \
            --env "$SP_TARGET_ENV" \
            --from -24h \
            --name "jenkins-${BUILD_NUMBER}" \
            --watch \
            --json > replay.ndjson
          PLAN_ID=$(grep '"command":"replay run"' replay.ndjson | tail -1 | jq -r '.data.planId // .data.replayPlanId')
          FAILED=$(sp replay case list --plan "$PLAN_ID" --failed --json | jq '[.data.items[]?] | length')
          test "$FAILED" -eq 0
        '''
      }
    }
  }
}
```

Webhook 触发可放在 **Post-deployment** 步骤：`curl -G …/api/createPlan`；在后续 stage 用 `sp replay status <planId> --watch` 等待并门禁。

安装 `sp`：在 agent 镜像中预装，或 `curl` 安装脚本与 GitHub Actions 相同。

## 安全与运维建议

* **`SP_TOKEN` 仅放在 CI 密钥库**，不要提交到 Git；轮换泄露的令牌 — [认证](/zh/testing/agents/authentication)。
* Webhook URL 若暴露在公网，应配合网络策略、IP 允许列表，并始终携带 **`access-token`**（与 CLI 相同）。
* 回放会向 **`targetEnv` 发送真实 HTTP**；仅在测试/预发实例上配置 Webhook，勿指向生产入口 — [回放与对比](/zh/testing/replay-and-diff)。
* 回放机上**降低或关闭录制**，避免回放过程中再录一套数据。

## 相关文档

* [回放与对比](/zh/testing/replay-and-diff)
* [快速开始](/zh/testing/getting-started)
* [CLI：replay](/zh/testing/commands/replay)
* [REST API 映射](/zh/testing/reference/api-mapping)
* [CLI 快速入门](/zh/testing/getting-started)

---

---
url: /zh/testing/download-java-agent.md
---

# 下载 Java Agent

下载最新版 Java Agent：

```bash
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
```

如需固定版本，将 `latest` 替换为具体版本：

```bash
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/v4.3.9/sp-agent.jar
```

## 可用版本

`latest` 是可变别名，始终指向当前 Java Agent。以下不可变版本来自 `install.softprobe.ai`。

命令行获取版本列表：

```bash
curl -fsSL https://install.softprobe.ai/artifacts/agent/versions.json
```

## 下一步

挂载 Agent 并配置 JVM 参数：[安装与配置](/zh/testing/java-agent)。

---

---
url: /zh/platform/advanced-guides/concepts.md
---
# 了解 Softprobe 核心概念

学习 Softprobe 的工作原理、系统架构以及实现高效可观测性的核心原则。

## 📋 概述

Softprobe 面向现代云原生应用的全栈可观测平台，提供：

* **全栈追踪**：覆盖服务与用户交互
* **性能监控**：丰富上下文与关联分析
* **认证中心化架构**：基于公钥认证
* **Kubernetes 原生集成**：无缝部署

## 🏗️ Softprobe 如何工作

### 架构总览

Softprobe 采用分布式架构：

```
┌─────────────────┐    ┌───────────────────┐    ┌────────────────────┐
│   业务应用      │    │   SP-Istio Agent  │    │  Softprobe 后端    │
│                 │    │                    │    │                    │
│  ┌─────────────┐│    │  ┌───────────────┐ │    │  ┌──────────────┐  │
│  │ SESSIFY    ├───────►│  WASM Plugin  ├───────►│  Collector    │  │
│  │ (前端)     ││    │  │ (Service Mesh)│ │    │  │              │  │
│  └─────────────┘│    │  └───────────────┘ │    │  └──────────────┘  │
│                 │    │                    │    │                    │
│  ┌─────────────┐│    │  ┌───────────────┐ │    │  ┌──────────────┐  │
│  │ 后端服务    ├───────►│  Envoy Filter ├───────►│  Processing   │  │
│  │             ││    │  │ (流量采集)    │ │    │  │ Engine       │  │
│  └─────────────┘│    │  └───────────────┘ │    │  └──────────────┘  │
└─────────────────┘    └───────────────────┘    └────────────────────┘
       │                         │                         │
       │                         │                         │
       └─────────────────────────┴─────────────────────────┘
                    认证的数据管道
```

### 关键组件

#### 1. SP-Istio Agent（服务网格层）

* **WASM 插件**：运行在 Istio Sidecar 中，捕获服务间流量
* **Envoy Filter**：注入服务标识头并管理流量采集
* **零代码修改**：无需改动现有应用

#### 2. SESSIFY（前端层）

* **自动埋点**：捕获用户交互、性能与错误
* **会话管理**：关联前后端请求
* **轻量**：对前端性能影响极小

#### 3. Softprobe 后端

* **Collector**：接收并处理各源数据
* **处理引擎**：关联、丰富并存储遥测
* **仪表盘**：可视化洞察与分析能力

## 🔄 数据流

### 请求在系统中的旅程

1. **用户交互**：用户与 Web 应用互动
2. **前端采集**：SESSIFY 捕获性能与行为
3. **后端请求**：应用发起服务间调用
4. **网格采集**：SP-Istio Agent 在 Service Mesh 中捕获流量
5. **数据丰富**：添加服务身份与上下文
6. **传输与认证**：数据经公钥签名认证后发送至 Softprobe 后端
7. **关联与分析**：后端关联前后端数据
8. **可视化**：在仪表盘中展示洞察

### 示例：用户登录流程

```
1. 用户点击“登录”按钮
   → SESSIFY 捕获点击事件

2. 前端向后端发送登录请求
   → SESSIFY 追踪该 API 调用

3. 后端服务处理登录
   → SP-Istio Agent 捕获服务流量

4. 后端访问用户数据库
   → SP-Istio Agent 捕获数据库调用

5. 所有数据以会话 ID 关联
   → Softprobe 展示完整用户旅程
```

## 🔐 认证与数据传输

### 公钥认证

Softprobe 采用非对称加密进行认证：

* **配置不含秘密**：公钥非敏感
* **密码学证明**：请求签名与校验
* **易旋转**：公钥可无停机替换
* **审计轨迹**：认证事件可审计

### 数据处理与传输

* **TLS 加密**：传输过程加密
* **不存敏感信息**：Agent 不持久化敏感数据
* **最小权限**：以最小权限运行
* **策略兼容**：适配常见安全策略

## 🎯 核心概念

### 分布式追踪

Softprobe 提供端到端追踪，覆盖：

* **前端应用**：用户交互与浏览器性能
* **后端服务**：服务间通信
* **外部依赖**：数据库调用、外部 API
* **基础设施**：Kubernetes Pod 与节点上下文

### 服务网格集成

基于 Istio 能力：

* **自动 Sidecar 注入**：无需改代码
* **流量捕获**：入站与出站
* **服务身份**：自动服务发现与命名
* **零信任策略**：兼容网格安全策略

### 可观测数据模型

Softprobe 采集并关联：

* **Span**：单个操作单元
* **Trace**：跨服务的完整请求旅程
* **Metrics**：性能与资源使用
* **Logs**：应用与基础设施日志
* **Events**：用户交互与业务事件

## ⚡ 性能考量

### 高效数据采集

* **采样**：可配置采样率
* **Body 截断**：限制负载大小
* **本地处理**：Agent 轻量处理
* **批量传输**：网络高效

### 资源优化

* **内存友好**：WASM 插件开销小
* **CPU 轻量**：算法优化
* **网络高效**：压缩与批量
* **可扩展架构**：支持高流量生产环境

## 🚀 入门概念

### 开发 vs 生产

* **开发**：本地集群与样例数据
* **预发**：生产拟合验证
* **生产**：全面部署与优化配置

### 环境策略

* **公钥分离**：为不同环境使用不同公钥
* **定向部署**：仅部署到必要的命名空间
* **渐进式发布**：先监控后扩大采集

## 📊 预期收益

### 立竿见影

* **服务依赖图谱**：自动发现
* **性能基线**：建立常态
* **错误检测**：识别失败请求与服务
* **用户体验洞察**：理解真实行为

### 高阶能力

* **根因分析**：深入性能问题
* **容量规划**：识别资源瓶颈
* **业务指标**：技术与业务关联
* **主动监控**：异常前置告警

## 🔧 集成模式

### 微服务架构

适合分布式系统：

* 多服务交互
* 复杂依赖关系
* 跨服务故障排查
* 性能优化需求

### 单体应用

同样适用：

* 内部组件追踪
* 外部依赖监控
* 用户体验关联
* 性能优化机会

### 混合环境

跨多样基础设施：

* Kubernetes 集群
* 虚拟机
* 云服务
* 私有部署

## 🎯 下一步

理解概念后：

1. **账户设置**：参见 [账户与公钥管理](../getting-started/account-setup)
2. **快速试用**：参见 [快速入门](../getting-started/quick-start)
3. **生产部署**：参见 [生产安装指南](../deployment/installation)
4. **高级配置**：参见 [配置指南](../configuration/config)

## ❓ 常见问题

### 问：Softprobe 是否影响应用性能？

**答**：影响极小。SP-Istio Agent 使用高效的 WASM 插件，SESSIFY 也非常轻量。可通过采样配置平衡洞察与开销。

### 问：我的数据如何传输与认证？

**答**：所有数据在传输中采用 TLS 加密，并通过公钥签名进行认证。Kubernetes 配置中不包含敏感私钥。

### 问：可与现有监控工具配合吗？

**答**：可以。Softprobe 提供更深的分布式追踪与用户体验关联，补充现有工具。

### 问：安装后多久能看到效果？

**答**：立即。只要 Agent 部署完成并有业务流量，数据就会进入仪表盘。

***

## 📚 参考资料

* [Istio 文档](https://istio.io/latest/docs/)
* [OpenTelemetry 概念](https://opentelemetry.io/docs/concepts/)

---

---
url: /zh/testing/installation/upgrade.md
---

# 升级

使用 `sp upgrade` 升级 Softprobe：

```bash
sp upgrade
```

升级复用全局安装器，并以一个 Softprobe 安装更新 CLI、Java Agent 和内部编码引擎。任何组件的部分失败都会被明确报出。

若已安装 **Spcode Service**（`spcode-web.service`），升级 `sp` 或 `spcode` 后请重启服务：

```bash
sudo systemctl restart spcode-web.service
journalctl -u spcode-web -n 20 --no-pager
```

---

---
url: /zh/testing/reference.md
---

# 参考

Testing 参考内容包括 API 映射、JSON 输出、退出码和回放日志标记。

---

---
url: /zh/testing/replay-and-diff.md
---

# 回放与对比

回放把 [录制](/zh/testing/recording) 攒下的用例变成一次**回归运行**：把当初的入口请求原样打到你的**测试实例**上，依赖调用（数据库、外部 HTTP…）由录制数据自动 Mock，跑完自动对比录制响应和回放响应，给出通过/失败。

继续用 `order-service` 的例子：生产流量已经录了一批用例，现在要在测试环境验证新版本代码有没有回归。

## 第 1 步 · 准备测试实例

在测试环境把**新版本**的应用跑起来，同样挂 Agent、用同一个 `appId`：

```bash
java -javaagent:sp-agent.jar \
     -Dsp.app.id=<你的 appId> \
     -Dsp.api.url=http://<后端主机>:8090 \
     -jar order-service-new.jar
```

记下它的访问地址，比如 `http://order-service.test:8080`——这就是 **`targetEnv`**，回放流量的目的地。

::: warning 在生产录制，在测试环境回放
回放会向 `targetEnv` 发送**真实 HTTP 请求**（只有下游依赖被 Mock），所以除非明确接受风险，回放目标应该是非生产实例。同时把回放机上的录制关掉或调到极低，避免回放流量又被录一遍污染用例库。
:::

## 第 2 步 · 发起回放

```bash
sp replay run --app <你的 appId> --env http://order-service.test:8080 --json
```

命令返回一个 `planId`。盯着它跑完：

```bash
sp replay status <planId> --watch
```

::: tip 两个 URL 别混
`--env`（`targetEnv`）是**被测服务**的地址；`SP_API_URL` 是 **sp-backend 后端服务**的地址。混淆二者是最常见的集成错误——见 [CLI 概念](/zh/testing/agents/concepts#replay-target-url-targetenv)。
:::

回放期间发生的事：调度服务把用例的 Mock 预加载进 Redis，逐条向 `targetEnv` 重发录制的入口请求；你的服务真实执行业务代码，但每次调依赖时 Agent 返回**录制的响应**，不碰真实数据库和外部系统；回放侧流量被存下来，与录制侧自动对比。

sp-backend 在每次入口请求发出前记录 **`Replay send start`**，发出后记录 **`Replay send done`** / **`Replay send failed`**——这是回放 HTTP 派发的进入/退出边界，日志排查见 [Replay send / 日志标记](/zh/testing/reference/replay-send-log-markers)。

## 第 3 步 · 读结果

对比没有发现实质差异的用例**通过**。**失败**的用例会给出差异场景：

* **值差异** — 依赖调了，但响应体不同
* **缺调用** — 录制时调过的依赖，回放时没调
* **主响应差异** — 入口响应与录制不一致

命令行快速排查：

```bash
sp replay case list --plan <planId> --json     # 哪些用例失败
sp diagnose replay <planId> --failed-only --out-dir .sp-work --json   # 失败详情 + diff 产物落盘
```

拿到某条差异的 `diffId` 后，看单条完整 diff：`sp replay diff get <diffId> --out-dir .sp-work --json`。

## 有失败？先别当 bug

**大多数失败不是 bug。** 时间戳、随机 ID、Pod IP、会话令牌每次运行都会变——它们永远会"不一样"，但并没有出错。接下来两步就是干这个的：

* **[审查差异](/zh/testing/review-diffs-in-the-web-ui)** — 在工作台里逐条看 diff，接受不是 bug 的差异，让真失败露出来
* **[配置对比规则](/zh/testing/compare-rules-web-ui)** — 把"永远会变"的字段配成规则，以后每次回放都不再误报

规则也能用 YAML 声明（`sp policy compare`），方便进 CI 和 GitOps——见 [策略 YAML 指南 · CompareRulePolicy](/zh/testing/policy-yaml-guide#comparerulepolicy)。

## 术语速查

| 概念 | 含义 |
|------|------|
| `targetEnv` / `--env` | 接收回放入口流量的服务基础 URL |
| `planId` | 整次运行的容器 |
| `planItemId` | 计划内的一个操作（API 路径） |
| `replayId` | 单个用例的一次回放执行 |
| Case | 一条录制的入口请求及其依赖 mocker |

## 回放范围

回放哪些用例由计划请求的时间范围、操作过滤，以及录制策略的 `operations` 包含/排除决定。想扩大覆盖，回到 [录制](/zh/testing/recording) 录更多流量。

## 自动化

人工看差异用工作台；CI 与 AI 代理用 `sp diagnose replay <planId> --json` 和 [输出约定](/zh/testing/agents/output-contract) 的 `--out-dir` 产物。部署后自动触发回放、流水线门禁的完整示例见 [Webhook 与 CI/CD](/zh/testing/webhook-and-ci)。

## 下一步

回放跑完有失败的用例 → **[审查差异](/zh/testing/review-diffs-in-the-web-ui)**：看懂它们，把噪声清掉。

---

---
url: /zh/testing/reference/replay-send-log-markers.md
---
# 回放发送日志标记

> 本页翻译可能滞后于英文版，如有出入以[英文版](/en/testing/reference/replay-send-log-markers)为准。

**agent 何时使用：** 解读每次回放 HTTP 分发前后 sp-backend 的 schedule 日志——这是回放调度与你的应用之间关键的进入和退出边界。

这些结构化日志行由 **sp-schedule**（`DefaultHttpReplaySender`）在每次入口 HTTP 调用之前和之后立即输出。当 `sp.source` 为 **`backend`** 且 `service_name` 通常为 **`sp-backend`** 时，它们会出现在关联日志查询（`sp logs --trace-id …`、统一 trace 日志下载）的结果中。

用 case 的 **`traceId`** 查询日志（当关联成功时，该值与消息中的 `recordedTraceId` 相同）。参见 [sp logs 命令](../commands/logs.md)。

***

## 消息前缀

| 前缀 | 级别 | 含义 |
|--------|----------|------|
| `Replay send start:` | INFO | schedule 即将发起已录制的入口 HTTP 请求 |
| `Replay send done:` | INFO | HTTP 交互完成；已记录响应状态和耗时 |
| `Replay send failed:` | WARN | HTTP 交互在正常完成之前失败 |

每个前缀后面跟着以逗号分隔的 **`key=value`** 对（SLF4J 结构化消息，而非 JSON）。

***

## 结构化字段

| 字段 | 出现于 | 说明 |
|-------|-----|------|
| `planId` | start、done、failed | 该批次运行的回放**计划** id |
| `targetEnv` | start、done、failed | 当发往配置的回放测试 URL（`--env` / `targetEnv`）时为 `true`；录制侧发送时为 `false` |
| `method` | start、done、failed | HTTP 方法（`GET`、`POST`、……） |
| `url` | start、done、failed | schedule 调用的完整 URL（包含路径和查询字符串） |
| `recordedTraceId` | start、done、failed | 来自**录制**的 W3C trace id；关联成功时应与日志行上的 `trace_id` 匹配 |
| `spanId` | start、done、failed | 发送路径上处于活动状态的 OpenTelemetry span id |
| `httpStatus` | done、failed | 响应的 HTTP 状态码，或在未收到响应（连接错误、超时、DNS 失败）时为 **`-1`** |
| `durationMs` | done、failed | HTTP 交互的墙钟毫秒数 |
| `error` | 仅 failed | 发送未成功完成时的异常或错误消息 |

***

## 示例日志行

```text
Replay send start: planId=6a3f2aad59f0c4655b0f99da, targetEnv=true, method=POST, url=http://order-service.test:8080/api/orders, recordedTraceId=2057ad46a7ce03d3955385f2a4142d29, spanId=8d10c94a2a6f4e11

Replay send done: planId=6a3f2aad59f0c4655b0f99da, targetEnv=true, method=POST, url=http://order-service.test:8080/api/orders, recordedTraceId=2057ad46a7ce03d3955385f2a4142d29, spanId=8d10c94a2a6f4e11, httpStatus=200, durationMs=142

Replay send failed: planId=6a3f2aad59f0c4655b0f99da, targetEnv=true, method=POST, url=http://order-service.test:8080/api/orders, recordedTraceId=2057ad46a7ce03d3955385f2a4142d29, spanId=8d10c94a2a6f4e11, httpStatus=-1, durationMs=30001, error=Connection refused
```

***

## 如何使用

1. 从 `sp replay case list --plan <planId> --failed --json` 或回放元数据获取 **`traceId`**。
2. 拉取失败点附近时间窗口的关联日志（参见[诊断回放失败示例](../examples/agent-diagnose-replay.md)）。
3. 过滤 **`sp.source=backend`** 且 **`body`** 中包含 `Replay send start`、`Replay send done` 或 `Replay send failed` 的行。

| 观察到的现象 | 可能原因 |
|-------------|--------------|
| 窗口内没有 `Replay send start` | case 可能尚未到达 HTTP 发送阶段（预加载、调度或计划被取消）——先检查计划状态 |
| 有 `start` 但没有匹配的 `done` 或 `failed` | 发送可能仍在进行中，或日志被截断——加宽时间窗口 |
| `failed` 且 `httpStatus=-1` | 到达 **`targetEnv`** 时出现网络问题或超时——核查 `--env` URL、DNS、防火墙，以及应用是否在运行 |
| `failed` 且为 4xx/5xx | 应用返回了错误的 HTTP 状态码——检查 `start` 时间戳之后的 agent 和应用日志 |
| `done` 且为 2xx 但 compare 仍然失败 | 入口请求已到达应用；检查 `done` 日志行之后的 agent mock/compare 行为 |

***

## 相关的 ERROR 日志行（同一次发送尝试）

以下**不是**进入/退出标记，但常常出现在同一次失败的发送中：

| 消息前缀 | 级别 | 含义 |
|----------------|----------|------|
| `Replay send error:` | ERROR | 发送失败后的堆栈跟踪（紧随 `Replay send failed`） |
| `Replay send result invalid:` | ERROR | 收到了响应，但缺少用于提取结果所需的 trace/replay 头 |

***

## 相关

* [回放与对比](/zh/testing/replay-and-diff)
* [诊断回放失败示例](../examples/agent-diagnose-replay.md)
* [sp logs 命令](../commands/logs.md)

---

---
url: /zh/platform/billing/pricing.md
---
# 套餐与价格

Softprobe 提供三种套餐以满足不同阶段与规模：

* 免费版（Free）— 固定配额，适合评估体验
* 商业版（Business/Pro）— 固定配额，适合成长型团队
* 企业版（Enterprise）— 按用量计费，随业务弹性扩展

命名已统一：仪表盘中的 “Pro” 与后端系统中的 “Business” 是同一套餐。

## 配额与限制（示例）

* 免费版：基础存储、席位与 API 速率限制；超额后受限
* 商业版：更大的存储与更高配额；可在超额时引导升级或计费
* 企业版：无固定配额，按实际用量计费

## 计费模型

* 免费版：￥0/月
* 商业版（固定配额）：固定月费
* 企业版（按用量）：按存储、API 调用、月活等维度计费

如需企业定制或更高配额，请联系 support@softprobe.ai。

---

---
url: /zh/testing/installation/server.md
---

# 安装 Softprobe 服务端

使用 Helm 在 Kubernetes 上安装统一的 Softprobe 后端。Chart 会在集群内部署 **Redis**，并选择部署**内置 MongoDB** 或连接您**已有的 MongoDB** 服务器。

Chart **v4.3.x+** 默认启用[统一日志管道](#unified-log-pipeline)（Vector、Parquet PVC、压缩）。全新安装只需配置下方的 MongoDB 与加密密钥——无需单独的 `logPipeline` 块。

**前置条件：** Kubernetes 1.24+、Helm 3.x、Softprobe 提供的 GCR 拉取凭证，以及用于静态载荷加密的 `encryption.secretKey`。

若使用**内置** MongoDB，集群需有默认或已配置的 `StorageClass` 供 MongoDB PVC 使用。

## MongoDB 模式（二选一）

在 values 文件中**仅配置一种**模式。若两者都设或都未设，`helm install` 会失败。

| 模式 | values 中设置 | Chart 是否部署 MongoDB？ |
|------|---------------|------------------------|
| **内置** | `mongodb.bundled.auth.password` | 是 — Deployment、Service、PVC |
| **外部** | `mongodb.connectionString` | 否 |

**共享外部 MongoDB：** 多个 Helm release 可共用同一 MongoDB 主机。请在各连接字符串中使用**唯一的数据库名**（例如 `acme_prod_sp_storage_db`）。

下载示例 values 文件（始终为当前版本）：

[values.example.yaml](https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/latest/values.example.yaml)

## 安装

### 1. Helm 仓库与命名空间

```bash
helm repo add softprobe \
  https://storage.googleapis.com/softprobe-published-files/helm/sp-backend
helm repo update

kubectl create namespace softprobe
```

### 2. GCR 拉取 Secret

```bash
kubectl create secret docker-registry softprobe-gcr-pull \
  --docker-server=https://gcr.io \
  --docker-username=_json_key \
  --docker-password="$(cat softprobe-registry-puller.json)" \
  --namespace softprobe
```

::: warning
需同时配置 `gcr.io` 与 `https://gcr.io` 认证项。若 plain `kubectl create secret docker-registry` 导致 `ImagePullBackOff`，请参阅 Chart README。
:::

### 3. Values 文件

将示例复制为 `values.yaml` 并按环境编辑。**在下方 MongoDB 块中二选一。**

#### 模式 A — 内置 MongoDB（集群内）

Chart 与 sp-backend 一同部署 MongoDB 7 与 Redis 7。

```yaml
image:
  tag: "v4.3.10"
  pullSecrets:
    - name: softprobe-gcr-pull

mongodb:
  bundled:
    auth:
      password: "CHANGE_ME_mongo_password"
    # 可选：storageClass、storageSize、resources、placement — 见 values.example.yaml

encryption:
  enabled: true
  secretKey: "CHANGE_ME_base64_32_byte_key"
```

#### 模式 B — 外部 MongoDB

使用已有 MongoDB 服务器。**不要**设置 `mongodb.bundled.auth.password`。

```yaml
image:
  tag: "v4.3.10"
  pullSecrets:
    - name: softprobe-gcr-pull

mongodb:
  connectionString: "mongodb://USER:PASS@mongo.host:27017/your_release_sp_storage_db?authSource=admin"

encryption:
  enabled: true
  secretKey: "CHANGE_ME_base64_32_byte_key"
```

### 4. Helm 安装

使用 Softprobe 发布版本对应的 Chart 版本与镜像 tag（`v4.3.10` → Chart `4.3.10`，镜像 `v4.3.10`）。

```bash
helm install softprobe softprobe/sp-backend \
  --version 4.3.10 \
  --namespace softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

## 验证

```bash
kubectl get pods -n softprobe
kubectl port-forward -n softprobe svc/softprobe-sp-backend 8090:8090
curl -s http://127.0.0.1:8090/actuator/health
```

**内置模式：** 应看到 `mongodb`、`redis`、`sp-backend`，以及（Chart v4.3.x+）`log-vector` Pod。

**外部模式：** 应看到 `redis`、`sp-backend` 与 `log-vector`（无 `{release}-mongo` Pod）。

## 升级已有 Release {#upgrade-existing-release}

使用安装时的 release 名称、命名空间与 `values.yaml`。Softprobe 发布 tag 与 Helm 对应关系：

| 发布 tag | Chart `--version` | `image.tag` |
|----------|-------------------|-------------|
| `v4.3.10` | `4.3.10` | `v4.3.10` |

### 升级前

1. **保留现有 `values.yaml`** — 无需整文件替换。Helm 会将您的文件与新 Chart 默认值合并（未设置的键使用默认值）。
2. **旧文件没有 `logPipeline`？** 若在 v4.3.5 或更早版本安装且仅有 `image`、`mongodb`、`encryption`，只需提升 `--version` 与 `image.tag`。缺失键继承 Chart 默认值 — **`logPipeline.enabled` 为 `true`**，升级时会添加 Vector、Parquet PVC（本地模式）、压缩与保留策略。建议先用 `--dry-run` 预览新资源。
3. **审阅可选覆盖项** — 下载当前的 [values.example.yaml](https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/latest/values.example.yaml)，仅合并所需项（PVC `storageClass`、`placement`、S3 后端）。**不要**更改 `encryption.secretKey` — 已有加密载荷依赖该密钥。
4. **保持 MongoDB 模式不变** — 升级时不要在内置与外部 MongoDB 之间切换。
5. **确认镜像仓库访问** — `softprobe-gcr-pull` Secret 对新 `image.tag` 仍有效。
6. **预览差异**（可选）：

```bash
helm repo update
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false \
  --dry-run
```

### 从 Helm 仓库升级

典型升级 — 使用安装时的 `values.yaml`，更新 Chart 与镜像版本：

```bash
helm repo update
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

若 release 名称不同，请将 `softprobe` 替换为实际名称。**`image.tag`** 须固定为 Softprobe 提供的 semver 发布版本 — 不要用 `latest`。

**示例 — 旧 values 文件无 `logPipeline` 块：** 文件仍为：

```yaml
image:
  tag: "v4.3.5"
  pullSecrets:
    - name: softprobe-gcr-pull
mongodb:
  bundled:
    auth:
      password: "your-existing-password"
encryption:
  enabled: true
  secretKey: "your-existing-key"
```

升级命令（无需编辑文件）：

```bash
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

Helm 会按 Chart 默认值添加日志管道资源。Rollout 完成后，为工作负载配置 `-Dsp.api.url` 指向 sp-backend（见 [Agent 日志导出](#agent-log-export)）。

### 从下载的 Chart 包升级

离线安装或从 GCS 校验 SHA-256 时：

```bash
curl -fLO "https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/v4.3.10/sp-backend-4.3.10.tgz"

helm upgrade softprobe ./sp-backend-4.3.10.tgz \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

### 升级后验证

```bash
kubectl rollout status -n softprobe deploy/softprobe-sp-backend
kubectl get pods -n softprobe
kubectl get pods,cronjob,pvc -n softprobe | grep -E 'log-vector|log-parquet|compaction|retention'
kubectl port-forward -n softprobe svc/softprobe-sp-backend 8090:8090
curl -s http://127.0.0.1:8090/actuator/health
```

预期 sp-backend 滚动重启（若 Chart 模板变更，Redis 也可能重启）。内置 MongoDB 在现有 PVC 上的数据会保留。新 Pod 启动后 sp-backend 可能需要约 2 分钟就绪（JVM 预热）。

v4.3.10+ 还应看到 `log-vector` 与 `log-parquet` PVC（本地存储）。为已插桩工作负载配置：

```text
-Dsp.api.url=http://<release>-sp-backend.<namespace>.svc.cluster.local:8090
```

日志通过 `{sp.api.url}/v1/logs` 导出；sp-backend 内部代理到 Vector。

### 自定义或禁用日志管道

管道**默认开启**。仅在需要非默认存储、放置策略或 S3 时，从 [values.example.yaml](https://storage.googleapis.com/softprobe-published-files/helm/sp-backend/latest/values.example.yaml) 合并覆盖项：

```yaml
logPipeline:
  parquet:
    storageSize: 100Gi
    storageClass: managed-csi
```

要在升级时**禁用**，在运行 `helm upgrade` 前添加：

```yaml
logPipeline:
  enabled: false
```

完整选项见[统一日志管道](#unified-log-pipeline)。

### 升级故障排查

| 现象 | 检查项 |
|------|--------|
| 升级后 `ImagePullBackOff` | 新 `image.tag` 存在于 GCR；`softprobe-gcr-pull` Secret 有效 |
| `helm upgrade` 因 MongoDB 失败 | 仍须设置 **`mongodb.connectionString` 或 `mongodb.bundled.auth.password` 之一** — 不可两者都清空 |
| 新日志 Parquet PVC Pending | 集群 `StorageClass` — 设置 `logPipeline.parquet.storageClass` |
| 启用管道后日志查询为空 | Agent OTLP 端点与 trace 时间范围 — 见[故障排查](#troubleshooting) |

## 统一日志管道 {#unified-log-pipeline}

通过 **sp-backend** Helm Chart 启用关联日志采集、Parquet 存储与 trace-id 查询（`sp logs` / `GET /api/recorder/logs`）。

**前置条件：** Chart **v4.3.x+** 上健康的 sp-backend release。管道**默认启用**（`logPipeline.enabled: true`）。已插桩工作负载需 `-Dsp.api.url` 指向 sp-backend（见 [Agent 日志导出](#agent-log-export)）。

### Chart 部署的资源

当 `logPipeline.enabled: true` 时，Helm 会添加：

| 资源 | 用途 |
|------|------|
| **Vector**（`{release}-log-vector`） | OTLP 日志采集（gRPC/HTTP + Agent JSON，端口 `:4320`） |
| **rclone 网关**（`{release}-log-rclone`，本地 + Azure Blob） | 共享 S3 网关（Deployment + Service），供 Vector 与 sp-backend 读写 Parquet |
| **Parquet PVC**（仅本地模式） | 持久存储，仅由 rclone 网关挂载；其他 Pod 均经网关的 S3 端点访问 |
| **Compaction CronJob**（所有后端） | 合并已关闭小时的分钟文件 → `part-hourly.parquet`（DuckDB） |
| **Retention CronJob**（可选，所有后端） | 清理早于 `ttlDays` 的 Parquet |

启用管道时 sp-backend 会配置 Parquet 读取，并导出自身诊断日志（`OTEL_ENABLED=true`，`OTEL_LOGS_EXPORTER=otlp-filtered`）。

### 在 Helm 中配置

Chart 默认启用管道。需要非默认存储、放置策略或禁用时，在 values 文件（或 install/upgrade 的 `--set`）中覆盖：

```yaml
logPipeline:
  enabled: true   # 默认；设为 false 可禁用
  storage:
    backend: local   # local | s3 | azure_blob — 详见"选择日志存储位置"
  parquet:
    storageSize: 100Gi
    # storageClass: managed-csi   # 可选 — 省略时使用集群默认
  retention:
    ttlDays: 4               # 设为 "" 可禁用 Retention CronJob
    cleanupSchedule: "0 3 * * *"
  compaction:
    enabled: true            # 仅本地存储
    schedule: "15 * * * *"   # 上一已关闭 UTC 小时
  # 使用 taint 时将 Vector 与维护任务固定到与 sp-backend 相同的节点池：
  placement:
    nodeSelector:
      workload: softprobe-backend
    tolerations:
      - key: workload
        operator: Equal
        value: backend
        effect: NoSchedule
```

**从没有 `logPipeline` 块的旧 `values.yaml` 升级？** 无需添加 — Chart 默认值会在升级时部署管道。见[升级已有 Release](#upgrade-existing-release)。

显式覆盖的可选升级示例：

```bash
helm upgrade softprobe softprobe/sp-backend \
  --version 4.3.10 \
  -n softprobe \
  -f values.yaml \
  --set image.tag=v4.3.10 \
  --set createNamespace=false
```

**`image.tag`** 须固定为 semver 发布版本（例如 `v4.3.10`），不要用 `latest`，以保证后端与 Chart 版本一致。

### 验证日志管道

```bash
kubectl get pods,cronjob,pvc -n softprobe | grep -E 'log-vector|log-parquet|compaction|retention'
kubectl port-forward -n softprobe svc/softprobe-sp-backend 8090:8090
```

运行固定查询（替换 trace id 与时间范围）：

```bash
export SP_API_URL=http://127.0.0.1:8090
sp logs --trace-id <32-hex> --since 2026-06-27T10:00:00Z --until 2026-06-27T10:05:00Z
```

或：

```bash
curl -s "$SP_API_URL/api/recorder/logs?trace_id=<id>&since=2026-06-27T10:00:00Z&until=2026-06-27T10:05:00Z"
```

v1 **没有**专用管道健康 API — 成功的 trace-id 查询可确认采集、存储与查询链路。

### Agent 日志导出 {#agent-log-export}

Java Agent 仅需指向 sp-backend：

```text
-Dsp.api.url=http://<release>-sp-backend.<namespace>.svc.cluster.local:8090
```

release 为 `softprobe`、命名空间为 `softprobe` 时：

```text
-Dsp.api.url=http://softprobe-sp-backend.softprobe.svc.cluster.local:8090
```

录制与回放期间，关联的应用与 Agent 日志导出至 `{sp.api.url}/v1/logs`。日志管道启用时，sp-backend 将 Agent JSON 代理到 Vector `:4320`，OTLP 代理到 `:4318`。

可选高级覆盖 — 直连 Vector（绕过 backend 代理）：

```text
-Dsp.otel.exporter.otlp.log.endpoint=http://<release>-log-vector.<namespace>.svc.cluster.local:4320/v1/logs
```

v1 不使用旧版采集标志（`sp.record.user.log`、`sp-capture-log`、`sp.user.log.level` 等）。

### 选择日志存储位置

通过 `logPipeline.storage.backend` 选择 Parquet 日志文件的存储位置，三选一：

| 后端 | `logPipeline.storage.backend` | 适用场景 | 需要提供的凭证 |
|------|-------------------------------|----------|----------------|
| **本地磁盘**（默认） | `local` | 单集群、最简部署 | 无（集群内卷） |
| **S3 兼容** | `s3` | AWS S3、MinIO、GCS 等任意 S3 API | `access-key-id` + `secret-access-key` |
| **Azure Blob** | `azure_blob` | Azure 存储账户 | `account-key` |

三种后端在查询、压缩与保留上的行为完全一致 — 仅存储位置与凭证不同。

#### 方式一 — 本地磁盘（默认）

无需配置。Chart 会创建 PersistentVolumeClaim 并将 Parquet 存于其中。仅在需要时调整大小/StorageClass：

```yaml
logPipeline:
  storage:
    backend: local
  parquet:
    storageSize: 100Gi
    storageClass: ""      # 集群默认；或如 managed-csi (AKS)、gp3 (EKS)
```

备份：对 `{release}-log-parquet` PVC 做快照。

#### 方式二 — S3 兼容 Bucket

**第 1 步 — 创建凭证 Secret。** 必须包含以下两个键：

```bash
kubectl create secret generic softprobe-log-s3-credentials -n softprobe \
  --from-literal=access-key-id='AKIA...' \
  --from-literal=secret-access-key='...'
```

| Secret 键 | 值 |
|-----------|----|
| `access-key-id` | Bucket 的 Access Key ID |
| `secret-access-key` | Bucket 的 Secret Access Key |

**第 2 步 — 让 Chart 指向您的 Bucket 与该 Secret：**

```yaml
logPipeline:
  storage:
    backend: s3
    s3:
      bucket: my-softprobe-logs
      endpoint: https://s3.amazonaws.com   # 或您的 MinIO / GCS / 其他 S3 端点
      region: us-east-1
      forcePathStyle: true                 # MinIO 保持 true
      prefix: ""                           # Bucket 内可选 key 前缀
      existingSecret: softprobe-log-s3-credentials
```

Bucket 需已存在。此模式下**不会**创建 Parquet PVC。

#### 方式三 — Azure Blob 容器

**第 1 步 — 创建凭证 Secret。** 必须且仅包含一个键 `account-key`，即存储账户访问密钥：

```bash
kubectl create secret generic softprobe-log-azure-credentials -n softprobe \
  --from-literal=account-key='<存储账户访问密钥>'
```

| Secret 键 | 值 |
|-----------|----|
| `account-key` | 存储账户访问密钥（Azure 门户 → **存储账户 → 安全性 + 网络 → 访问密钥**，或 `az storage account keys list --account-name <account> --query '[0].value' -o tsv`） |

**第 2 步 — 让 Chart 指向您的容器与该 Secret：**

```yaml
logPipeline:
  storage:
    backend: azure_blob
    azureBlob:
      container: my-softprobe-logs
      accountName: <account>
      endpoint: https://<account>.blob.core.windows.net
      prefix: ""                           # 容器内可选前缀
      existingSecret: softprobe-log-azure-credentials
```

容器需已存在。Softprobe 使用账户名 + 访问密钥（Azure Shared Key）认证。此模式下**不会**创建 Parquet PVC。

#### 压缩与保留（所有后端）

两者均自动运行，且在 `local`、`s3`、`azure_blob` 上行为一致 — 无需云端生命周期规则：

* **压缩**（`logPipeline.compaction`，每小时，默认开启）：将每个已关闭 UTC 小时的分钟文件合并为单个 `part-hourly.parquet`。
* **保留**（`logPipeline.retention.ttlDays`，默认 `4`）：删除超过 N 天的 Parquet。设 `ttlDays: ""` 可永久保留。

> 压缩使用 `softprobe/duckdb:1.1.3` 镜像（`linux/amd64`）。Apple Silicon 开发集群可本地构建/加载 `arm64` 镜像（`make duckdb-image DUCKDB_PLATFORM=linux/arm64`）并覆盖 `logPipeline.compaction.image`。

**安全：** 终端用户与 Agent Skills **绝不得**获得 Bucket 或存储账户凭证 — 仅通过 `sp logs` / `GET /api/recorder/logs` 查询。

### Helm values 参考

| 值 | 说明 |
|----|------|
| `logPipeline.enabled` | 部署 Vector、存储与查询 wiring（默认 `true`） |
| `logPipeline.storage.backend` | `local`（PVC）、`s3` 或 `azure_blob` |
| `logPipeline.parquet.storageSize` / `storageClass` | 本地 Parquet PVC 大小与 StorageClass |
| `logPipeline.vector.image` | Vector 镜像（默认 `timberio/vector:0.56.0-debian`） |
| `logPipeline.vector.resources` | 日志采集器 CPU/内存 |
| `logPipeline.otlp.httpPort` / `grpcPort` / `agentJsonPort` | OTLP 端口（默认 `4318` / `4317` / `4320`） |
| `logPipeline.retention.ttlDays` | 清理 TTL（天）；`""` 禁用 Retention CronJob |
| `logPipeline.retention.cleanupSchedule` | Retention CronJob 调度（默认 `0 3 * * *`） |
| `logPipeline.compaction.enabled` / `schedule` / `image` | 每小时压缩，适用于所有后端（默认开启，`15 * * * *`，`softprobe/duckdb:1.1.3`） |
| `logPipeline.placement` | Vector 与维护 CronJob 的 `nodeSelector` / `tolerations` / `affinity` |
| `logPipeline.agentLogEndpointProperty` | 文档化 JVM 属性：`sp.otel.exporter.otlp.log.endpoint` |

Chart 默认值中的 `logPipeline.parquet.localRoot`（`/data/parquet/logs`）须与内部存储布局一致 — 运维通常**不要**覆盖。

### 维护任务

| CronJob | 条件 | 作用 |
|---------|------|------|
| `{release}-log-vector-retention` | 已设置 `ttlDays`（任意后端） | 删除早于 TTL 的 Parquet 文件/对象 |
| `{release}-log-vector-compaction` | `compaction.enabled`（任意后端） | DuckDB 合并上一小时的 `part-*.parquet` → `part-hourly.parquet` |

查看最近运行：

```bash
kubectl get cronjob,jobs -n softprobe -l 'app.kubernetes.io/component=log-pipeline-maintenance'
kubectl logs -n softprobe job/<compaction-job-name>
```

### v1 范围外

* Iceberg、即席 SQL、终端用户直接访问 Parquet。
* 本地 PVC 与 S3 双写。
* 专用日志管道健康/状态 API。

## 卸载

```bash
helm uninstall softprobe -n softprobe
```

内置 MongoDB PVC 默认保留。需要时可手动删除：

```bash
kubectl delete pvc -n softprobe -l app.kubernetes.io/instance=softprobe
```

## Java Agent

将已插桩应用指向集群内服务：

```text
-Dsp.api.url=http://softprobe-sp-backend.softprobe.svc.cluster.local:8090
```

## 故障排查 {#troubleshooting}

| 现象 | 检查项 |
|------|--------|
| `helm install` 因 MongoDB 失败 | 须设置 **`mongodb.connectionString` 或 `mongodb.bundled.auth.password` 之一** |
| `sp-backend` Pod `Init:0/1`（内置） | MongoDB 或 Redis 未就绪 — `kubectl get pods -n softprobe` |
| `sp-backend` 启动慢 | JVM 预热 — 最多约 2 分钟（startup probe） |
| `ImagePullBackOff` | 缺少 `softprobe-gcr-pull` Secret 或 `image.tag` 错误 |
| Mongo PVC Pending（内置） | 无 StorageClass — 设置 `mongodb.bundled.storageClass` |
| 外部 MongoDB 连接错误 | URI 从集群可达；数据库名唯一；`authSource` 正确 |
| 预期有数据但 `GET /api/recorder/logs` 为空 | 中断的压缩留下不完整 `part-hourly.parquet` — 删除 hourly 文件或等待下次压缩；确认存在 minute `part-*.parquet` |
| Vector Pod 未就绪 | `kubectl logs -n softprobe deploy/<release>-log-vector -c vector` |
| arm64 上 Compaction `ImagePullBackOff` | 用本地 `arm64` 构建覆盖 `logPipeline.compaction.image` |
| Agent 日志缺失 | `sp.api.url` 须可达 sp-backend；日志管道已启用；导出须带 `trace_id` |
| sp-backend 日志缺失 | `logPipeline.enabled` 会在 sp-backend 上自动启用 OTLP 导出 |

## 下一步

sp-backend 健康后，在开发者机器上安装 Softprobe 客户端：[安装 Softprobe（客户端）](./)。

Linux 上共享团队 Web 工作台见客户端安装页的 [Spcode Service](./index#spcode-service)。

相关：[`sp logs`](/zh/testing/commands/logs) · [日志关联 ID](/zh/testing/reference/log-correlation-ids)

---

---
url: /zh/testing/installation.md
---

# 安装 Softprobe（客户端）

::: tip 前置条件
请先在集群中部署 Softprobe 后端：[安装 Softprobe 服务端](./server)。
:::

使用全局安装器安装 Softprobe：

```bash
curl -fsSL https://install.softprobe.ai/install.sh | bash
```

默认安装包含 `sp` CLI、Java Agent 以及测试用的 Web UI。

安装后先确认 `sp` 可用：

```bash
sp -v
```

若能输出版本号，即可继续下面的步骤。

## 设置

`sp setup` 配置自托管或本地部署的 Softprobe 后端 URL。

```bash
sp setup
```

若未提供 `--api-url`，向导会**交互式**询问后端 URL。在 Linux 上还可能询问是否安装 Spcode Service（见[下文](#spcode-service)）。

| 参数 | 说明 |
|------|------|
| `--api-url` | 后端 URL（提供后可跳过交互提示） |

模型提供商配置由 `sp code` 负责，不在 `sp setup` 中配置。

非交互或脚本场景可显式传入 URL：

```bash
sp setup --api-url http://127.0.0.1:8090 --json
```

URL 写入 Softprobe 配置 `~/.config/softprobe/config.jsonc`。

## Spcode Service（仅 Linux） {#spcode-service}

在 **Linux** 上，`sp setup` 还可选安装 **Spcode Service** —— 供团队在内网浏览器共用的 Web 工作台。若只需在本机使用 UI，见 [手动启动 Softprobe Web UI](./code)。

```bash
sp setup --api-url http://sp-backend.corp:8090 --install-spcode-service
sp setup --install-spcode-service --spcode-service-port 5000
sp setup --uninstall-spcode-service
```

| 参数 | 说明 |
|------|------|
| `--install-spcode-service` | 安装共享 Web 工作台（**仅 Linux**） |
| `--uninstall-spcode-service` | 卸载 Spcode Service |
| `--spcode-service-port` | 安装时监听端口（默认 `4096`） |

不可同时使用 `--install-spcode-service` 与 `--uninstall-spcode-service`。

### 安装

```bash
# 交互：后端 URL → 是否安装 → 端口（默认 4096）
sp setup

# 非交互
sp setup --api-url http://sp-backend.corp:8090 --install-spcode-service

# 无 TTY 自动化
sudo sp setup --api-url http://sp-backend.corp:8090 --install-spcode-service
```

请用将拥有本机 Softprobe 设置的账号安装 Softprobe 并运行 `sp setup`。Spcode Service 使用同一套设置（后端 URL、MCP、Agent 说明、skills）。修改后请重启服务。

### 运维

| 操作 | 命令 |
|------|------|
| 状态 | `systemctl status spcode-web.service` |
| 停止/启动 | `systemctl stop spcode-web.service` / `systemctl start spcode-web.service` |
| 日志 | `journalctl -u spcode-web -n 50 --no-pager` |

若启动失败，先查看 `journalctl -u spcode-web` 再重试。

可选 MCP、Agent 说明与 skills：见 [客户端配置](./configuration)。

### 卸载

```bash
sp setup --uninstall-spcode-service
```

移除 systemd 单元；安装账号下的 Softprobe 配置文件会保留，除非手动删除。

## 下一步

CLI 装好了，接下来把你的 Java 应用接进来：

1. [支持的框架](/zh/testing/supported-frameworks) — 先确认你的技术栈能被织入
2. [接入 Java Agent](/zh/testing/java-agent) — 挂载 Agent、注册应用
3. [启动 Web UI](./code) — 在浏览器里查看录制与回放

工具与进阶：[Doctor](./doctor)（排障）· [升级](./upgrade) · [客户端配置](./configuration)（MCP、`AGENTS.md`、skills）

---

---
url: /zh/testing/review-diffs-in-the-web-ui.md
---

# 审查回放差异

[回放](/zh/testing/replay-and-diff) 跑完后，工作台会显示哪些用例失败了、为什么失败。本页讲在**网页控制台**里人工审查的流程——读一处 diff，接受那些不是真 bug 的差异。更喜欢命令行？`sp replay case list --plan <planId>` 和 `sp diagnose replay <planId>` 给出同样的失败数据（见 [回放与对比](/zh/testing/replay-and-diff)）。

大多数差异不是 bug。时间戳、随机 token、ID 每次调用都在变——它们永远会「不一样」，但并没有出错。你要做的，是接受它们，让真正的失败露出来。

## 接受一处差异：只改这一次，还是永久

让一处差异不再计数有两种方式，选对哪一种是关键：

* **在这条用例上标记通过** —— *「这一次回放里，这条差异没问题。」* 立即生效，只影响这一次回放。用于：你看过某条用例、确认它不是真 bug。
* **配一条对比规则** —— *「以后每次回放，都别比这个字段。」* 永久生效，影响所有未来回放。用于：某字段天生每次都不同（时间戳、随机 ID），本就不该参与比对。

一句话选：**只想改这一次的结果 → 标记通过；想一劳永逸 → 配规则。**

| | 标记通过 | 对比规则 |
|---|---|---|
| 作用范围 | 只这一次回放 | 以后每一次回放 |
| 持续多久 | 一次性——下次回放就没了 | 永久——存在策略里 |
| 用于 | 你已经看过并接受的一条用例 | 一个总在变的字段 |

::: tip 规则只在下次回放才生效
一条对比规则不会动一次已经跑完的回放。要把新规则应用到你正在看的这次回放、又不重放，用 [重新比对](#recompare-apply-rules-to-this-run)。
:::

## 打开一条用例，读它的差异

1. 打开一次**回放**，选一个**失败用例**——左侧面板用红点列出它们。
2. 点右边调用树里的一个 **span**。

diff 打开：左边是录制响应，右边是回放响应。不一致的字段被高亮；头部显示这条用例有几处差异。

![diff 在调用树旁打开](/img/docs/replay/trace-diff-open.png)

## 忽略一个字段 {#ignore-a-field}

**这是你最常用的操作。** 像时间戳这样的字段每次回放都不同，你根本不想比它。

1. 在 diff 里**悬停**那一行差异。左边缘会出现一个 **eye-off 图标**——点它。（或者右键那一行，或用顶部的 **忽略规则** 按钮。）
2. 选 **忽略此字段的差异**。
3. 选 **仅此用例**。

字段被划上删除线、差异消失、用例通过率更新。

![忽略菜单](/img/docs/replay/ignore-menu.png)

### 忽略得更大范围

**仅此用例** 只改你眼前这一条。但一个时间戳会在*每一条*用例上都不同，一条条忽略太累。要更大范围地不再比一个字段，在同一个菜单里选更大的范围：

| 选 | 忽略这个字段的范围 | 何时生效 |
|---|---|---|
| **仅此用例** | 只这一条用例 | 立即 |
| **仅此接口** | 这个接口的每一条用例 | 下次回放 |
| **所有接口** | 整个应用 | 下次回放 |

后两个更大的范围会写一条**对比规则**，它*下次*回放才生效——所以你正在看的这次回放暂时不变。Softprobe 会给一个 **重新比对** 按钮，让你现在就应用它；见 [重新比对](#recompare-apply-rules-to-this-run)。

::: tip 忽略所有同名字段
菜单里还有 **忽略所有 "…" 字段**——当同一个字段名（比如 `updatedAt`）出现在好几个地方、你想一次全清掉时很方便。
:::

## 标记一条用例通过

有时差异是真的，但你能接受——一次故意的变更，或一次 Softprobe 造成的假差异。与其一个个忽略字段，不如接受**整条用例**。

1. 在失败用例上，点 **标记通过**。
2. 选一个原因：**差异符合预期（设计如此）**，或 **Softprobe 采集/对比问题**。
3. 可选加个备注，然后确认。

用例移到通过、通过率更新。你的原因和备注会存下来供日后查看。

![标记通过表单](/img/docs/replay/mark-passed.png)

## 重新比对：把规则应用到本次回放 {#recompare-apply-rules-to-this-run}

你刚写的规则只在**下次**回放才起作用——它不会改变一次已经跑完的回放。**重新比对** 把你最新的规则应用到你正在看的这次回放，重新检查它已存的响应。它**不**重放任何流量。

当你从 diff 里加了或删了一条规则，run 头部会出现一个 **重新比对** 按钮。点它。Softprobe 重新检查这次回放，把用例列表、计数、通过率一起更新。

![重新比对完成，显示多少用例变了](/img/docs/replay/recompare-done.png)

::: warning 重新比对不是回放
回放会把流量重新发到你的服务。重新比对只对已经存下的响应、按你当前的规则重新判定——什么都不会打到你的服务。
:::

## 取消一次忽略

改主意了？**右键那条删除线行**，选 **取消忽略（恢复比对）**。字段就回到参与对比。

你不需要记得当初是怎么忽略的——Softprobe 会删掉挡着这处差异的东西（用例上的标记，或那条规则），并告诉你删了什么。如果规则配在全局默认里，它会指引你去对比规则页处理，而不是让你卡住。

## 查看已经忽略的一切

不用逐个打开 span，就能看到一条用例上忽略了什么：点用例顶部的 **「N 处已忽略」** chip。一个面板会按调用分组，列出每个被忽略字段和挡下它的规则。悬停任一行就能就地取消忽略。

![忽略总览面板](/img/docs/replay/ignored-summary.png)

## 下一步

同一个字段在每条用例上都要忽略一遍？该把它沉淀成规则了 → **[配置对比规则](/zh/testing/compare-rules-web-ui)**：一次配置，以后每次回放都不再误报。

---

---
url: /zh/testing/installation/configuration.md
---

# 客户端配置

本页是客户端（`sp` CLI、Java Agent、编码引擎）配置文件与高级选项的参考。Softprobe 使用一个共享的 XDG 配置命名空间。

| 路径 | 用途 |
|------|------|
| `~/.config/softprobe/config.jsonc` | 共享后端 URL 和凭证 |
| `~/.config/softprobe/sp.jsonc` | CLI 配置和 profile |
| `~/.config/softprobe/spcode.jsonc` | 内部编码引擎设置 |
| `~/.local/share/softprobe/agent/sp-agent.jar` | 已安装的 Java agent |
| `~/.local/share/softprobe/bin/spcode` | 内部编码引擎二进制 |

后端 URL 用 `sp setup`。编码引擎与模型提供商就绪用 `sp code`。

## Spcode Service（Linux systemd）

通过 [`sp setup --install-spcode-service`](./#spcode-service) 安装后，共享工作台使用**执行安装的那个账号**的同一套 Softprobe 设置（后端 URL、MCP、Agent 说明、skills）。

请用该安装账号配置 Softprobe。修改后如需服务生效，重启：

```bash
sudo systemctl restart spcode-web.service
```

## MCP 工具与 Agent 说明 {#spcode-service-mcp-agents}

`sp setup` / `sp code` **不会**自动创建编码引擎的 MCP 配置或全局 Agent 说明。请写入：

| 文件 | 用途 |
|------|------|
| `~/.config/spcode/opencode.jsonc` | MCP 与引擎设置 |
| `~/.config/spcode/AGENTS.md` | 全局 Agent 说明 |

个人 `sp code web` 与 Linux Spcode Service 都使用上述路径。修改后请重启 UI 进程（或 `spcode-web.service`）。

### 示例：飞书 / Lark MCP + `AGENTS.md`

::: warning 前置条件：Node.js
本 MCP 示例通过 **`npx`** 启动 `@larksuiteoapi/lark-mcp`，因此主机必须已安装 **Node.js**（自带 `npx`）。Softprobe 安装**不会**自动安装 Node.js。

检查：

```bash
node -v
npx -v
```

若命令不存在，请从 [nodejs.org](https://nodejs.org/) 安装 Node.js（LTS），或使用系统包管理器安装（例如 Debian/Ubuntu：`apt install nodejs npm`；RHEL/Fedora：`dnf install nodejs`）。若使用 Spcode Service，请再确认 root 下可用：`sudo npx -v`（服务进程以 root 运行）。
:::

1. 创建 `~/.config/spcode/opencode.jsonc`（将占位符替换为你的应用凭证；**不要**把真实密钥提交到仓库）：

```jsonc
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "lark-mcp": {
      "type": "local",
      "enabled": true,
      "command": [
        "npx",
        "-y",
        "@larksuiteoapi/lark-mcp",
        "mcp",
        "-a",
        "YOUR_FEISHU_APP_ID",
        "-s",
        "YOUR_FEISHU_APP_SECRET",
        "--token-mode",
        "tenant_access_token",
        "--domain",
        "https://open.feishu.cn",
        "-t",
        "docx.v1.document.get,docx.v1.document.rawContent"
      ]
    }
  }
}
```

主机还需能访问飞书开放平台（`https://open.feishu.cn`）。

2. 创建 `~/.config/spcode/AGENTS.md`，告知 Agent 如何使用该 MCP。示例：

```md
# Softprobe 服务端 Agent 说明

关于应用、Git 分支、录制/回放仓库信息，**不要硬编码**。

请使用 **lark-mcp** MCP 服务，从以下飞书文档获取最新内容：

https://example.feishu.cn/docx/YOUR_DOC_TOKEN

优先调用 `docx.v1.document.rawContent`（或 lark-mcp 中的等价工具）读取该文档，并以返回的表格/文本作为应用名称与分支配置的唯一真实来源。
```

请将飞书文档链接替换为你们自己的文档。`opencode.jsonc` 中配置的任意 MCP 均可使用，不限于飞书。

## Skills {#skills}

Skill 是包含 `SKILL.md` 的目录。Softprobe 会自动从以下位置加载：

| 位置 | 路径 |
|------|------|
| 本机全局 | `~/.config/spcode/skill/<name>/SKILL.md` 或 `~/.config/spcode/skills/<name>/SKILL.md` |
| 项目内 | 打开的仓库中 `.opencode/skill/<name>/SKILL.md` 或 `.opencode/skills/<name>/SKILL.md` |

每个 `SKILL.md` 至少需要 frontmatter 中的 `name` 与 `description`。示例：

```md
---
name: my-skill
description: Use when the user asks about release checklists.
---

# My skill

Steps the agent should follow…
```

若要从其他目录加载 skill，可在 `~/.config/spcode/opencode.jsonc` 中设置 `skills.paths`。

没有单独的 “create skill” 命令——自行添加目录与文件即可（也可让编码 Agent 协助）。若 Spcode Service 使用了全局 skill，修改后请重启 `spcode-web.service`。

---

---
url: /zh/testing/how-it-works.md
---

# 录制回放如何工作

Softprobe 测试采集的是一次**事务**：一条入口请求，以及该调用链上代码触达的每个依赖。回放时同一代码路径再次执行；外部调用由存储数据满足，而非访问真实系统。

## 端到端时序

```mermaid
sequenceDiagram
  participant App as 被测应用
  participant Agent as Java Agent
  participant Storage as sp-backend 存储
  participant Redis as Redis Mock 缓存
  participant Mongo as MongoDB
  participant Schedule as 回放调度

  Note over App,Mongo: 录制
  App->>Agent: 依赖调用
  Agent->>Storage: 上传 SpMocker
  Storage->>Mongo: 持久化

  Note over Schedule,Redis: 回放准备
  Schedule->>Storage: 预加载 Mock
  Storage->>Redis: record 分类键

  Note over App,Redis: 回放
  App->>Agent: 相同调用路径
  Agent->>Storage: 查询 Mock
  Storage->>Redis: 取录制响应
  Storage-->>Agent: Zstd 载荷
  Agent-->>App: Mock 依赖

  Note over Schedule,Mongo: 对比
  Schedule->>Storage: 录制 vs 回放列表
  Storage->>Schedule: 差异结果
```

## 录制阶段 {#recording-phase}

开启录制后，Agent 拦截：

| 类型 | 示例 | 存储内容 |
|------|------|----------|
| **入口** | `Servlet`、`DubboProvider`、`NettyProvider` | 入站 API 请求与响应 |
| **依赖** | `HttpClient`、`Database`、`Redis`、`DubboConsumer`、`DynamicClass` 等 | 每次对外调用的请求与响应 |

每次交互是一条 **mocker** 记录，按 `appId`、trace/用例标识、分类与操作名索引。

**如何产生用例：** [录制流量](/zh/testing/recording) · **录制范围配置：** [录制策略](/zh/testing/policies#recording-policy)

::: tip
用例**仅由**已织入 Agent 的流量产生。CLI 不支持手写用例；请在 Agent 录制期间向应用发送真实或合成流量。
:::

## 回放阶段

**回放计划**选择已录制用例，并将入口 HTTP 流量驱动到 **`targetEnv`** — 被测服务的基础 URL（例如 `http://order-service.test:8080`）。这与指向 sp-backend 的 `SP_API_URL` **不是**同一个地址。

回放过程中：

1. 调度服务将录制的入口请求发往 `targetEnv`。
2. 被测应用必须挂载 Agent（回放机器上可将录制频率设为 0）。
3. 每次依赖调用时，Agent 向存储查询**匹配的录制响应**。
4. 已预加载时从 Redis 读取；否则从 MongoDB 加载。

**入口与依赖的行为差异：**

* **入口**（`entryPoint` 分类）：存储会记录回放侧请求；业务代码中的 Controller/Handler 仍会真实执行。
* **依赖**：存储返回录制的 Mock 体，应用不会访问真实数据库或外部 API。

## 对比阶段

回放结束后，对比引擎配对录制与回放的 mocker。失败表示主响应或某依赖不一致（缺调用、值不同、多调用等）。

常见差异类型：

* **值差异** — 依赖被调用了，但响应体不同
* **缺调用** — 回放未调用录制时存在的依赖
* **多调用** — 回放调用了录制中不存在的依赖

通过[对比策略](/zh/testing/policies)与[回放与对比](/zh/testing/replay-and-diff)忽略噪声字段（时间戳、令牌、IP 等）。

## 示例说明

下面方法解析 IP 并调用校验逻辑：

```java
public Integer parseIp(String ip) {
    int result = 0;
    if (checkFormat(ip)) {
        String[] ipArray = ip.split("\\.");
        for (int i = 0; i < ipArray.length; i++) {
            result = result << 8;
            result += Integer.parseInt(ipArray[i]);
        }
    }
    return result;
}
```

**录制** — 当 `needRecord()` 为真时，Agent 保存入参与返回值。

**回放** — Agent 用存储结果短路执行，使 `checkFormat` 与解析行为与采集时一致，即使测试环境不同。

本地缓存、加解密、系统时间等**动态类**同样适用；通过录制策略与 Mock 策略配置，无需改业务代码。

## 存储结构（概念）

| 存储 | 作用 |
|------|------|
| **MongoDB** | 持久化录制、回放计划、对比结果 |
| **Redis** | 回放热路径 Mock 缓存（`record:{category}:{recordId}:…`） |

自托管时，`sp-backend` 通常是单一进程，在 **8090** 端口同时提供 API、存储与调度。

## 相关文档

* [Java Agent](/zh/testing/java-agent)
* [策略](/zh/testing/policies)
* [回放与对比](/zh/testing/replay-and-diff)
* [CLI 概念](/zh/testing/agents/concepts)

---

---
url: /zh/platform/support/faq.md
---
# 常见问题 (FAQ)

本文整理了使用 Softprobe 过程中最常见的问题与答案，帮助你快速定位与解决问题。

## 安装与配置

### 如何选择部署环境？

建议按阶段部署：开发 → 预发 → 生产，并分别配置公钥，逐步扩大采样范围以降低风险。

### 需要修改应用代码吗？

不需要。SP-Istio Agent 运行在服务网格层，SESSIFY 可按需集成前端，无需改动后端代码。

### 支持哪些 Kubernetes 环境？

支持主流的 Kubernetes 环境，包括自建集群与云厂商托管方案。Istio 版本需与 SP-Istio Agent 兼容。

## 性能与稳定性

### 会影响服务性能吗？

影响极小。Agent 与 SDK 均为轻量实现，支持采样、批量传输与本地优化。可根据业务负载调整参数。

### 如何处理高并发场景？

通过批量传输、队列缓冲与采样策略保持稳定。建议在生产环境按需调优采样率与队列大小。

## 数据安全

### 是否存储敏感数据？

不持久化敏感数据，传输全程 TLS 加密，并采用公钥认证签名。

### 如何进行密钥轮换？

公钥可热更新，无需停机。对不同环境使用不同公钥以降低风险。

## 运维与排错

### 构建或预览出现告警/断链怎么办？

先查看日志，定位断链来源；若为跨语言链接，使用相对中文路径或回退到英文页面以确保无断链。

### 如何确认数据采集正常？

查看仪表盘中的服务依赖图与追踪样例，确认前后端数据均有接入。必要时检查 Sidecar 注入与 SDK 初始化。

## 重要功能点速览

* 仪表盘：现代化自适应 UI、实时可视化、热力图与会话回放；技术栈为 React + Vite + TailwindCSS。
* OTEL 后端：接收 Trace/Log/Metric；使用 BigQuery 存储，并以 request\_body\_hash 建立索引实现快速关联与查找。
* 认证服务：多租户认证与授权；Public Key 全生命周期；JWT 工具；基于角色的权限（OWNER/ADMIN/MEMBER）。

::: info 为什么重要？
上述组件协同提供端到端可观测性与会话关联能力，确保数据高效查询并满足多租户的安全与隔离要求。
:::

## 注意事项与最佳实践

* 环境隔离：生产与非生产分别使用不同租户/公钥；先小范围采样，逐步放大以降低风险。
* Public Key 管理：完整密钥仅在创建时展示；妥善保存并定期轮换；切勿将密钥提交到版本库。
* 速率限制：每个 Public Key 有每小时请求上限；客户端需实现重试/退避，并关注使用计数。
* 数据敏感性：避免在属性中发送敏感个人信息；依赖签名校验与 TLS；用哈希的请求体进行关联而非明文。
* 性能建议：BigQuery 索引随数据增长；结合时间窗口与 request\_body\_hash 过滤；必要时启用分区或缓存。
* 发布流程：使用 CI/CD（如 ArgoCD）配合镜像 Tag；仪表盘版本与后端 schema 变更需同步。

::: tip 最佳实践
为不同项目/环境分别创建租户与公钥，提升审计与访问控制的清晰度。
:::

::: warning 注意
请勿在工单或聊天工具中粘贴完整 Public Key。UI 中仅展示掩码版本，若疑似泄露请立即轮换。
:::

## 套餐与配额

* 存储配额（按租户）：可配置（例如租户设置中的 storageQuotaGb）；建议设置用量告警与容量规划。
* API 速率限制：按密钥的每小时请求上限（rateLimitPerHour）；需依据预估写入/查询负载进行容量规划。
* 多租户边界：资源（数据集/存储桶等）按租户与环境隔离，确保安全与成本控制。
* 企业版：支持更高配额、SLO 与自定义保留周期等高级能力，详情请联系 support@softprobe.ai。

***

如遇本文未覆盖的问题，请在社区或支持页面提交反馈。

---

---
url: /zh/platform.md
---

# Softprobe 平台

**零代码改动 · 全上下文可见性 · 成本优化**

::: info
Softprobe 将每一次用户旅程捕获为会话图谱——让交互可分析、可自动化，并能经济地长期留存。
:::

## 快速链接

* [快速入门](/zh/platform/getting-started/quick-start)
* [生产部署](/zh/platform/deployment/installation)
* [SESSIFY](/zh/platform/sessify)
* [仪表盘用户指南](/zh/platform/production/dashboard-user-guide)

## 录制与回放（Java）

Java 流量的捕获、回放与差异比对，请参见 [Softprobe 测试](/zh/testing/getting-started)。使用 [CLI 快速入门](/zh/testing/getting-started) 进行自动化。

## 工作原理

* **服务端：** Istio Envoy sidecar 中的 Wasm 插件 —— [SP-Istio Agent（GitHub）](https://github.com/softprobe/softprobe)
* **客户端：** [SESSIFY](/zh/platform/sessify) 以路由、指标与交互事件丰富会话

![Softprobe 架构](/img/docs/how-it-work.png)

::: tip 当前可用

* 数据采集：SESSIFY 与 Istio/Envoy OpenTelemetry 链路
* 可视化：Context View（会话图谱）
  :::

![会话图谱](/img/docs/context-view.png)

---

---
url: /zh/testing/recording.md
---

# 录制流量

录制就是让挂了 Agent 的应用**正常处理真实请求**——每条经过的请求连同它触发的依赖调用（数据库、HTTP、Redis…）会自动存成一条**用例**，成为后面回放的素材。

本页以 `order-service` 为例：它已按 [接入 Java Agent](/zh/testing/java-agent) 挂载 Agent、按快速开始注册过应用（`appId` 在手）。录制到配置对比规则这几步都用这个应用。

::: tip 开箱即录，不需要先配策略
内置的全局默认策略（priority 0）让录制开箱即用，并已排除 `/health` 等探针流量。只有当你要调整采样率、时间窗口或操作范围时，才需要写应用级策略——见本页末尾 [调整录制范围](#调整录制范围)。
:::

## 第 1 步 · 确认 Agent 在线

在**要采集流量的环境**（通常是生产或预发）启动应用：

```bash
java -javaagent:sp-agent.jar \
     -Dsp.app.id=<你的 appId> \
     -Dsp.api.url=http://<后端主机>:8090 \
     -jar order-service.jar
```

确认 Agent 已上报：

```bash
sp app status <你的 appId> --json
```

多环境部署时给实例打标签（如 `-Dsp.mocker.tags=env=prod`），之后筛选用例、匹配策略都靠它。

## 第 2 步 · 让真实流量流过

什么都不用做——用户请求、业务调用、压测流量经过应用就会被采集。没有自然流量的环境（如预发），主动向接口发几笔业务请求即可。

::: tip 用例只能录出来，不能手写
CLI 不支持手工构造用例。想要更多用例，就让更多流量流过应用。
:::

## 第 3 步 · 确认录到了

```bash
sp record case list --app <你的 appId> --since -1h --json
```

列表里出现用例，第 1 步就完成了——直接进入 [回放](/zh/testing/replay-and-diff)。

需要按链路核对完整性时用 `sp record completeness <traceId> --json`。

## 没录到？按这张表排查

| 现象 | 优先排查 |
|------|----------|
| `sp record case list` 为空 | 应用级策略把 `ratePerHundredSeconds` 设成了 0；当前时间在 `timeWindow` 外；操作被 `exclude` |
| Agent 显示不录制 | `machineCountLimit` 过小；另有实例占满配额 |
| 有用例但很少 | 采样上限；`include` 白名单过窄 |
| 完全无上报 | `appId` 与策略 `selector` 不一致；`SP_API_URL` 不可达；Agent 不在线 |

## 录制环境与回放环境分开

| 环境 | Agent 录制 | 说明 |
|------|------------|------|
| 生产 / 预发 | 开启 | 采集真实流量建用例库 |
| 测试 / CI 回放机 | 关闭或极低采样 | 避免回放时再录一套数据污染用例库 |

按来源环境筛选用例时，保持录制与查询使用一致的 `sp.mocker.tags`（如 `env=prod`）。

## 调整录制范围 {#调整录制范围}

默认策略不满足时——比如要控制采样率、只录部分接口、限定录制时段——写一份应用级 `RecordingPolicy`（`priority > 0` 覆盖全局默认）：

```bash
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
```

可调项：`ratePerHundredSeconds`（采样）、`timeWindow`（时段）、`operations.include/exclude`（接口范围）、`serializeSkip`、`timeMock`。逐字段说明与完整示例见 [策略 YAML 指南 · RecordingPolicy](/zh/testing/policy-yaml-guide#recordingpolicy)。

::: warning `machineCountLimit: 1` 慎用
该字段限制同环境**同时录制**的实例数。设为 `1` 时，首个占坑实例下线后配额可能长期不释放，其它实例会显示不录制。生产策略建议省略该字段（不限）或设为不小于实例数。
:::

::: info 两个已知边界

* 录制路径上的 `spec.sensitiveData` 目前**不会**改变入库内容；查看时脱敏见 [SensitivePolicy](/zh/testing/policy-yaml-guide#related-configuration)。
* 修改 `operations` 包含/排除也会影响**回放调度**的操作范围。
  :::

## 下一步

用例已经躺在库里了 → **[回放与对比](/zh/testing/replay-and-diff)**：把它们变成一次回归运行。

---

---
url: /zh/platform/getting-started/quick-start.md
---
# 快速入门

使用本地 Kind Kubernetes 集群，在几分钟内开始使用 SP-Istio Agent。

::: tip 用 sp CLI 做 Java 录制回放
需要 Java 流量的录制、回放与差异测试（agent、策略、回放）？见 [测试快速开始](/zh/testing/getting-started)；用 `sp` 做自动化见 [CLI 快速入门](/zh/testing/getting-started)。
:::

::: info SESSIFY 已预装
本快速入门演示环境已预装并启用 SESSIFY（`@softprobe/sessify`），无需再次安装。要把 SDK 集成进你自己的前端应用，见 [SESSIFY 集成](/zh/platform/sessify)。
:::

## 先决条件

* **操作系统**: macOS (或带有 Docker 的 Linux)
* **所需工具**:
  * [Docker Desktop](https://www.docker.com/products/docker-desktop)
  * [Kind](https://kind.sigs.k8s.io/) - `brew install kind`
  * [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/) - `brew install kubectl`
  * [Istio CLI](https://istio.io/latest/docs/setup/getting-started/#download) - `brew install istioctl`

### 一次性安装所有工具

```bash
brew install kind kubectl istioctl
```

## 步骤 1：设置带有 Istio 的 Kind 集群

创建一个 Kind 集群并安装带有 OpenTelemetry Operator 的 Istio：

```bash
curl -L https://raw.githubusercontent.com/softprobe/softprobe/refs/heads/main/scripts/cluster-setup.sh | sh
```

该脚本将：

* 使用 Kind 创建一个本地 Kubernetes 集群
* 安装 Istio 服务网格
* 安装 OpenTelemetry Operator 用于遥测数据收集

## 步骤 2：安装 Travel 演示

部署带有 SP-Istio Agent 的演示应用程序：

```bash
# 安装 Softprobe Istio WASM 插件 (使用从账户设置下载的 minimal.yaml)
kubectl apply -f minimal.yaml

# 安装演示应用
kubectl apply -f https://raw.githubusercontent.com/softprobe/softprobe/refs/heads/main/examples/travel/apps.yaml

# 暴露演示
sleep 10 && kubectl port-forward -n istio-system svc/istio-ingressgateway 8080:80
```

## 步骤 3：试用演示

1. 在浏览器中打开 [`http://localhost:8080/`](http://localhost:8080/)
2. 选择一 **对** 城市
3. 搜索航班
4. 使用任何测试信息完成预订
5. 使用虚假信息处理付款

## 步骤 4：在 Softprobe 仪表板中查看结果

在产生一些流量后：

1. 前往 [Softprobe 仪表板](https://dashboard.softprobe.ai)
2. 在左侧导航菜单中导航到 **Travel View**
3. 探索捕获的请求和业务级追踪

### 演示视频

https://github.com/user-attachments/assets/dc8c68db-dd8b-4da8-a6e2-346adf6ecffb

## 清理

完成演示后，清理 Kind 集群：

```bash
kind delete cluster --name sp-demo-cluster
```

::: tip 恭喜！
你已成功跑通 Softprobe 并看到实际效果。下一步：

* [生产部署](/zh/platform/deployment/installation) —— 部署到你的生产集群
* [配置指南](/zh/platform/configuration/config) —— 自定义采集规则
* [高级概念](/zh/platform/advanced-guides/concepts) —— 了解 Softprobe 的工作原理
  :::

---

---
url: /zh/testing/getting-started.md
---

# Softprobe 测试快速开始

欢迎使用 Softprobe 测试！本指南将指导您在数分钟内运行预构建的 Travel OTA 演示项目以掌握录制与回放的完整流程，并在页面底部提供将您自己的 Java 服务接入 Softprobe 的极简指南。

::: tip 选择你的操作界面
下方涉及注册应用、查看录制、回放的步骤，都可以在**网页控制台**或 **`sp` 命令**里完成。用每个卡片顶部的切换选一种——你的选择会记住，并在整个文档站点保持一致。
:::

***

## 前置条件

* 本地安装了 **Java 8 或更高版本**（推荐 Java 17/21）且已配置到 `PATH` 环境变量中。
* 已安装 **`sp` 命令** (`curl -fsSL https://install.softprobe.ai/install.sh | bash`)——详见 [安装 CLI](/zh/testing/installation/)。
* 您的 **Softprobe Helm chart** 已安装并运行（Softprobe 后端服务可用，例如：`http://<您的后端主机地址>:8090`）——还没部署的话按 [服务端（Helm）](/zh/testing/installation/server) 先装好。

***

## 1. 验证 Java 安装

确保本地 Java 已正确配置，并可在终端中使用：

```bash
java -version
```

## 2. 下载演示应用 JAR 包

下载预构建的 [Travel OTA](https://github.com/softprobe/demo-ota) 应用程序 JAR 包（您可以 [点击下载 travel-ota.jar](https://github.com/softprobe/demo-ota/releases/download/v1.1.0/travel-ota.jar) 直接通过浏览器下载，也可以运行下方命令）：

```bash
curl -L -O https://github.com/softprobe/demo-ota/releases/download/v1.1.0/travel-ota.jar
```

## 3. 下载 Softprobe Agent 包

下载 Softprobe Java Agent 软件包（您可以 [点击下载 sp-agent.jar](https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar) 直接通过浏览器下载，也可以运行下方命令）：

```bash
curl -fsSL -o sp-agent.jar https://install.softprobe.ai/artifacts/agent/latest/sp-agent.jar
```

更多不可变发布版本见 [下载 Java Agent](/zh/testing/download-java-agent)。

## 4. 注册应用程序

在系统中注册该应用，以获取一个唯一的 `appId`（一个 16 位的十六进制标识符）：

1. 打开您的 Softprobe Dashboard 控制台。
2. 导航至 **Apps** 并点击 **Create App**。
3. 输入 `travel-ota` 作为应用名称并点击 **Save**。
4. 复制自动生成的 **App ID**。

在终端中运行以下命令以注册应用：

```bash
export SP_API_URL=http://localhost:8090   # 指向您的 Helm/本地后端
sp app create travel-ota
```

保存 JSON 响应中返回的 `appId`。

## 5. 启动附带 Agent 的应用

运行应用并加上 `-javaagent` 参数，并传入您的 `appId`：

```bash
java -javaagent:sp-agent.jar \
     -Dsp.app.id=<您的应用ID> \
     -Dsp.api.url=http://localhost:8090 \
     -jar travel-ota.jar
```

*(如果您的 Helm 部署后端服务在其他地址，请将 `http://localhost:8090` 替换为实际的后端地址)*。

应用现在已经在本地运行，可通过浏览器访问：<http://localhost:8080>。

## 6. 进行一笔交易 (产生录制流量)

在浏览器中打开 <http://localhost:8080>：

1. 点击 **Search** 浏览可用航班。
2. 选择任意航班，点击 **Book**。
3. 完成支付与结算流程 (Checkout)。

Softprobe Agent 会自动拦截并捕捉这一连串的调用流量。

## 7. 查看录制的数据

1. 登录 Softprobe Dashboard 控制台。
2. 进入 **Workbench** (工作台) 或 **Recordings** 页面。
3. 从应用下拉菜单中选择 `travel-ota`。
4. 直观地浏览刚刚捕获的调用链 (Trace) 及深层依赖关系。

列出过去 10 分钟内为该应用录制的所有用例：

```bash
sp record case list --app <您的应用ID> --since -10m
```

## 8. 回放录制数据

回放功能会将录制的请求发送到目标环境以检测代码变更是否引起回归，并且会自动 Mock 外部依赖，您无需配置数据库或外部第三方服务。

1. 进入 **Replays** 页面，点击 **New Replay Plan**。
2. 选择 `travel-ota` 应用。
3. 选择要回放的测试用例。
4. 设置目标环境为 `http://localhost:8080` 并点击 **Run**。

在终端中发起回放计划指向您本地运行的实例：

```bash
sp replay run --app <您的应用ID> --env http://localhost:8080
```

实时监控并查看回放进度直到执行完毕：

```bash
sp replay status <回放计划ID> --watch
```

***

## 接入您自己的应用

接入您自己的应用程序与运行 Travel OTA 演示项目**完全相同**！您的应用也只是另外一个由相同的 Softprobe Java Agent 和您的自定义应用 ID 启动的 `.jar` 软件包。

接入步骤十分简单：

1. **注册应用**：在 Web 控制台的 **Apps** → **Create App** 中，或者在命令行中运行以下命令，注册您的服务以获取全新的 16 位 App ID：
   ```bash
   sp app create <您的应用名称>
   ```
2. **挂载 Agent**：使用相同的 `-javaagent` 参数启动您自己的 JVM 服务，传入新的 App ID 和 Helm 部署的后端地址：
   ```bash
   java -javaagent:sp-agent.jar \
        -Dsp.app.id=<您的应用ID> \
        -Dsp.api.url=http://<您的后端主机地址>:8090 \
        -jar your-own-application.jar
   ```
3. **录制与回放**：就像运行演示应用一样，正常发起测试流量，然后列出录制用例并随时发起回放。

## 下一步：核心流程

跑通演示只是开始。接下来沿**核心流程**四步，把你自己的应用真正管起来：

1. **[录制流量](/zh/testing/recording)** — 在生产/预发采集真实用例，建起回归用例库
2. **[回放与对比](/zh/testing/replay-and-diff)** — 每次发版前跑一次回归
3. **[审查差异](/zh/testing/review-diffs-in-the-web-ui)** — 看懂失败用例，接受不是 bug 的差异
4. **[配置对比规则](/zh/testing/compare-rules-web-ui)** — 把总在变的字段沉淀成规则，不再误报

更深入的配置见 [Java Agent 配置](/zh/testing/java-agent)（JVM 参数、Tomcat/Docker）与 [策略概览](/zh/testing/policies)（声明式 YAML，进 CI/GitOps）。

---

---
url: /zh/testing/installation/code.md
---

# 手动启动 Softprobe Web UI

大多数测试工作都在浏览器里的 **Softprobe Web UI** 中完成——录制、回放、查看 trace 等。

本页说明如何在**自己的开发机**上启动该 UI，适合在笔记本或工作站上做个人测试。这与团队共用的**共享 Web 工作台**（同一内网 URL、多人访问）不是同一种方式；可选的服务端安装见安装页中的 [Spcode Service](./#spcode-service)。

## 在开发机上启动 UI

```bash
sp code web --port 4096
```

然后在浏览器打开 `http://127.0.0.1:4096`（或你指定的地址与端口）。UI 会使用你在 `sp setup` 中配置的**个人**后端 URL 与设置。

也可以在终端中启动编码体验（不打开浏览器）：

```bash
sp code
```

若 Web UI 无法启动，请运行 `sp doctor` 并按提示修复，或通过 `sp upgrade` 更新安装。

## 开发机 vs 共享工作台

| | 开发机（本页） | 共享 Web 工作台（可选） |
|--|----------------|-------------------------|
| **适用对象** | 你本人、本机 | 组织内多人共用同一入口 |
| **典型场景** | 本地开发时自测 | 同事在同一网络用浏览器打开同一工作台 |
| **启动方式** | 在 shell 中运行 `sp code web` | 安装时选择 `sp setup --install-spcode-service`（[Spcode Service](./#spcode-service)） |
| **配置** | `sp setup` 写入的 Softprobe 设置 | 与安装 Spcode Service 的账号相同的 Softprobe 设置 |

只有你使用这台机器时，用开发机方式即可。需要多人通过浏览器共用同一工作台时，再考虑 Spcode Service。

## 下一步

UI 能打开后，接入流程就齐了。还没跑过完整流程的话从 [快速开始](/zh/testing/getting-started) 走一遍；已有录制用例的话，回放后在这里 [审查差异](/zh/testing/review-diffs-in-the-web-ui)。

---

---
url: /zh/testing/installation/metrics-data-plane.md
---
# 指标数据平面

Softprobe 指标复用**与日志相同的采集器和 Parquet 存储体系**：OTLP → Vector → 一分钟聚合 → `metrics/` 数据集下的 Parquet → 产品 HTTP 查询。它面向的是**临时诊断**场景（例如“我们是否收到了 agent 日志？”），而非 Prometheus/Grafana 仪表盘。

需要 chart **v4.3.x+** 并启用[统一日志管道](./server#unified-log-pipeline)（默认启用）。指标会复用该 Vector Deployment —— 无需第二个时序数据库，也无需额外的 pod。

## 摄入 —— `POST /v1/metrics`

客户端将 OpenTelemetry 指标 POST 到 Softprobe。当遥测管道就绪后，sp-backend 会将其代理转发给 Vector。

| Content-Type | Payload |
|--------------|---------|
| `application/x-protobuf` | OTLP `ExportMetricsServiceRequest` |
| `application/json` | OTLP metrics JSON（`resourceMetrics` …） |

**接受的格式：** 仅 OTLP protobuf 和 OTLP JSON。该端点**不**接受扁平 JSON 的指标方言。OTLP JSON 在 Softprobe 处被接受，并以 protobuf 形式转发给 Vector。

| Pipeline state | Response |
|----------------|----------|
| Ready | 接受/代理后返回 `200` |
| Not ready | `503` —— Softprobe **不会**在丢弃数据的同时返回成功 |

示例（OTLP JSON）：

```bash
curl -sS -X POST "$SP_API_URL/v1/metrics" \
  -H 'Content-Type: application/json' \
  --data-binary @export-metrics.json
```

已埋点的后端还会在进程内向同一条 Vector 指标路径**发出**目录计数器（不通过 HTTP 自我 POST）。

## 查询 —— `GET /api/recorder/metrics`

对 `metrics/` 下的 Softprobe Parquet 进行有界、只读的查询。

**必填查询参数：**

| Parameter | Description |
|-----------|-------------|
| `metric_name` | 要选择的指标名称 |
| `since` | ISO-8601 UTC，闭区间（含） |
| `until` | ISO-8601 UTC，开区间（不含）—— 时间窗为 `[since, until)` |

可选的精确匹配过滤器仅作用于提升后的标签列：`status`、`kind`、`source`、`result`、`content_type`。高基数选择器（`trace_id`、完整 URL、异常消息）以及非目录键（例如 `app_id`）会被拒绝。

除了要求提供有效的边界外，产品层面**没有**行数 `limit`，也**没有**最大时间跨度限制（与日志查询理念一致）。缺失的分区会返回 HTTP 200 且 `rows` 为空。

```bash
curl -sS "$SP_API_URL/api/recorder/metrics?metric_name=sp.logs.ingest.requests&since=2026-07-10T18:00:00Z&until=2026-07-10T18:05:00Z"
```

成功响应的示例结构：

```json
{
  "lookup": {
    "metric_name": "sp.logs.ingest.requests",
    "windows": [{ "since": "2026-07-10T18:00:00Z", "until": "2026-07-10T18:05:00Z" }],
    "filters": {}
  },
  "rows": [
    {
      "timestamp": "2026-07-10T18:01:00Z",
      "metric_name": "sp.logs.ingest.requests",
      "metric_type": "sum",
      "service_name": "sp-backend",
      "value": 3,
      "status": "ok",
      "kind": "agent_json"
    }
  ],
  "warnings": []
}
```

错误响应体不得暴露 Parquet 路径、bucket 名称或存储凭证。

## 后端 P0 目录（R1）

Softprobe 从日志摄入/转发路径发出以下序列：

| Name | Meaning |
|------|---------|
| `sp.logs.ingest.requests` | 每一次 `POST /v1/logs` 尝试（`status`、`kind`） |
| `sp.logs.ingest.records` | 转换后的记录数（`kind`、`source`） |
| `sp.logs.forward.results` | 导出结果（`success` / `failure` / `skipped_*`） |
| `sp.logs.forward.duration_ms` | 导出耗时直方图 |

经过 Vector 的一分钟聚合后，预计约一分钟内即可查询到相应行（CI 最多轮询 **70 秒**）。

## 不在范围内

* 将 Prometheus 抓取、Grafana 或 PromQL 作为 Softprobe 的产品路径
* `sp metrics` CLI（R1 契约为 HTTP API）
* Agent 侧的 `sp.agent.logs.*` 发射器（后续轮次）
* 面向终端用户的直接 Parquet 或存储凭证

---

---
url: /zh/testing/supported-frameworks.md
---

# 支持的框架（Java）

Softprobe 测试通过 Agent 模块插件织入常见 Java 技术栈。下列为当前文档记载的覆盖范围；[`sp-agent-java`](https://github.com/softprobe/softprobe/tree/main/backend/sp-agent-java) 中会持续新增集成。

::: tip
若未列出你的栈，请确认是否使用已支持的客户端（例如基于 MyBatis 或 Hibernate 的 JDBC 层）。入口必须由 Agent 已织入的 HTTP/RPC Provider 模块承接。
:::

## 基础能力

* Java Executors
* 系统时间（`DynamicClass` — 默认策略在回放时强制 Mock）
* 用户配置的动态类型

## 缓存

* Caffeine Cache
* Guava Cache
* Spring Cache

## Spring / HTTP 入口

* Spring Boot 1.4+、2.x+
* Servlet API 3+、5+

## HTTP 客户端

* Apache HttpClient 4.0+
* OkHttp 3.0 – 4.11
* Spring WebClient 5.0+
* Spring `RestTemplate`
* Feign 9.0+
* Elasticsearch Client 7.x

## Redis

* `RedisTemplate`
* Jedis 2.10+、4+
* Redisson 3.0+
* Lettuce 5.x、6.x

## 持久化

* MyBatis 3.x、MyBatis-Plus、TkMyBatis
* Hibernate 5.x

## NoSQL

* MongoDB 驱动 3.x、4.x

## RPC

* Apache Dubbo 2.x、3.x
* Alibaba Dubbo 2.x
* SOFA RPC（后端模型中的 provider/consumer 分类）

## 认证

* Spring Security 5.x
* Apache Shiro 1.x
* jCasbin 1.x
* Auth0 JWT 3.x
* JJWT 0.1+、jjwt-api 0.10+

## Netty

* Netty 服务端 3.x、4.x

## 配置中心

* Apollo Config 1.x、2.x

## 后端依赖分类

回放匹配在 Mock 策略中使用**依赖**分类（非入口类型）。与 [`MockCategoryType`](https://github.com/softprobe/softprobe) 对齐的常用名称：

| 分类 | 典型用途 |
|------|----------|
| `HttpClient` | 对外 HTTP |
| `Database` | JDBC / ORM SQL |
| `Redis` | Redis 命令 |
| `DubboConsumer` / `SofaConsumer` | RPC 客户端 |
| `QMessageProducer` | 消息发送 |
| `DynamicClass` | 时间、随机、缓存、加解密 |
| `ConfigFile` | 配置读取（对比时常跳过） |

入口分类（`Servlet`、`DubboProvider` 等）表示被录制的**主** API。

## 相关文档

* [Java Agent](/zh/testing/java-agent)
* [Mock 策略](/zh/testing/policies#mock-policy)
* [工作原理](/zh/testing/how-it-works)

---

---
url: /zh/testing/reference/log-correlation-ids.md
---
# 日志关联 ID

> 本页翻译可能滞后于英文版，如有出入以[英文版](/en/testing/reference/log-correlation-ids)为准。

当回放失败时，你需要一个 **`trace_id`** 和一个有界的时间窗口，才能在一次查询中，同时拉取被测应用、Java agent 和 sp-backend 三方的关联日志。本页说明每个 id 的含义、**在哪里找到它**，以及**如何分诊**统一日志的查询结果。

需要在你的部署环境或本地 compose 栈中启用统一日志管道（Vector → Parquet）。如果该管道被禁用，日志查询会快速失败并给出明确的错误——它们不会回退到其他日志存储。

**命令参考：** [sp logs](/zh/testing/commands/logs) · **API：** `GET /api/recorder/logs?trace_id=…&since=…&until=…`

***

## ID 速查

| ID | 支持 v1 日志查询？ | 它标识什么 | 用于日志检索 |
|----|----------------|--------------------|--------------------|
| **`traceId`** | **是**（`--trace-id`） | 一次请求流的 W3C OpenTelemetry trace | **唯一的 v1 查询键**——`[since, until)` 内该 trace 上所有 agent/app/backend 日志行 |
| **`replayId`** | 否 | 某个已录制 case 的一次回放**尝试** | 定位失败的 case 及 diff；从同一行复制 **`traceId`** 用于查日志 |
| **`planId`** | 否 | 一个回放**计划**（`sp replay run` 产生的批次） | 限定诊断 / case 列表范围；复制每个 case 的 **`traceId`** 用于查日志 |
| **`planItemId`** | 否 | 计划内的一个 **case/操作** | 同上——与 case 列表配合使用，不能作为日志查询键 |
| **`diffId`** | 否 | 一行 compare/diff 结果 | 与 `sp replay diff get` 配合使用，不用于查日志 |

每一行日志都带有 **`source`**：`agent`（Java agent 诊断日志）、`app`（由 agent 采集的被测应用日志）或 `backend`（sp-backend 诊断日志）。API/Parquet 使用不带前缀的列名（`source`、`replay_id`、……）。在 Vector 归一化之前，OTLP 线路格式可能使用 `sp.*` 键。

**不在 v1 日志查询结果中的字段：** `sessionId` / `sp.session_id`。

**在 v1 中被拒绝的项：** `--replay-id`、`--plan-id`、`--plan-item-id`、`--include-recording-log`、`GET /api/record-logs/*`、`GET /api/replay-logs/*`。

***

## 在哪里找到 `traceId`（查日志的主键）

| 来源 | 方法 |
|--------|-----|
| 失败回放 case 列表 | `sp replay case list --plan <planId> --failed --json` → `data.items[].traceId` |
| 回放元数据 | `sp replay metadata <replayId> --json` → `traceId` |
| 诊断工作流 | `sp diagnose replay <planId> --failed-only --json` → 再看 case 列表取 `traceId` |
| CI / `make e2e` | pytest 失败信息下的 **Softprobe correlation** 区块 → 每个 case 的 `trace_id` 字段 |
| 已录制 case | `sp record case list --app <appId> --since -24h --json` → 入口 case 的 trace id |

**重要：** 对于回放失败，请使用**回放 case 上的 `traceId`**（与录制时的 trace 相同——schedule 会把它放在 `traceparent` 上）。**不要**从最新的录制列表页或健康检查流量（`/index.html`、`/`）里去猜——那些 trace 是不相关的噪声。

pytest 输出中的 `replayId`、`planId` 和 `planItemId` 仅供人工参考以及 diff/diagnose 命令使用——它们**不是** v1 日志查询键。

***

## 查询关联日志

每次查询都需要 **`trace_id`** 外加 **`since`** 和 **`until`**。使用 **ISO-8601 UTC**（例如 `2026-06-27T10:00:00Z`）。`since` 为**闭区间**，`until` 为**开区间**（`[since, until)`）。

### CLI（主要方式）

```bash
sp logs --trace-id <traceId> --since <start> --until <end> [--json]
```

给脚本和 AI agent 使用时加上 **`--json`**。默认输出是人类可读的文本。

### HTTP API（当 `sp` 不在 PATH 中时）

```bash
export SP_API_URL="${SP_API_URL:-http://127.0.0.1:18090}"
TRACE_ID="2057ad46a7ce03d3955385f2a4142d29"
SINCE="2026-06-27T10:00:00Z"
UNTIL="2026-06-27T10:05:00Z"

curl -s "${SP_API_URL}/api/recorder/logs?trace_id=${TRACE_ID}&since=${SINCE}&until=${UNTIL}" \
  -H "Accept: application/json" -o /tmp/trace-logs.json
```

### 选择 `since` / `until`

**当你有 case 时间戳（`recordTime` 和 `replayTime`）时：** 分别做**两次** ±2 分钟的查询——每个锚点各查一次——然后合并结果行。绝不要在一次查询中把录制时间跨到回放时间。参见 [sp logs — 按 case 范围查询](/zh/testing/commands/logs#case-scoped-lookup-dual-windows)。

**否则：**

1. 从失败点附近的窗口开始（例如从计划完成前五分钟到完成后一分钟）。
2. 如果你预期的某个 `source`（`agent`、`app`、`backend`）行数为零，则加宽窗口。
3. 执行 `make compose-parquet-clean` 后的 E2E：重跑会话，然后轮询最多约 180 秒等待 Vector 摄取。
4. v1 会返回窗口内**所有**匹配的行——输出量大时请在本地重定向或用管道处理：

```bash
sp logs --trace-id <id> --since … --until … > /tmp/trace.log
grep ERROR /tmp/trace.log | head -20
```

v1 日志查询没有 `--limit` 参数。

***

## 分诊统一日志结果

拉取日志后，按以下顺序处理。

**`sp --json logs`** 会把 API body 包裹在 `.data` 里——请使用 `jq '.data.rows'`、`jq '.data.warnings'`。**`curl`** 直接保存 API JSON——请使用 `jq '.rows'`、`jq '.warnings'`。

```bash
# 使用 sp --json logs 之后（CLI 信封结构）
jq '.data.rows | length' /tmp/trace-logs.json
jq '[.data.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/trace-logs.json
jq '.data.warnings' /tmp/trace-logs.json
jq -r '.data.rows[] | select(.source=="backend" and .severity=="ERROR") | "\(.timestamp) \(.body)"' /tmp/trace-logs.json | head -20

# 使用 curl GET /api/recorder/logs 之后（API body 位于顶层）
jq '.rows | length' /tmp/trace-logs.json
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/trace-logs.json
jq '.warnings' /tmp/trace-logs.json
jq -r '.rows[] | select(.source=="backend" and .severity=="ERROR") | "\(.timestamp) \(.body)"' /tmp/trace-logs.json | head -20
```

### 症状对照表

| 你看到的现象 | 可能含义 | 下一步 |
|--------------|----------------|--------|
| `rows` 为空 + `warnings` 非空 | Parquet 读取器/schema 不匹配（例如 backend 镜像过旧） | 重新构建 sp-backend；确认 API 行使用的是 `source` 而非 `sp.source` |
| `rows` 为空 + `warnings` 为空 | `trace_id` 错误、窗口过窄，或摄取有延迟 | 使用回放 case 的 `traceId`；加宽 `[since, until)`；重跑回放 |
| 同时有 `agent`、`app` 和 `backend` 的行 | 该 trace 的管道健康 | 阅读 `body` 中的 ERROR/WARN；compare 失败时用 `sp diagnose` 的 diff |
| 只有 `backend`，没有 `app`/`agent` | agent 未导出或应用日志沉默 | 检查 agent 挂载、`sp.enable.debug`、应用 logger 级别 |

**同一个业务请求**的录制阶段与回放阶段日志行，在回放时共享一个 **`trace_id`**（录制的 trace 在 `traceparent` 上被复用）。一次 trace 查询即可同时包含两者，无需 `--include-recording-log`（在 v1 中已移除）。

***

## 读取响应

**HTTP API** 顶层字段：

* `lookup` —— 类型 `trace`、其值，以及调用方给的 `[since, until)` 边界
* `rows[]` —— 每一行：`timestamp`、`severity`、`body`、`service_name`、`source`，以及可选的 `trace_id`、`span_id`、`replay_id`、`plan_id`、`plan_item_id`
* `warnings` —— 非致命的 schema 跳过或类似情况（可能为空）

**`sp --json logs`** 返回一个 CLI 信封：`{"ok":true,"command":"logs","data":{...}}` —— 在脚本中请使用 `.data.rows` 和 `.data.warnings`。

v1 响应**不**包含 `source_summary`。请用 `jq` 在本地计算每个 source 的计数（见上文）。

完整字段参考见 [日志查询字段](/zh/testing/commands/log-query-fields)。

***

## Pytest / `make e2e` 失败

当出现回放相关的测试失败时，pytest 会打印：

1. **Softprobe correlation** —— `trace_id`（用于日志查询），以及供参考的 `replay_id`、`plan_id`、`plan_item_id`。
2. **Unified logs** —— 可选摘要：当 hook 执行了 curl 时，给出行数以及每个 `source` 的计数。

从该区块复制 `trace_id` 以及建议的 `since`/`until`，然后先运行上面的分诊命令，再深入应用代码。

***

## 典型失败排查流程

```text
1. sp replay case list --plan <planId> --failed --json
      → 复制 traceId（以及用于 diff/diagnose 的 replayId）

2. sp logs --trace-id <traceId> --since … --until … [--json]
      → 分诊：计数 → 来源 → warnings → 阅读 backend/agent/app 的 body
      （仅当 sp 不可用时才用 curl GET /api/recorder/logs）

3. sp diagnose replay <planId> --failed-only --out-dir .sp-work --json
      → 阅读 diff 产物，查看字段级 compare 失败

4. 若清空 Parquet 后日志为空：重跑 replay/e2e 并轮询摄取
```

***

## 等效 API

```http
GET /api/recorder/logs?trace_id=<id>&since=<ts>&until=<ts>
```

不支持的查询参数（`replay_id`、`plan_id`、`plan_item_id`、`include_recording_log`、……）会在读取 Parquet 之前就被拒绝。

验证规则和 JSON 结构见 [sp logs](/zh/testing/commands/logs)。

***

## 相关

* [概念 — 回放计划与各类 id](/zh/testing/agents/concepts#trace-replay-and-plan-ids)
* [诊断回放失败示例](/zh/testing/examples/agent-diagnose-replay)
* [sp replay case](/zh/testing/commands/replay-case)
* [sp trace](/zh/testing/commands/trace)

---

---
url: /zh/testing/commands/log-query-fields.md
---
> 本页翻译可能滞后于英文版，如有出入以[英文版](/en/testing/commands/log-query-fields)为准。

# 日志查询字段

**代理何时使用本页：** 用于解读 [`sp logs`](./logs) 或 `GET /api/recorder/logs` 返回的行——包括字段名称、含义，以及关联 id 何时可能缺失。

本参考仅描述 **CLI 与 API 的查询输出**。它不涉及 Parquet 文件路径、分区布局，也不说明如何直接查询存储。所有查询请使用 [`sp logs`](./logs) 或规范的 HTTP API。

**查询键：** v1 **只接受 `trace_id`**。可选的关联标签在数据被采集时可能出现在行中——它们不是过滤键。

**命名：** API 与 Parquet 的行使用**无前缀**的列名（`source`、`replay_id`……）。在传输链路上，OTLP 在 Vector 将它们映射进存储之前可能使用 `sp.source`、`sp.replay_id` 等带前缀的名称。

***

## 行出现在哪里

每次成功查询都会在 `data.rows`（CLI `--json`）或 API 的 `rows` 数组中返回一条**按时间顺序排列的行流**。行按事件 `timestamp` 升序排列。

CLI 的人类可读输出打印的逻辑字段与 API JSON 相同。

关于 `--trace-id` 查询键、排障流程和必需的时间范围，请参见 [sp logs](./logs)。

***

## 字段参考

每一行都包含下面的核心字段。关联字段**仅在发出方运行时于打日志时知晓其值**时才包含——在缺乏上下文的行上它们可能不存在（参见[缺失的关联字段](#缺失的关联字段)）。

| Field | Always present | Description |
|-------|----------------|-------------|
| `timestamp` | yes | 日志行的事件时间。JSON 中为 ISO-8601 UTC（例如 `2026-06-27T10:00:10.123Z`）。用于按时间顺序排序以及调用方的 `[since, until)` 过滤。 |
| `severity` | yes | 发出方 logger 归一化后的严重级别文本（例如 `DEBUG`、`INFO`、`WARN`、`ERROR`）。每个组件通过**其自身原生的日志配置**控制发出哪些级别——Softprobe 不施加产品级的严重级别过滤。 |
| `body` | yes | 完整的日志消息文本。v1 返回完整的 `body`；查询结果不会截断消息内容。 |
| `service_name` | yes | 该行的运行时服务标识（例如 `travel-ota`、`sp-backend`）。标识产生该行的是哪个进程。 |
| `source` | yes | 产生该行的 v1 流水线来源。固定取值：`agent`、`app` 或 `backend`（参见 [source 取值](#source-取值)）。 |
| `trace_id` | when known | 打出该行时处于活动状态的请求或工作单元的 W3C OpenTelemetry trace id。**v1 唯一的查询键。** |
| `span_id` | when known | 打出该行时活动 span 的 OpenTelemetry span id。 |
| `replay_id` | when known | 打日志时回放上下文处于活动状态的那一次回放**尝试**的 id。可选标签——不是查询键。 |
| `plan_id` | when known | 打日志时计划上下文处于活动状态的回放**计划** id。可选标签——不是查询键。 |
| `plan_item_id` | when known | 回放计划内的某个 case 或操作。可选标签——不是查询键。 |

**不在 v1 查询结果中：** `session_id` / `sp.session_id`。

***

## `source` 取值

| Value | Meaning | Typical `service_name` examples |
|-------|---------|--------------------------------|
| `agent` | Java agent 自诊断（插桩、导出、agent 内部日志） | 插桩下的应用服务名 |
| `app` | 由 agent 捕获的被测应用日志（Logback、Log4j2、JUL） | `travel-ota`、客户应用 id |
| `backend` | 通过 OpenTelemetry 导出的 sp-backend 诊断日志 | `sp-backend` |

当你在同一次 trace 查询中只想要应用行、agent 诊断行或 backend 行时，可按 `source` 过滤或扫描。

```bash
jq '[.rows[].source] | group_by(.) | map({source: .[0], n: length})' /tmp/sp-logs.json
jq -r '.rows[] | select(.source=="backend") | .body' /tmp/sp-logs.json | head -20
```

***

## 缺失的关联字段

当运行时在打日志时**确实没有请求或回放上下文**时，关联字段（`trace_id`、`span_id`、`replay_id`、`plan_id`、`plan_item_id`）会被**省略或为空**。这是预期行为——并非查询缺陷。

常见情形：

| Situation | Typical absent fields | Why |
|-----------|----------------------|-----|
| 进程**启动**或**关闭** | 部分或全部关联字段 | 尚无活动的 HTTP/RPC 请求或回放派发，或上下文已被清除 |
| **后台 / 内务处理**行 | `trace_id`、`span_id`、replay/plan id | 录制/回放流量之外的线程或定时器工作 |
| **agent 或 backend 空闲**诊断 | `replay_id`、`plan_id`、`plan_item_id` | 仅带 trace 上下文的诊断行，或没有入站 W3C 上下文 |
| 行上**未设置计划上下文** | `plan_id`、`plan_item_id` | 即便在回放期间，该行也是在计划项派发之外打出的 |

诊断失败的回放时，请按回放 case 或 pytest 关联块中的 **`trace_id`** 查询。当你需要限定在回放范围内的行时，在本地输出中按可选的 `replay_id` 过滤。

**限定 case 的诊断：** 当 case 行包含 **`recordTime`**（API 中为 `requestDateTime`）和 **`replayTime`** 时，请使用两个各 ±2 分钟的窗口（每个锚点一个），而不是从录制到回放的单一跨度。回放窗口通常包含相关行；录制窗口往往为空。参见 [sp logs — 限定 case 的查询](./logs#case-scoped-lookup-dual-windows)。

***

## 各组件的日志归属

| `source` | 谁控制 `severity` 以及发出什么 |
|----------|-----------------------------------------------|
| `agent` | Agent / JVM 日志配置（`sp.log.path`、`sp.log.console`、`sp.enable.debug` 等） |
| `app` | 应用的 Logback、Log4j2 或 JUL 设置 |
| `backend` | sp-backend 日志与 OpenTelemetry 日志导出配置 |

Softprobe 附加关联 id 并转发各 logger 已经发出的行。它不会更改应用的日志级别，也不会在产品层面过滤严重级别。

***

## 示例行（JSON）

来自 [`sp logs --json`](./logs) 或 `GET /api/recorder/logs`：

```json
{
  "timestamp": "2026-06-27T10:00:10.123Z",
  "severity": "WARN",
  "body": "Replay comparison mismatch",
  "service_name": "sp-backend",
  "source": "backend",
  "trace_id": "2057ad46a7ce03d3955385f2a4142d29",
  "span_id": "8d10c94a2a6f4e11",
  "replay_id": "6891fd300c676b31",
  "plan_id": "6a3f2aad59f0c4655b0f99da",
  "plan_item_id": "6a3f2aad59f0c4655b0f99da:1"
}
```

同一服务的一条启动行可能完全省略关联字段：

```json
{
  "timestamp": "2026-06-27T09:59:55.000Z",
  "severity": "INFO",
  "body": "Started SpBootApplication in 4.2 seconds",
  "service_name": "sp-backend",
  "source": "backend"
}
```

***

## 范围之外（v1）

本参考仅涵盖统一流水线的**查询输出**。除非另行说明，以下内容不属于 v1：

* 录制 trace 表、指标表、回放读迁移、历史回填
* 非回放路径的服务日志（dashboard、auth 及其他 Helm/workspace 服务）
* 直接访问 Parquet 文件、对象存储凭证或独立查询工具

***

## 相关内容

* [sp logs](./logs) — 命令参考、flag、排障与 API 映射
* [日志关联 ID — 查找与使用 id](/zh/testing/reference/log-correlation-ids)
* [诊断回放失败示例](/zh/testing/examples/agent-diagnose-replay)

---

---
url: /zh/testing/examples.md
---

# 示例

Testing 示例展示如何用 `sp` 自动化录制、回放、诊断和 CI 策略检查。

---

---
url: /zh/testing/policies.md
---

# 策略概览

Softprobe 测试用**声明式 YAML 策略**（`apiVersion: softprobe.ai/v1`）控制行为，而非临时请求参数。策略按应用匹配、按 `metadata.priority` 合并，由 sp-backend 在运行时生效。

::: tip 策略是可选的调优，不是前提
内置的全局默认策略（priority 0）让录制与回放开箱即用。只有当你要**改变**默认行为——控制采样、缩小操作范围、忽略噪声字段——才需要写自己的策略（`priority > 0` 覆盖）。先能跑通核心流程，再回来收紧。
:::

**操作流程**与**策略配置**分开阅读：

| 阶段 | 操作文档 | 策略配置 |
|------|----------|----------|
| **1 · 录制** | [录制流量](/zh/testing/recording) | 本节 [RecordingPolicy](#recording-policy) |
| **2 · 回放** | [回放与对比](/zh/testing/replay-and-diff) | 本节 [MockPolicy](#mock-policy)、[CompareRulePolicy](#compare-policy) |

逐字段说明与完整 YAML 示例：[策略 YAML 指南](/zh/testing/policy-yaml-guide) · [sp policy 命令](/zh/testing/commands/policy)

## CLI 速查

```bash
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
sp policy mock apply -f mock.yaml --json
sp policy compare apply -f compare.yaml --json
```

冲突时 **`metadata.priority`** 更高者生效。内置 priority 0 全局默认；应用策略请设 `priority > 0`。

## 按生命周期分阶段

| 阶段 | Kind | 何时配置 | CLI |
|------|------|----------|-----|
| **1 · 录制** | `RecordingPolicy` | 产生流量**之前** | `sp policy recording` |
| **2 · 回放** | `MockPolicy` | 执行 `sp replay run` **之前** | `sp policy mock` |
| **2 · 回放** | `CompareRulePolicy` | 执行 `sp replay run` **之前** | `sp policy compare` |

完整生命周期：[快速开始](/zh/testing/getting-started)

***

## RecordingPolicy {#recording-policy}

**用于[录制](/zh/testing/recording)阶段 · 产生流量之前应用**

控制 Agent **录什么**：采样、时间窗口、操作包含/排除、序列化跳过、录制时时间 Mock。

* **采样** — `ratePerHundredSeconds`（每 100 秒上限；`0` = 不录）、`machineCountLimit`（并发录制实例上限，省略 = 不限）
* **时间窗口** — `daysOfWeek`、`from` / `to`（Agent JVM 本地时区）
* **操作过滤** — `exclude`（黑名单 Glob）；非空 `include` 时变为白名单模式
* **序列化跳过** — `serializeSkip` 按类名与字段名
* **`timeMock`** — 录制时固定 `java.time.*`

**操作步骤：** [录制流量](/zh/testing/recording)

**YAML 字段与示例：** [策略 YAML 指南 · RecordingPolicy](/zh/testing/policy-yaml-guide#recordingpolicy)

::: info 说明

* `spec.sensitiveData` 在 Agent 录制路径**尚未生效**；Mock 键噪声用 `matchTolerance`，查看脱敏用 `SensitivePolicy`（见 YAML 指南相关配置）。
* 修改 `operations` 包含/排除会影响**回放调度**的操作范围，无需单独改调度文档。
  :::

```bash
sp policy recording validate -f my-recording.yaml --json
```

***

## MockPolicy {#mock-policy}

**用于[回放](/zh/testing/replay-and-diff)阶段 · 执行 `sp replay run` 之前应用**

控制回放时**依赖是否 Mock**、Mock 键容差、跨应用依赖与无匹配 Mock 时的回退。

* **`mockByDefault`** — 默认 Mock 全部依赖（`skipMock` 为例外），或反之以 `forceMock` 为例外
* **`Category:operationGlob`** — 如 `HttpClient:/payment/**`（**不是** `Servlet` 等入口类型）
* **`matchTolerance`** — 忽略易变头、查询参数、body 路径
* **`multiServiceDependencies`** — 同会话内下游应用 Mock
* **`fallback`** — `FAIL`（默认）、`PASS_THROUGH`、`RETURN_DEFAULT`

内置全局策略 **强制 Mock** `DynamicClass:SystemTime.**` / `RandomSource.**`；用户 `skipMock` 无效。

**YAML 字段与示例：** [策略 YAML 指南 · MockPolicy](/zh/testing/policy-yaml-guide#mockpolicy)

[完整依赖分类列表](/zh/testing/policy-yaml-guide#mock-categories)

***

## CompareRulePolicy {#compare-policy}

**用于[回放](/zh/testing/replay-and-diff)阶段 · 执行 `sp replay run` 之前应用**

控制回放**差异对比**中的噪声（非 Mock 行为）。

* **`excludePaths` / `includePaths`** — JSON Pointer（支持 Glob）
* **`defaults.timeToleranceMs`** 与 CEL **`validations`** — 按规则丢弃差异（含按 `category` 忽略，如 DATABASE body）
* **`operationSpecs`** — 按入口操作覆盖（**不要**在 `selector` 上写操作名）

**YAML 字段与示例：** [策略 YAML 指南 · CompareRulePolicy](/zh/testing/policy-yaml-guide#comparerulepolicy)

回放后用 `sp replay diff` 排查，再收紧策略而非改业务代码。

***

## 动态类（非 RecordingPolicy）

本地缓存等方法在**动态类配置**（控制台/API）登记，不在 `RecordingPolicy` 中。回放 Mock 通过 **MockPolicy** 的 `UserDynamic` / `DynamicClass` 规则控制。见 [策略 YAML 指南 · 相关配置](/zh/testing/policy-yaml-guide#related-configuration)。

## 策略类型与服务端模块

| Kind | CLI | 服务端模块 |
|------|-----|-----------|
| `RecordingPolicy` | `sp policy recording` | `RecordingPolicyService` |
| `MockPolicy` | `sp policy mock` | `MockPolicyService` |
| `CompareRulePolicy` | `sp policy compare` | `CompareRulesService` |

示例文件位于 `sp-policy-rules/src/main/resources/examples/`。

## Agent 工作流

```bash
# apply 前务必先 validate
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
```

CI 应在 `valid: false` 或非零退出码时失败。

## GitOps

```bash
sp policy recording export prod-policy-id -o policies/recording-prod.yaml
git commit -m "chore: sync recording policy"
```

见 [GitOps 策略](/zh/testing/examples/gitops-policies)。

## 与旧版配置的关系

policy-rules 迁移后，`sp config legacy schedule` 与 Mongo 的 `ServiceCollectConfiguration` **不再**是配置的唯一来源。用 `sp policy recording` 管理同时影响 agent 与回放范围的 operation include/exclude。

## 相关文档

* [录制流量](/zh/testing/recording)
* [回放与对比](/zh/testing/replay-and-diff)
* [策略 YAML 指南](/zh/testing/policy-yaml-guide)
* [CLI：policy 命令](/zh/testing/commands/policy)

---

---
url: /zh/testing/policy-yaml-guide.md
---

# 策略 YAML 指南

Softprobe 测试通过**声明式 YAML 策略**（`apiVersion: softprobe.ai/v1`）控制录制、回放 Mock 与差异对比。策略按应用（及可选的环境、操作）匹配，按优先级合并，由 sp-backend 在运行时生效。

通过 CLI 管理三种已支持的策略类型：

```bash
sp policy recording validate -f recording.yaml --json
sp policy recording apply -f recording.yaml --json
sp policy mock apply -f mock.yaml --json
sp policy compare apply -f compare.yaml --json
```

| Kind | 控制内容 | CLI |
|------|----------|-----|
| `RecordingPolicy` | 采样、时间窗口、操作包含/排除、序列化跳过、录制时时间 Mock | `sp policy recording` |
| `MockPolicy` | 跳过/强制 Mock、查找容差、跨应用依赖、无 Mock 时的回退 | `sp policy mock` |
| `CompareRulePolicy` | 忽略路径、解压、转换、数组匹配、CEL 校验 | `sp policy compare` |

JSON Schema 位于后端模块 `sp-policy-rules`（`src/main/resources/schema/`）。在 YAML 顶部添加 `# yaml-language-server: $schema=...` 可在编辑器中获得补全与校验。

## 文档通用结构

每种策略的外层结构相同：

```yaml
apiVersion: softprobe.ai/v1
kind: RecordingPolicy   # 或 MockPolicy / CompareRulePolicy
metadata:
  name: my-app-recording
  priority: 100
  description: "可选说明"
  enabled: true         # 仅 RecordingPolicy / MockPolicy
selector:
  appIds: [my-app-id]
  envTags:
    env: [prod]
spec:
  # 各 kind 专有字段
```

### 通用字段 {#common-fields}

| 字段 | 类型 | 必填 | 说明 |
|------|------|------|------|
| `apiVersion` | string | 是 | 必须为 `softprobe.ai/v1` |
| `kind` | string | 是 | 上表三种之一 |
| `metadata.name` | string | 是 | 唯一标识（小写连字符） |
| `metadata.priority` | integer | 否 | 默认 `0`；数值越大越晚合并，冲突时优先 |
| `metadata.description` | string | 否 | 自由文本 |
| `metadata.enabled` | boolean | 否 | 仅录制/Mock；默认 `true` |
| `selector.matchAll` | boolean | 否 | 匹配所有应用（系统默认在 priority 0 使用） |
| `selector.appIds` | string\[] | 三选一\* | 精确应用 ID |
| `selector.appIdPattern` | string | 三选一\* | Glob，如 `order-*` |
| `selector.excludeAppIds` | string\[] | 否 | 从匹配结果中排除 |
| `selector.envTags` | map | 否 | 标签键 → 允许值列表，如 `env: [prod, staging]`。与 Agent 上报标签匹配（`-Dsp.mocker.tags=env=prod`）。**合取**：声明的每个键都必须满足；省略 = 所有环境 |
| `selector.operationNames` | string\[] | 否 | 精确操作名。**仅录制与 Mock** — CompareRulePolicy 的 selector 不允许 |
| `selector.operationNamePatterns` | string\[] | 否 | Glob，如 `/api/order/*`。**仅录制与 Mock** |

\*`selector` 至少包含 `matchAll`、`appIds`、`appIdPattern` 之一。

**选择器维度**为合取：应用匹配 AND（可选）环境匹配 AND（可选）操作匹配。未约束操作维度时表示「匹配应用下的所有操作」。

### 合并规则

多条策略匹配同一应用时，按 **priority 升序**合并（数值大的后应用）：

| Kind | 合并行为 |
|------|----------|
| **Compare** | 标量覆盖；列表/映射合并；同键后者覆盖 |
| **Recording** | 标量覆盖；列表合并；`timeMock` 任一为 true 则 true；`serializeSkip` 按 `className` + 字段名合并 |
| **Mock** | 标量覆盖；`skipMock`/`forceMock` 合并；`matchTolerance`、`multiServiceDependencies` 按 pattern/应用键后者覆盖 |

sp-backend 内置 priority 0 的 `default-global-*-policy.yaml`；用户策略请设置 `priority > 0` 覆盖。

### 运行时流水线

```text
YAML → 解析 → 校验 → Mongo → 解析合并 → 编译 → PolicyCache（fresh 60s，stale 24h）
```

Mongo 加载超时时安全降级（录制：不录；Mock：透传）。

### 编写建议

1. 在编辑器中启用 JSON Schema 做行内校验。
2. 应用级策略避免滥用 `matchAll: true` — 系统默认已提供兜底。
3. YAML 注释写**原因**；字段含义以 schema 为准。

***

## RecordingPolicy {#recordingpolicy}

录制策略。

控制 **Agent 录什么**：采样、时段、操作过滤、脱敏元数据、序列化跳过、录制时时间 Mock。

### `spec` 字段

| 区块 | 字段 | 类型 | 语义 |
|------|------|------|------|
| `sampling` | `ratePerHundredSeconds` | integer ≥ 0 | 每 100 秒窗口最多录制请求数；`0` = 不录 |
| `sampling` | `machineCountLimit` | integer ≥ 1 或省略 | 同环境组内同时录制的实例上限。**省略 = 不限**。慎用 `1`（易导致占坑） |
| `timeWindow` | `daysOfWeek` | `MON`…`SUN` 或 `*` | 省略整段 = 7×24 |
| `timeWindow` | `from` / `to` | `HH:mm` | 必须成对；`from` 严格早于 `to`。Agent 使用 JVM 本地时区 |
| `operations` | `exclude` | string\[] | 不录制的操作 Glob（黑名单） |
| `operations` | `include` | string\[] | 非空时为**白名单**：只录这些；`exclude` 仍在 include 之后生效 |
| `sensitiveData` | `headers` | string\[] | 头名（见下方说明） |
| `sensitiveData` | `bodyPaths` | string\[] | JSONPath，必须以 `$` 开头 |
| `sensitiveData` | `queryParams` | string\[] | 查询参数名 |
| `sensitiveData` | `placeholder` | string | 替换占位符，默认 `***` |
| `serializeSkip` | `className` | string | Java 类名 |
| `serializeSkip` | `fieldNames` | string\[] | 跳过的字段 |
| | `timeMock` | boolean | 录制时 Mock `java.time.*` |
| | `extras` | map | 原样下发到 Agent `extendField` |

::: warning 录制路径上的 `sensitiveData`
后端会把 `sensitiveData` 写入 Agent 线协议，但 **Java Agent 尚未在录制时执行脱敏**。回放 Mock 键噪声请用 `MockPolicy.spec.matchTolerance`；查看/查询时脱敏请用 `SensitivePolicy`（REST，暂无 `sp policy` CLI）。
:::

修改录制策略的包含/排除列表会影响调度服务构建的回放范围，无需单独改调度文档。

### 完整示例

```yaml
apiVersion: softprobe.ai/v1
kind: RecordingPolicy
metadata:
  name: order-service-prod
  description: "生产录制 — 50 次/100s，工作时间，脱敏元数据"
  priority: 100
  enabled: true
selector:
  appIds: [order-service]
  envTags:
    env: [prod]
  operationNamePatterns:
    - "/api/order/*"
spec:
  sampling:
    ratePerHundredSeconds: 50
    machineCountLimit: 3
  timeWindow:
    daysOfWeek: [MON, TUE, WED, THU, FRI]
    from: "09:00"
    to: "18:00"
  operations:
    exclude:
      - "/health"
      - "/metrics/**"
      - "/actuator/**"
  sensitiveData:
    headers: [authorization, cookie, x-api-key]
    bodyPaths:
      - "$.password"
      - "$.user.phone"
    queryParams: [token, sessionId]
    placeholder: "***"
  serializeSkip:
    - className: com.example.order.domain.Order
      fieldNames: [internalNote]
  timeMock: false
  extras: {}
```

### 仅白名单录制示例

```yaml
apiVersion: softprobe.ai/v1
kind: RecordingPolicy
metadata:
  name: order-service-whitelist
  priority: 110
selector:
  appIds: [order-service]
spec:
  sampling:
    ratePerHundredSeconds: 100
  operations:
    include:
      - "/api/order/**"
    exclude:
      - "/api/order/internal/**"
```

***

## MockPolicy {#mockpolicy}

Mock 策略。

控制**回放时**依赖 Mock：哪些调用用录制数据、哪些走真实下游、Mock 键容差、跨应用依赖、无匹配 Mock 时的行为。

### `spec` 字段

| 区块 | 字段 | 类型 | 语义 |
|------|------|------|------|
| | `mockByDefault` | boolean | 默认 `true`：除 `skipMock` 外全部 Mock；`false`：除 `forceMock` 外走真实下游 |
| `operations` | `skipMock` | string\[] | `Category:operationGlob` — 不 Mock，调真实下游 |
| `operations` | `forceMock` | string\[] | `Category:operationGlob` — 必须 Mock；优先于 `skipMock` |
| `matchTolerance` | `operationPattern` | string | 入口操作 Glob |
| `matchTolerance` | `ignoreHeaders` | string\[] | Mock 指纹中忽略的请求头 |
| `matchTolerance` | `ignoreQueryParams` | string\[] | 忽略的查询参数 |
| `matchTolerance` | `ignoreBodyPaths` | string\[] | 忽略的 body 路径 |
| `multiServiceDependencies` | `downstreamApp` | string | 下游应用 ID |
| `multiServiceDependencies` | `operations` | string\[] | 精确下游操作 |
| `multiServiceDependencies` | `operationPatterns` | string\[] | Glob（`operations` 与 `operationPatterns` 至少填一项） |
| `fallback` | `strategy` | enum | `FAIL`（默认）、`PASS_THROUGH`、`RETURN_DEFAULT` |
| `fallback` | `defaultResponse` | object | `RETURN_DEFAULT` 时必填：`statusCode`、`contentType`、`body`、`headers` |

### `skipMock` / `forceMock` 的依赖分类 {#mock-categories}

必须使用 **`Category:operationGlob`** — 无前缀的裸 Glob 会被**拒绝**。

| Category | 典型 `operationGlob` |
|----------|----------------------|
| `HttpClient` | `/payment/charge`、`/internal/**` |
| `Database` | `select_*`、`*` |
| `Redis` | `GET user:*` |
| `DubboConsumer` | `com.foo.Service#method` |
| `SofaConsumer` | RPC 操作名 |
| `DubboStreamProvider` | 流式操作 |
| `QMessageProducer` | 主题或消息 ID |
| `ConfigFile` | 配置键 |
| `UserDynamic` | `com.foo.Cache.get`（用户配置的动态类） |
| `DynamicClass` | `SystemTime.**`、`RandomSource.**（内置；全局默认强制 Mock，用户 `skipMock`无效） |
|`Encryption`|`com.foo.Crypto.encrypt`|
|`IbmMQ`| MQ 目的地 |
|`SoapClient`| SOAP 操作 |
|`DSR\` | DSR 操作 |

操作段 Glob：`*`（单段）、`**`（多段）、`?`（单字符）。

::: warning 分类键是依赖类型
不要在 Mock 规则中使用 `Servlet`、`DubboProvider` 等**入口**类型 — 仅上表**依赖**分类有效。
:::

### 完整示例

```yaml
apiVersion: softprobe.ai/v1
kind: MockPolicy
metadata:
  name: order-service-replay
  description: "order-service 回放默认"
  priority: 100
  enabled: true
selector:
  appIds: [order-service]
  envTags:
    env: [staging, prod]
spec:
  mockByDefault: true
  operations:
    skipMock:
      - "HttpClient:/health"
      - "HttpClient:/internal/admin/**"
    forceMock:
      - "HttpClient:/payment/charge"
      - "Database:*"
  matchTolerance:
    - operationPattern: "/api/order/**"
      ignoreHeaders: [x-request-id, x-trace-id, x-b3-*]
      ignoreQueryParams: [_t, timestamp]
  multiServiceDependencies:
    - downstreamApp: payment-service
      operations: ["/charge", "/refund"]
    - downstreamApp: inventory-service
      operationPatterns: ["/reserve", "/release"]
  fallback:
    strategy: FAIL
```

### 透传回退示例

```yaml
apiVersion: softprobe.ai/v1
kind: MockPolicy
metadata:
  name: order-service-isolated-replay
  priority: 100
selector:
  appIds: [order-service]
spec:
  fallback:
    strategy: PASS_THROUGH
```

### `RETURN_DEFAULT` 回退示例

```yaml
apiVersion: softprobe.ai/v1
kind: MockPolicy
metadata:
  name: order-service-default-mock
  priority: 100
selector:
  appIds: [order-service]
spec:
  operations:
    forceMock:
      - "HttpClient:/legacy/ping"
  fallback:
    strategy: RETURN_DEFAULT
    defaultResponse:
      statusCode: 200
      contentType: application/json
      body: '{"status":"ok"}'
      headers:
        x-mock: "true"
```

***

## CompareRulePolicy {#comparerulepolicy}

对比策略。

控制回放**差异对比**中的噪声：忽略路径、解压、归一化、数组匹配、对比后 CEL 过滤。

::: warning 操作范围
**不要**在 `selector` 上写 `operationNames` 或 `operationNamePatterns` — 保存时会被拒绝。按操作规则请用 `spec.operationSpecs[]`。
:::

### `spec` 字段

| 区块 | 字段 | 类型 | 语义 |
|------|------|------|------|
| `defaults` | `timeToleranceMs` | integer ≥ 0 | CEL 中 `time_tolerance_ms` 及时间类规则 |
| `defaults` | `ignoreHeaderPatterns` | string\[] | 对比时跳过的头名 Glob |
| | `includePaths` | string\[] | JSON Pointer 白名单；空 = 对比全部 |
| | `excludePaths` | string\[] | JSON Pointer 黑名单（预过滤） |
| `decompress[]` | `path` | string | JSON Pointer 或 Glob |
| `decompress[]` | `codec` | enum | `PLAIN_JSON`、`BASE64_JSON`、`GZIP_BASE64_JSON` |
| `transforms[]` | `path` | string | JSON Pointer |
| `transforms[]` | `expression` | string | 对比前归一化的 CEL |
| `arrays[]` | `path` | string | 数组字段路径 |
| `arrays[]` | `strategy` | enum | `BY_INDEX`（默认）或 `BY_KEY` |
| `arrays[]` | `keys` | string\[] | `BY_KEY` 时必填 |
| `arrays[]` | `references[]` | object | `field`、`target`、`targetKey` |
| `validations[]` | `id` | string | 规则唯一 ID |
| `validations[]` | `name` | string | 显示名 |
| `validations[]` | `expression` | string | CEL，结果为 bool |
| `validations[]` | `action` | enum | 仅 `DROP`（丢弃该差异） |
| `validations[]` | `enabled` | boolean | 默认 `true` |
| `validations[]` | `message` | string | 说明 |
| `operationSpecs[]` | `operationNames` / `operationNamePatterns` | string\[] | 覆盖的入口操作 |
| `operationSpecs[]` | `spec` | object | 与顶层 spec 相同的叶子字段（不可嵌套 `operationSpecs`） |

路径为 **JSON Pointer**（`/foo/bar`）。Glob：`*` 单段，`**` 任意深度。

### CEL 变量与函数

用于 `validations` 与 `transforms`：

| 名称 | 类型 | 含义 |
|------|------|------|
| `left` | string | 录制侧值 |
| `right` | string | 回放侧值 |
| `path` | string | JSON Pointer |
| `pointer` | string | 同 `path` |
| `fieldName` | string | 叶子字段名 |
| `category` | string | 依赖分类，如 `DATABASE` |
| `time_tolerance_ms` | int | 来自 `defaults.timeToleranceMs` |
| `isTimestamp(s)` | 函数 | 是否为可识别时间戳 |
| `toTimestamp(s)` | 函数 | 转为 epoch 毫秒 |
| `matches` | 方法 | 字符串正则 |

按分类忽略差异（如整类 `DATABASE` 的 `body`）通过 **CEL `validations`** 实现，无单独「按分类忽略」字段。

### 完整示例（应用默认 + 按操作覆盖）

```yaml
apiVersion: softprobe.ai/v1
kind: CompareRulePolicy
metadata:
  name: order-service-compare
  description: "order-service 对比规则"
  priority: 100
selector:
  appIds: [order-service]
spec:
  defaults:
    timeToleranceMs: 60000
    ignoreHeaderPatterns: [x-request-id, x-b3-*]
  excludePaths:
    - "/response/headers/date"
    - "/response/body/metadata/generatedAt"
  decompress:
    - path: "/response/body/payload"
      codec: GZIP_BASE64_JSON
  arrays:
    - path: "/response/body/items"
      strategy: BY_KEY
      keys: [skuId]
  validations:
    - id: ignore-uuids
      name: 忽略 UUID 对
      expression: >-
        left.matches("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
        && right.matches("^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")
      action: DROP
      enabled: true
      message: 两侧均为 UUID
    - id: ignore-db-body
      name: DATABASE 类忽略 body
      expression: 'category == "DATABASE" && fieldName == "body"'
      action: DROP
      enabled: true
  operationSpecs:
    - operationNamePatterns: ["/api/order/export"]
      spec:
        excludePaths:
          - "/response/body/exportJobId"
        validations:
          - id: export-timestamp-tolerance
            expression: >-
              isTimestamp(left) && isTimestamp(right)
              && math.abs(toTimestamp(left) - toTimestamp(right)) <= time_tolerance_ms
            action: DROP
            enabled: true
```

***

## 相关配置 {#related-configuration}

### 动态类

**不属于** `RecordingPolicy`。在**动态类配置**（控制台或存储 API）中登记方法。回放时分类为 `UserDynamic` 或内置 `DynamicClass`（如 `SystemTime.*`）。用 **MockPolicy** 的 `forceMock` / `skipMock` 控制行为。

回放匹配顺序：先按请求参数**精确**匹配，再无命中时按方法签名**模糊**匹配。

### SensitivePolicy（敏感数据策略）

第四种策略，用于**查询/查看时**脱敏（REST API；暂无 `sp policy` CLI）。与 `RecordingPolicy.spec.sensitiveData` 分离。

| 字段 | 说明 |
|------|------|
| `spec.fieldNameRules[]` | `pattern`（Java 正则）、`type`：`NAME`、`ID_CARD`、`PHONE`、`EMAIL`、`PASSPORT`、`DEFAULT`、`NONE` |
| `spec.contentRules[]` | 同上，匹配字段**值** |

```yaml
apiVersion: softprobe.ai/v1
kind: SensitivePolicy
metadata:
  name: order-service-sensitive
  priority: 100
selector:
  appIds: [order-service]
spec:
  fieldNameRules:
    - pattern: "(?i)^password$"
      type: NAME
  contentRules:
    - pattern: "^1[3-9]\\d{9}$"
      type: PHONE
```

## 相关文档

* [策略概览](/zh/testing/policies)
* [回放与对比](/zh/testing/replay-and-diff)
* [CLI：policy 命令](/zh/testing/commands/policy)

---

---
url: /zh/platform/billing/subscriptions.md
---
# 计费与订阅

本文说明订阅的常见状态、计费周期与用量、取消与恢复，以及套餐变更的影响。

## 订阅生命周期（常见状态）

* ACTIVE：订阅激活，按周期或用量计费
* SUSPENDED：订阅暂停，服务受限或停止
* CANCELLED：订阅取消，在当前周期结束后停止服务（或立即结束，取决于设置）

典型变化：支付失败触发暂停；用户主动取消；风控触发降级或停服。

## 计费周期与用量

* 固定套餐：固定月费，不随周期内用量变化
* 按用量套餐：按月聚合用量并在周期末结算
* 可在仪表盘查看用量与账单记录

## 取消与恢复

* 取消：支持“立即取消”或“在周期末取消”
* 恢复：清缴欠费或完成支付后恢复 ACTIVE
* 数据保留：暂停或取消期间通常保留，宽限期后可能清理（详见条款）

## 套餐变更（升级/降级/切换计费模型）

* Free → Business：支持；可立即生效或在下个周期生效
* Business → Enterprise：支持；可设置用量上限与提醒
* 降级策略：从较高配额降级后，超配数据可能受限或需迁移
* 按比例计费：周期中途变更可能触发比例结算

## 风控与消费限额

* 余额阈值提醒（如￥10 提醒、￥5 紧急）
* 降级策略：RESTRICTED（严格限流）、SUSPENDED（停服）
* 日/月消费上限与自动停服保护，避免异常成本

## Stripe 集成与价格映射

* 每个套餐都映射到 Stripe 的产品与价格（固定与按量计费）
* 订阅将基于这些映射进行计费
* 价格变更将在下个计费周期生效，或根据设置按比例结算

## 常见操作

* 查看当前订阅与计费周期：仪表盘 → Subscription 面板
* 变更套餐：仪表盘 → Subscription 面板（Pro/Business、Pay‑as‑you‑go）
* 查看用量与成本：仪表盘 → Usage & Cost
* 取消或暂停：仪表盘 → Subscription 面板（支持在周期末取消）

## 支持

* 企业定制：support@softprobe.ai
* 技术问题：参见 API 文档与常见问题（Support）

---

---
url: /zh/platform/getting-started/account-setup.md
---
# 账户设置

设置您的 Softprobe 账户并为 SP-Istio Agent 生成 API 密钥。

## 📋 概述

本指南将引导您完成：

* 创建 Softprobe 账户
* 设置您的租户组
* 生成 API 密钥
* 下载配置文件

## 🚀 入门

### 步骤 1：创建您的账户

1. 访问 [Softprobe Dashboard](https://dashboard.softprobe.ai)
2. 点击 **"注册"** 创建新账户
3. 填写您的详细信息：
   * **电子邮件地址** (将作为您的登录用户名)
   * **密码** (最少 8 个字符)
4. 点击发送到您收件箱的链接验证您的电子邮件地址

### 步骤 2：访问设置并创建租户组

电子邮件验证并登录后，您可以访问“设置”页面管理租户组：

1. 导航到 **[设置](https://dashboard.softprobe.ai/settings)** 页面
2. **创建您的第一个租户组**
   * 点击“创建组”按钮
   * **租户组名称**：为您的组织选择一个唯一的名称
     * 这将用于组织您的服务和数据
     * 示例：`my-company-prod`, `acme-corp`, `team-alpha`
   * **描述** (可选)：添加您的组的简要描述
3. 点击 **"创建租户组"**

::: tip
请仔细选择您的租户组名称，因为它以后无法更改。使用一个能清晰识别您的组织或团队的名称。
:::

### 步骤 3：生成 API 密钥

创建租户组后：

1. 导航到仪表板侧边栏中的 **"API 密钥"**
2. 点击 **"生成新 API 密钥"**
3. 提供以下信息：
   * **密钥名称**：一个描述性名称（例如，`production-cluster`, `dev-environment`）
4. 点击 **"生成密钥"**

::: warning 重要

* 您的 API 密钥将**只显示一次**
* 立即复制并安全存储
* `minimal.yaml` 配置文件将自动下载
* 关闭对话框后，您无法再次检索密钥
  :::

## 📁 配置文件

当您生成 API 密钥时，`minimal.yaml` 文件会自动下载。此文件包含：

* 您的个性化 API 密钥
* 预配置的端点
* 默认采集规则
* 所有必要的 Kubernetes 资源

### 文件结构

下载的 `minimal.yaml` 包含：

```yaml
# WasmPlugin configuration with your Public key
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: sp-istio-agent
spec:
  pluginConfig:
    public_key: "your-generated-api-key"
    # ... other configurations
```

## 🔐 安全最佳实践

### API 密钥管理

* **安全存储**: 将 API 密钥保存在安全的凭证管理系统中
* **定期轮换**: 定期生成新密钥并停用旧密钥
* **使用描述性名称**: 根据其用途和环境命名密钥
* **监控使用情况**: 检查仪表板中的 API 密钥活动

这有助于更好地跟踪和安全隔离。

## 🔧 下一步

完成账户设置后：

1. **快速测试**: 遵循 [快速入门](../getting-started/quick-start)
2. **生产环境**: 遵循 [生产安装](../deployment/installation)
3. **自定义配置**: 查看 [配置指南](../configuration/config)

## ❓ 故障排除

### 常见问题

**无法访问仪表板？**

* 检查您的互联网连接
* 验证 URL：`https://dashboard.softprobe.ai`
* 尝试清除浏览器缓存

**未收到电子邮件验证？**

* 检查您的垃圾邮件/垃圾箱文件夹
* 确保电子邮件地址正确
* 如果问题仍然存在，请联系支持

**API 密钥生成失败？**

* 确保您的租户组已正确设置
* 检查您是否具有必要的权限
* 尝试刷新页面并重新生成

### 获取帮助

如果在账户设置过程中遇到问题：

* **文档**: 查看我们的 [故障排除](#-故障排除)
* **支持**: 通过仪表板联系我们的支持团队
* **社区**: 加入我们的社区讨论以获取同行帮助

***

## 常见问题

### Q1: 如果我丢失了 minimal.yaml 配置文件怎么办？

**A:** `minimal.yaml` 文件只在您创建 API 密钥时下载一次。如果您丢失了它：

1. 从仪表板中删除当前的 API 密钥
2. 创建新的 API 密钥（这将生成新的 `minimal.yaml` 文件）
3. 立即安全保存新的配置文件

### Q2: 如果我忘记保存我的 API 密钥怎么办？

**A:** 使用新系统，您无需手动复制 API 密钥。它会自动嵌入到您创建密钥时下载的 `minimal.yaml` 文件中。只需确保安全保存该文件即可。

### Q3: API 密钥可以重复使用吗？

**A:** 是的，API 密钥可以在多个环境中使用，但建议：

* 为生产环境使用单独的 API 密钥
* 为开发/测试环境使用单独的 API 密钥
* 定期轮换 API 密钥以提高安全性

### Q4: 如何管理多个项目？

**A:** 建议为不同的项目创建不同的租户组：

1. 为每个项目创建单独的租户组
2. 为每个项目生成单独的 API 密钥
3. 邀请相关团队成员加入相应的租户组

### Q5: API 密钥有使用限制吗？

**A:** API 密钥的使用限制包括：

* 请求速率限制
* 数据存储配额
* 功能权限限制
* 请查看您的订阅计划以了解具体限制

### Q6: 如果配置后没有数据怎么办？

**A:** 请检查：

1. `minimal.yaml` 文件是否已使用 `kubectl apply -f minimal.yaml` 正确应用
2. Istio 服务网格是否已在您的集群中正确安装
3. 网络连接是否正常
4. 应用 Pod 是否在应用配置后已重启
5. 应用日志中是否有任何错误消息

***

## 技术支持

如果您在使用过程中遇到任何问题，可以通过以下方式获得帮助：

### 📧 联系支持

* **电子邮件**: support@softprobe.ai
* **响应时间**: 工作日 24 小时内

### 📚 文档资源

* **API 文档**: <https://docs.softprobe.ai>
* **开发者指南**: <https://developers.softprobe.ai>
* **常见问题**: <https://softprobe.ai/faq>

### 🐛 问题反馈

如果您发现错误或有功能建议：

1. 登录到仪表板
2. 点击右上角的反馈按钮
3. 详细描述问题或建议
4. 我们将及时跟进

***

## 🎉 开始使用

恭喜！您已完成 Softprobe 的基本配置。现在您可以：

1. **监控应用程序性能**: 实时查看应用程序的运行状态
2. **分析用户行为**: 通过热力图了解用户习惯
3. **优化用户体验**: 根据数据分析优化您的产品
4. **团队协作**: 邀请团队成员一起分析数据

开始探索 Softprobe 的强大功能，让数据驱动您的产品决策！

***

*最后更新: 2024 年 1 月*
*版本: v1.0*

---

---
url: /zh/testing/compare-rules-web-ui.md
---

# 配置对比规则

对比规则决定哪些回放差异**不算数**。原则：*配了的就跳过，没配的严格对比。* [审查差异](/zh/testing/review-diffs-in-the-web-ui) 里逐条接受只管一次；把总在变的字段（时间戳、随机 ID）沉淀成规则，以后每次回放都不再误报——到这一步整个流程就闭环了。

本页讲在**网页控制台的可视化编辑器**里配规则——选一种规则、填一两个字段、点添加。日常用可视化编辑器最友好；同样的规则也能声明式地写进 [策略 YAML 指南](/zh/testing/policy-yaml-guide)，用于 GitOps 与 CI。

## 在哪里配置规则

有两个位置，对应两种作用范围：

| | 全局默认规则 | 应用规则 |
|---|---|---|
| 从哪打开 | 设置 → 对比规则 | 工作台 → 配置 → 对比规则 |
| 作用于 | 每个应用 | 仅当前应用 |
| 规则类型 | 全部类型，分六个 tab | 三种常用类型加接口专属 overlay；其余走 YAML |

应用面板顶部会链接到全局默认，所以你随时看得到还有什么在生效。

![应用对比规则面板](/img/docs/replay/compare-rules-panel.png)

全局页把规则类型分成六个 tab——每种一个，对应下面各节。

![六个规则 tab](/img/docs/replay/rules-global-tabs.png)

## 按路径忽略一个字段

**最常用的规则——不再对比某个字段。** 适合易变字段：时间戳、traceId、随机 token。这个字段及其下面的一切都会从对比里剔除。

**怎么加：** 打开 **按路径忽略** tab，在 **忽略字段** 里填字段路径，点 **添加**。

![路径 tab](/img/docs/replay/rule-exclude-path.png)

**填什么：** 一个字段路径。`data.traceId`（点号形态）和 `/data/traceId`（JSON Pointer）都行。用 `*` 匹配一层、`**` 匹配任意深度，如 `/data/*/updatedAt`。

::: tip 白名单输入框（极少用）
同一个 tab 上面有一个 **包含路径（白名单）** 输入框。往这里加东西后，**只**对比这些路径、其余全忽略——和忽略列表相反。留空（常态）=对比一切。
:::

## 整类忽略某种依赖

**一个粗开关——丢掉某种下游调用的全部差异。** 例如忽略所有 Redis 差异，或某个数据库查询的全部差异。

**怎么加：** 打开 **依赖类型** tab。填 **类型**（如 `Redis`、`Database`、`Dubbo`、`HttpClient`），可选再填某个具体依赖 **名字**。名字留空=忽略整类。

![依赖类型 tab](/img/docs/replay/rule-ignore-category.png)

在回放里，被这样忽略的调用显示一个 **「本类已整类忽略」** chip，而不是逐字段删除线——整个调用被一次性挡下了。

## 按条件忽略（CEL）

**最灵活的规则——条件成立时忽略一处差异。** 当按路径或字段名匹配不够用时用它，例如「忽略任何录制值和回放值都是时间戳的字段」。对比跑完后，每一处差异都用你的条件检验；命中就丢弃。

**怎么加：** 打开 **按条件忽略（CEL）** tab，点 **添加规则**，可选起个名字，写条件。模板选择提供现成条件。

![CEL 规则 tab，含示例规则](/img/docs/replay/rule-cel.png)

可用变量：`left` / `right`（录制值 / 回放值）、`path` / `pointer`（字段路径）、`fieldName`、`category`、`time_tolerance_ms`。辅助函数：`isUUID`、`isIP`、`isTimestamp`、`toTimestamp`、`toNumber`。

示例：

* 两侧都是时间戳：`isTimestamp(left) && isTimestamp(right)`
* 一个生成的 ID：`fieldName == "requestId" && isUUID(right)`

## 对比前归一化一个值

**四舍五入或重塑一个值，让噪声不登记**——例如给浮点四舍五入，让精度差异不算差异。

**怎么加：** 打开 **值转换** tab，填字段 **路径** 和一个作用于值的 CEL **表达式**（`value` 是该字段原始值），点 **添加转换规则**。

![值转换 tab](/img/docs/replay/rule-transform.png)

示例——四舍五入到两位小数：路径 `/data/orders/*/total`，表达式 `math.round(value * 100) / 100`。

## 对比前解码一个编码字段

**解码 base64/gzip 的 JSON，让对比看到真实数据**，而不是报「这两个编码串不一样」。

**怎么加：** 打开 **解压配置** tab，填字段 **路径**，选 **编码格式**（`Base64 + JSON`、`Gzip + Base64 + JSON`、`Plain JSON`）。

![解压配置 tab](/img/docs/replay/rule-decompress.png)

## 顺序会变时匹配数组元素

**把无序数组当集合比，而不是按位置比。** 默认数组按索引比——当元素顺序在录制和回放之间会变时，会产生假的「缺失/新增元素」差异。

**怎么加：** 打开 **数组匹配** tab，填数组 **路径**，选 **策略**，（对「按主键」）填 **主键字段**。点 **添加数组配置**。

![数组匹配 tab](/img/docs/replay/rule-arrays.png)

| 策略 | 什么时候用 |
|---|---|
| 按索引 | 默认——逐位置对比。 |
| 按主键 | 按一个主键字段配对元素（填主键，如 `orderId`）。 |
| LCS 算法 | 最长公共子序列——没有主键时的尽力对齐。 |

## 只对特定接口叠加规则

上面的规则对每个接口生效。要只给某些接口加规则，用应用面板的 **接口专属** 区。点 **添加接口规则**，填要匹配的接口——精确名和/或像 `/api/order/*` 的 glob 模式——然后填这个组自己的规则表。

## diff 里的快捷规则去了哪

当你[在回放里忽略一个字段](/zh/testing/review-diffs-in-the-web-ui#ignore-a-field)时，它会写进上面某种规则：

| diff 动作 | 变成 |
|---|---|
| 忽略此字段的差异 | 一条按路径忽略规则 |
| 忽略所有 "…" 字段 | 一条匹配该字段名的 CEL 规则 |
| 设为数组主键 | 一条数组规则（按主键） |

作用于某接口时落进接口专属组，作用于整个应用时落进顶层。

## 下一步

规则配好后，回到 [回放](/zh/testing/replay-and-diff) 再跑一次（或在 [审查差异](/zh/testing/review-diffs-in-the-web-ui) 里用**重新比对**立即验证），失败列表里剩下的就都是值得看的真差异了。

想把整个流程自动化——部署后自动回放、流水线门禁——见 [Webhook 与 CI/CD](/zh/testing/webhook-and-ci)；规则进 GitOps 见 [策略 YAML 指南](/zh/testing/policy-yaml-guide)。

---

---
url: /zh/platform/configuration/config.md
---
# 配置指南

了解如何为您的特定用例配置 SP-Istio Agent。

## 📋 概述

本指南涵盖：

* 理解配置结构
* 设置采集规则
* 配置服务发现
* 高级配置选项

## 🏗️ 配置结构

SP-Istio Agent 配置在 `minimal.yaml` 文件中定义，该文件包含三个主要组件：

### 1. WasmPlugin 资源

将 SP-Istio Agent 加载到 Istio 的 Envoy 代理中的核心组件：

```yaml
apiVersion: extensions.istio.io/v1alpha1
kind: WasmPlugin
metadata:
  name: sp-istio-agent
  namespace: istio-system
spec:
  url: oci://docker.io/softprobe/softprobe:latest
  pluginConfig:
    # 您的配置在此处
```

### 2. EnvoyFilter 资源

配置 Envoy 代理设置以实现最佳性能：

```yaml
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: sp-istio-agent-config
  namespace: istio-system
spec:
  configPatches:
    # Envoy 特定配置
```

### 3. 网络资源

定义网络策略和服务发现设置。

## ⚙️ 核心配置选项

### 基本设置

| 字段             | 类型   | 描述                    | 默认值                   |
| ---------------- | ------ | ----------------------- | ------------------------ |
| `public_key`     | string | 您的 Softprobe API 密钥 | 必填                     |
| `sp_backend_url` | string | Softprobe 采集端点      | `https://o.softprobe.ai` |
| `service_name`   | string | 覆盖服务名称检测        | 自动检测                 |

### 基本配置示例

```yaml
pluginConfig:
  public_key: "您的 API 密钥"
  sp_backend_url: "https://o.softprobe.ai"
  traffic_direction: "server"
  collectionRules:
    http:
      server:
        - path: ".*"
```

## 🎯 采集规则

采集规则定义要捕获和分析的流量。它们通过 `traffic_direction` 设置支持 `SERVER`（入站）和 `CLIENT`（出站）两种模式。

### 规则结构

**对于 SERVER 模式（入站流量）：**

```yaml
pluginConfig:
  traffic_direction: "server"
  collectionRules:
    http:
      server:
        - path: ".*" # URL 路径的正则表达式模式
```

**对于 CLIENT 模式（出站流量）：**

```yaml
pluginConfig:
  traffic_direction: "client"
  collectionRules:
    http:
      client:
        - host: ".*" # 主机名的正则表达式模式
          paths: [".*"] # 路径的正则表达式模式列表
```

### SERVER 模式规则

捕获到您服务的入站请求：

```yaml
pluginConfig:
  traffic_direction: "server"
  collectionRules:
    http:
      server:
        # 捕获所有 API 端点
        - path: "/api/.*"
        # 捕获特定端点
        - path: "/health"
        - path: "/metrics"
        - path: "/api/v1/users"
```

### CLIENT 模式规则

捕获从您服务发出的出站请求：

```yaml
pluginConfig:
  traffic_direction: "client"
  collectionRules:
    http:
      client:
        # 捕获所有到任何主机的出站请求
        - host: ".*"
          paths: [".*"]
        # 捕获到外部服务的特定 API 调用
        - host: ".*\\.googleapis\\.com"
          paths: ["/maps/api/.*", "/drive/v3/files"]
        # 捕获到特定服务的请求
        - host: "my-api\\.com"
          paths: ["/api/.*"]
```

***

## 采集规则说明 (`collectionRules`)

`collectionRules` 是 SP-Istio Agent 中最核心和灵活的配置项。它允许您精确定义要收集哪些 HTTP 流量，从而避免不必要的数据报告，节省成本并提高效率。

### 使用正则表达式

在所有采集规则中，`path`、`host` 和 `paths` 字段的值都是**正则表达式 (Regex)**。这为您提供了强大的匹配能力。

* 要匹配所有内容，请使用 `.*`。
* 对于精确匹配，请直接写入字符串，例如 `"/api/users"`。
* 要匹配特定模式，请使用正则表达式语法，例如 `"/api/v[0-9]+/items"` 可以匹配 `/api/v1/items` 和 `/api/v2/items`。

**注意**：如果提供了无效的正则表达式，系统将自动回退到**精确字符串匹配**。

### SERVER 模式下的采集

当 `traffic_direction` 设置为 `server` 时，Agent 会收集**入站**流量。在这种情况下，您需要配置 `collectionRules.http.server`。

```yaml
# ...
pluginConfig:
  traffic_direction: "server"
  collectionRules:
    http:
      server:
        - path: ".*" # 这是一个采集规则
```

* **`server`**: 包含多个采集规则的数组。
* **`path`**: 用于匹配入站请求的 **URL 路径** 的正则表达式。

**工作原理**：对于进入您服务的每个请求，Agent 会获取其 URL 路径（例如，`/api/users/123`）并将其与 `path` 字段中定义的正则表达式进行匹配。如果匹配成功，则收集该请求及其相应的响应。

**示例**：

```yaml
# 仅收集 /api/v1/ 下的请求
collectionRules:
  http:
    server:
      - path: "/api/v1/.*"

# 收集与用户和订单相关的请求
collectionRules:
  http:
    server:
      - path: "/users/[^/]+$"  # 匹配 /users/some-id
      - path: "/orders/[0-9]+" # 匹配 /orders/12345
```

### CLIENT 模式下的采集

当 `traffic_direction` 设置为 `client` 时，Agent 会收集**出站**流量。在这种情况下，您需要配置 `collectionRules.http.client`。

```yaml
# ...
pluginConfig:
  traffic_direction: "client"
  collectionRules:
    http:
      client:
        - host: ".*\\.external-service\\.com"
          paths: ["/api/data/.*", "/api/auth"]
```

* **`client`**: 包含多个采集规则的数组。
* 每个规则包含 `host` 和 `paths` 字段。
* **`host`**: 用于匹配出站请求的**目标主机名** 的正则表达式。
* **`paths`**: 字符串数组，其中每个字符串都是用于匹配出站请求的**URL 路径** 的正则表达式。

**工作原理**：对于从您服务发出的每个请求，Agent 将：

1. **确定目标主机和路径**：Agent 尝试从 HTTP 头部（例如 `Referer`、`Origin` 或 `Host`）中提取目标主机名和路径。
2. **匹配 `host`**：将提取的目标主机名与 `host` 字段中的正则表达式进行匹配。
3. **匹配 `paths`**：如果主机名匹配，Agent 将继续将提取的 URL 路径与 `paths` 数组中的**任何一个**正则表达式进行匹配。

**只有当 `host` 和 `paths` 中的至少一个路径规则都成功匹配时，才会收集出站请求及其相应的响应。**

**示例**：

```yaml
# 收集所有到 my-api.com 的出站请求，无论路径如何
collectionRules:
  http:
    client:
      - host: "my-api\\.com"
        paths: [".*"]

# 收集到 googleapis.com 的特定 API 调用
collectionRules:
  http:
    client:
      - host: ".*\\.googleapis\\.com"
        paths: ["/maps/api/.*", "/drive/v3/files"]
```

***

## 服务发现：自动识别您的服务

在复杂的微服务环境中，准确识别每个服务至关重要。SP-Istio Agent 通过一个名为 `inject-app-name-header` 的 `EnvoyFilter` 实现这一点。

此 `EnvoyFilter` 在处理每个请求之前执行一小段 Lua 脚本。该脚本的唯一目的是**查找当前服务的名称**并将其添加到特殊的 HTTP 头部 `x-sp-service-name` 中。

**它是如何工作的？**

该脚本按顺序尝试从几个常见的 Kubernetes 环境变量（例如 `OTEL_SERVICE_NAME`、`APP_NAME`、`SERVICE_NAME`）获取服务名称，或者通过从 `POD_NAME` 中解析它。这种多源方法大大提高了自动服务识别的准确性，并且**您通常不需要任何额外的配置**。

***

## 外部通信：连接到 Softprobe 后端

为了使 SP-Istio Agent 能够将数据发送到 `o.softprobe.ai`，我们需要在 Istio 中明确授权此外部通信。这通过两个资源完成：`ServiceEntry` 和 `DestinationRule`。

* **`ServiceEntry`**: 将 `o.softprobe.ai` 添加到 Istio 的服务注册表中，使其成为合法的外部服务。
* **`DestinationRule`**: 为到 `o.softprobe.ai` 的流量配置 TLS 加密，确保安全数据传输。

简而言之，这两个资源共同为 SP-Istio Agent 打开了一个到 Softprobe 后端的安全通道。

我们希望这份详细指南能帮助您更好地理解和使用 SP-Istio Agent。如果您有任何问题，请随时联系我们！