Using the Index testing tool

Before you begin: Complete Build a gRPC RTD service, and implement the mutations you need. See Mutation reference for the schema of all 6, and Implementing bid shading for extra depth if that's one of them. If you're deploying in Index Cloud, also complete Building your container so your container exposes the health check and metrics endpoints this tool checks.

This tool sends test traffic to your container and checks the result against the same rules Index applies in production. This page walks through it in five steps:

1. Get access

2. Know your two required flags

3. Pick your request samples

4. Run the tool

5. Read the results

1. Get access

Send your Docker Hub account email to your Index Representative. Once you have access, log in, then pull the image:

docker login
docker pull indexexchangehub/ext-data-provider-test-tools:2026.07.07

Quick reference: full command template

If you already know your values, copy this and fill in the blanks; every flag included, with what to change marked:

docker run \
  --network="host" \
  -t --rm --name send-rtd-requests \
  -e COMMAND_LINE_FLAGS="\
-e <YOUR_ENDPOINT>              # your gRPC service, as host:port \
-base-url <YOUR_METRICS_URL>    # REQUIRED -- HTTP URL for health + metrics checks \
-m grpc                         # required -- omitting this defaults to legacy HTTP mode \
-lifecycle <LIFECYCLE_NAME>     # required if you're not using -f \
-c 100                          # concurrent workers (default) \
-q 1000                         # total requests/sec across all workers (default) \
-d 0                            # test duration in seconds; 0 = run until you Ctrl+C (see CI note below) \
-s 5                            # latency SLA in ms; also used as gRPC tmax (default) \
-err 50                         # max allowed failure rate % (default) \
-t 30                           # per-request timeout in ms (default) \
" \
  indexexchangehub/ext-data-provider-test-tools:2026.07.07

(The line breaks and comments above are for readability only; flatten it to one line, without comments, before running.)

About --network="host": every example command includes it so the test container can reach services running on your own machine (via host.docker.internal or localhost); that's what you need if you're testing a container running locally on the same machine. If your -e and -base-url instead point at a remote or already-deployed endpoint A URL which is configured to interact with a server in a specific way. (not on this machine), you can drop --network="host" entirely.

2. Know your two required flags

Every run needs -e and -base-url. Everything else has a default. These two trip people up because they're not the same URL; here's why:

Why two flags instead of one: your service speaks gRPC on one port, but it still exposes plain HTTP metrics (SERVER_PORT) on a separate port. -e points at the gRPC port; -base-url points at the HTTP metrics port.

If you forget -base-url, the tool exits immediately, before sending a single request.

Example command

docker run \
  --network="host" \
  -t --rm --name send-rtd-requests \
  -e COMMAND_LINE_FLAGS="-e host.docker.internal:50051 -m grpc -q 75000 -c 20 -base-url http://host.docker.internal:8080 -lifecycle LIFECYCLE_PUBLISHER_BID_REQUEST -s 50" \
  indexexchangehub/ext-data-provider-test-tools:2026.07.07

• -e host.docker.internal:50051 → your gRPC service

• -base-url http://host.docker.internal:8080 → your separate metrics port

All flags

Always pass -m grpc explicitly. If -m is omitted, it defaults to a legacy HTTP mode that is no longer documented or supported here. Always include -m grpc in your command to make sure you're not accidentally running in that mode.

3. Pick your request samples

The tool needs a stream of sample requests to fire at your container. Decide this before you run; the flow depends only on -f and -lifecycle:

Did you pass -f?
  Yes -> that file is used. Done.
  No -> you MUST pass -lifecycle. Only two values are supported:
        - LIFECYCLE_PUBLISHER_BID_REQUEST
        - LIFECYCLE_DSP_BID_RESPONSE
    Anything else (or leaving both -f and -lifecycle unset)
    makes the tool exit immediately with an error.

If you want to test with your own traffic shapes, use -f with a file of one protojson RTBRequest per line. Field names below are camelCase, matching what your container actually receives in production (protojson also accepts the underlying snake_case proto field names if that's easier to generate):

{"lifecycle":"LIFECYCLE_PUBLISHER_BID_REQUEST","id":"2b8ddfe9-26d4-4568-b8c2-ea4c24894ae5","tmax":5,"bidRequest":{"id":"1ef0d748-e1f5-4b0e-81b0-71338678a772","imp":[{"id":"AdThrive_Footer_1_phone","banner":{"format":[{"w":300,"h":50}],"pos":1}}],"site":{"domain":"example.com"}},"originator":{"type":"TYPE_PUBLISHER"},"applicableIntents":["ACTIVATE_DEALS"]}

4. Run the tool

With your flags decided, run the docker run command from step 2. Before it sends any load, the tool checks your container is reachable and healthy; if this fails, it exits immediately and sends zero traffic:

Once that passes, it cycles through your sample requests across -c workers at -q QPS Queries Per Second (QPS). The number of bid requests a DSP processes per second. Also known as impressions per second. for -d seconds; sending each one as a GetMutations call.

5. Read the results

The tool prints a stats snapshot every 10 seconds while it runs, then a final summary at the end. Everything comes out as JSON log lines. Here's roughly what you'll see (values shortened for readability; your keys and structure will match, exact numbers won't):

Every 10 seconds, a "Stats" line:

{"level":"info","msg":"Stats",
  "Total Requests": 74981,
  "Latency": {
    "50th percentile timing (microseconds)": 2210,
    "95th percentile timing (microseconds)": 6320,
    "99th percentile timing (microseconds)": 11450,
    "99.9th percentile timing (microseconds)": 24870
  },
  "Results": {
    "success": 74812,
    "validation_error": 140,
    "DeadlineExceeded": 29,
    "Total Mutations": 52104,
    "Non-Empty Responses": 48310,
    "Intent": {"Distribution": {"ACTIVATE_DEALS": 31200, "ADJUST_DEAL_FLOOR": 20904}},
    "Validation Errors": {"intent_not_applicable:ADD_METRICS": 98, "request_response_id_mismatch": 42}
  }
}

Once -d seconds have elapsed, one "result summary: " line:

{"level":"info","msg":"result summary: ",
  "total requests": 449886,
  "sla used in testing": 0.05,
  "tolerated error threshold": 50,
  "total errors": 612,
  "total requests over sla": 890,
  "percent errors": 0.14,
  "percent over sla": 0.2,
  "non-empty responses": 289774,
  "percent non-empty": 64.4,
  "total mutations": 312890,
  "avg mutations/request": 0.7,
  "total validation errors": 140
}

Immediately after that, the line you actually want to grep for:

{"level":"info","msg":"test passed"}

or

{"level":"info","msg":"test failed"}

That single line is the tool's own verdict; everything above it is the evidence for how it got there.

These numbers are directional, not a production forecast. The test environment isn't the same as production, so don't expect your latency and error-rate numbers here to match what you'll see once live; use this to catch regressions and confirm you're in the right ballpark against your SLA, not as a precise prediction of production performance.

Tool-side outcomes (a problem with the generated sample payload, not your container):

• unmarshal_error / marshal_error: a sample payload couldn't be parsed or built

Container-side outcome:

• validation_error: your container responded, but the response itself failed a check. This does indicate a problem with your container's response; see Validation Errors below for the specific issue to fix.

Validation error reference:

Pass or fail

The run's overall pass/fail is decided by exactly two numbers, measured across the whole run:

1. % of requests that errored

2. % of requests that exceeded the -s SLA threshold

If either exceeds -err (default 50%), the run fails; even if the 10-second snapshots looked healthy along the way. The run also fails, independent of those numbers, if the post-run /metrics re-check doesn't find the expected rpc_server_duration histogram. That re-check exists to confirm you're emitting the same metric Index scrapes from you in production; not just answering requests correctly.

Exit codes, if you're wiring this into your own CI

Important if you plan to automate this: exit code 0 on manual interrupt means an open-ended run (-d 0, the default) that you stop yourself will always report success, no matter what actually happened. For CI or any automated gate, set -d to a fixed, non-zero duration so the tool finishes and evaluates on its own; never rely on the exit code from a run you interrupted.

Troubleshooting: the tool exits immediately

Next steps: If you are deploying in Index Cloud, continue working through the Validating and deploying your container flow to share your image and begin the onboarding process. If you are hosting the service yourself, your container is ready to deploy to your own infrastructure.