Build a gRPC RTD service

This topic covers implementing a gRPC-based real-time data integration service. All new containers should be built for gRPC.

gRPC integration build guide

Complete the following steps to set up the proto files and generate the language-specific bindings your service requires. This uses protoc, the Protocol Buffers compiler, which converts .proto schema files into language-specific code. Installation instructions are available at grpc.io/docs/protoc-installation.

1. Download the official OpenRTB An open industry standard for communication between buyers and sellers of online advertising in real-time bidding auctions. It's published by the IAB. 2.6 Protocol Buffers specification from the IAB Tech Lab's openrtb2.x repository.

2. Clone the ARTF reference implementation repository, which contains the Makefile, build scripts, and proto files referenced in the steps below.

3. Generate the language-specific OpenRTB and gRPC bindings:

   1. Install make and protoc v27 or later. This is required for edition = "2023" support. The version in most base package managers is too old and errors on --experimental_editions.

   2. Install the language plugins. For example, for Go:

      go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
      go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

   3. Open the Makefile and choose the languages under LANGUAGES.

   4. Run make bindings. Plain make builds the reference agent, not the bindings.

4. Alternatively, skip generation and use the pre-generated Go bindings included in the repository under pkg/pb/ (pkg/pb/artf, pkg/pb/openrtb).

Service contract

Your service implements one gRPC service with one RPC, both defined in the proto files you generated bindings from above. That single RPC is where all of your logic lives.

In Index Cloud, your container also needs the following:

Latency

Two separate numbers govern your response time, and they do different things. Getting them the wrong way round is the most common source of confusion here.

tmax is the hard deadline for a single request. It is fixed at 5ms, at both extension points, and it does not vary per request. A response that arrives after tmax is not applied to that auction.

Your 95th percentile response time is what determines whether you keep receiving full traffic. The threshold depends on which extension point the request came from:

Cross the p95 threshold for your extension point and traffic to your container is automatically throttled, and Index may lower your QPS Queries Per Second (QPS). The number of bid requests a DSP processes per second. Also known as impressions per second. accordingly.

Note: At the Publisher Request extension point this gives you real margin. A response between 5ms and 30ms misses tmax and is discarded for that auction, but it does not by itself put you over the p95 bar. At the DSP Bid Response extension point there is no margin, because the p95 bar and tmax are both 5ms. If you are implementing Bid Price Modification, that is the bar you are working to. See Implementing Bid Price Modification.

Calls use keepalive connections to minimize network overhead.

Response codes

Standard gRPC status codes should be returned.

Circuit breaker logic

If Index receives any status code other than OK, or more than 5% of requests have exceeded an internal timeout value of 30ms, Index's circuit breaker triggers and requests to your service are throttled. Requests that time out also trigger this logic.

Circuit breaker logic protects both Index and partner systems from cascade failures. Once error rates return to nominal levels, measured over a rolling window, circuit breakers reopen and regular traffic volume resumes automatically. No action is required from your side during recovery.

Note: The 30ms internal timeout applies to both gRPC and HTTP integrations. It is the outer timeout after which Index treats a request as failed, which is a different thing from tmax. See Latency above for how the two relate.

gRPC-supported lifecycle extension points and mutations

The following table summarizes all supported lifecycle extension points, the mutations available at each, and their paths. For full schema details and examples, see the Mutation reference.

What your service receives

At the Publisher Request extension point, your GetMutations handler receives an RTBRequest whose bid_request field holds the full privacy-safe OpenRTB payload. For the fields included in that payload, see the Publisher request payload reference. Index calls GetMutations in real time during auction processing, once per extension point.

Two other RTBRequest fields matter for every call:

• tmax is the maximum number of milliseconds Index waits for your response, fixed at 5ms. Treat it as a hard deadline. See Latency above.

• applicable_intents is the list of mutation intents Index accepts on that call. Filter your mutations to only these. A mutation whose intent is not in applicable_intents is silently discarded.

Note: The bid_response field is empty at the Publisher Request extension point. Note that device.ip, device.geo.lat, device.geo.lon, and device.geo.zip may be anonymized, masked, or withheld depending on the request user's privacy settings. See the Publisher request payload reference for the full list of modifications.

The following is an example of a complete RTBRequest at the Publisher Request extension point, in protojson format:

{
   "id": "req-1",
   "lifecycle": "LIFECYCLE_PUBLISHER_BID_REQUEST",
   "tmax": 5,
   "applicable_intents": [
      "ACTIVATE_SEGMENTS", "ACTIVATE_DEALS", "SUPPRESS_DEALS",
      "ADJUST_DEAL_FLOOR", "ADD_CIDS"
   ],
   "bid_request": {
      "id": "brq-1",
      "imp": [{
 "id": "1",
 "banner": { "w": 300, "h": 250, "pos": 1, "format": [{ "w": 300, "h": 250 }] },
 "bidfloor": 0.5,
 "ext": { "gpid": "/1234567/home/mpu/atf", "sid": "abc123" }
      }],
      "site": {
 "id": "12345", "domain": "example.com",
 "page": "https://example.com/section/",
 "publisher": { "id": "555555", "domain": "example.com", "name": "Publisher Name" }
      },
      "device": {
 "connectiontype": 2, "devicetype": 2, "dnt": 0,
 "ip": "2001:db8::", "language": "en", "lmt": 0,
 "ua": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/114.0.0.0"
      },
      "regs": { "coppa": 0 }
   }
}

Note: lifecycle, id, tmax, and bid_request are required on every RTBRequest and must be set on every call. Echo id back unchanged in RTBResponse.id.

What your service returns

Your GetMutations handler returns an RTBResponse containing the following fields:

Implementation pattern

Structure your handler as follows:

1. Respect tmax. Set a deadline on any inner work using context.WithDeadline.

2. Read signals from bid_request.

3. Filter by applicable_intents. Only build mutation types that are eligible for this call.

4. Build mutations per the schema (intent, path, op, value). Use one mutation per entity.

5. Return the response with the same id, your mutations, and metadata.

At the DSP Bid Response extension point

When lifecycle is LIFECYCLE_DSP_BID_RESPONSE, the bid_response field is populated, and that is what you operate on. On this post-bid path, read signals from bid_response as well as bid_request.

Note: bid_request is still required on every RTBRequest at this extension point. Omitting it fails to marshal.

For the BID_SHADE mutation used at this extension point, see Implementing Bid Price Modification.

Next: Implement whichever mutations you need. See Mutation reference for the schema of all six, and Implementing Bid Price Modification for additional depth if Bid Price Modification is one of them. Then continue to Building a Docker container to package your container.