Sending segment files to Index using Amazon S3 buckets

Integration Partners
Last Updated: March 03, 2026•

If you're a supported data partner enriching transactions on our exchange, you can send your segment files to Index Exchange (Index) using Amazon S3 buckets for ingestion into our platform.

Workflow

To send segment files to Index, you must follow this workflow:

  1. You must have an Amazon S3 bucket owned by your Amazon Web Services (AWS) account that is ready for Index ingestion. If you don't yet have your S3 bucket configured, you must:

    1. Create the bucket.

    2. Permission the bucket for Index. Create an IAM role in your AWS account that trusts the Index internal role, attach an S3 read policy (ListBucket/GetObject) for your bucket to that role, and share the role ARN with Index. For instructions on how to permission the bucket, see Grant Index Exchange access to your Amazon S3 bucket.

  2. All segments must be registered in the Index ecosystem before segment data can be sent to Index. If the file you want to send contains any new, unregistered segments, you must register those segments first. If all segment IDs have previously been registered with Index, this step can be skipped. You can register the segments using an API or by uploading a taxonomy file.

  3. Create the segment data files that contain segment information using the required CSV format.

Register a segment using a taxonomy file

All segments must be registered in the Index ecosystem. If the segment data file contains any new segments, you must first register them using a taxonomy file. The taxonomy file has a maximum size of 30 kB. If all segments cannot fit within this limit, we recommend using multiple taxonomy files. Both the externalSegmentID and externalSegmentName are limited to 255 characters and must use UTF-8 encoding.

  1. Create the taxonomy file using the valid taxonomy file structure. Taxonomy files must be valid JSON and follow this structure:

    { 
	"version": "2.0", // The version which must be "2.0"
	"segments": [  
		{ 
			"externalSegmentID": "segment_id_1", // The external-facing segment identifier
			"externalSegmentName": "Segment Name 1", // The human-readable name
			"accountID": 1234567 // The Index generated account ID for the owner of the segment.
		}, 
		{ 
			"externalSegmentID": "segment_id_2", // The external-facing segment identifier
			"externalSegmentName": "Segment Name 2", // The human-readable name
			"accountID": 2345678 // The Index generated account ID for the owner of the segment.
		} 
	] 
}

  2. Upload the taxonomy file to your designated S3 bucket and the specific metafiles/ folder.

Create the segment data files to upload

Segment data files contain the data-to-segment associations to be ingested by Index. For a list of supported ad request signals that can be included in the data file, see List of supported ad request signals that can be included in the data file.

To be able to successfully upload a file, the following requirements must be met:

  • Files must be:

    • UTF-8 encoded

    • GZIP compressed (.gz)x

  • File size must not exceed 5 GB after being compressed. Split large files into multiple smaller files and compress each one.

  • File uploads are limited to 6 GB per UTC day. Files beyond this limit are processed the next day in upload timestamp order.

  1. Create a CSV file that contains the data-to-segment associations using the following format, where:

    • Each line must include at least one segment ID

    • There is a maximum of 1000 segment IDs per line

    • No duplicate segment IDs can exist in a single line

    <ID>,<externalSegmentID_1>,<externalSegmentID_2>,...

    Example

    id123,segA,segB,segC  
id456,segA  
id789,segB,segC

  2. Name the file using this naming structure, <randomSuffix>.<operationType>.csv.gz, where:

    • randomSuffix: Short alphanumeric string to ensure uniqueness. File names must be unique within each folder. A recommended way to keep filenames unique is to use a timestamp in the filename.

    • operationType: Can either be append to add new segments or delete to remove segments.

    3. Examples

    f3a9c2.append.csv.gz 
92b7fd.delete.csv.gz

  3. Upload the segment file to your designated S3 bucket and the correct folder. The folder name must tell Index what type of identifier is used in the first column of your file. All files in a folder must use the same identifier type. Uploading a file to the wrong folder may result in validation or processing errors.

    Examples

    s3://your-bucket/deviceip/f3a9c2.append.csv.gz
    s3://your-bucket/url/f3a9c2.append.csv.gz
    s3://your-bucket/id5/f3a9c2.append.csv.gz

Common mistakes to avoid

  • Referencing segment IDs that are not registered. Make sure all segments have been registered before including them in the segment data file.

  • Uploading files larger than 5 GB (compressed).

  • Uploading more than 6 GB per UTC day.

  • Uploading files to the wrong folder. All files in a folder must use the same identifier type.

  • Not using the correct filename or data types.