S3 Telemetry Integration

Integrating your telemetry data into Finout using a S3 bucket of your choosing, requires setting up a consistent export process to an S3 bucket, for which Finout has been granted read access. This ensures that Finout can retrieve and process your data, which should be structured to include a daily timestamp, the telemetry value for each day, and the granularity of the data collected.

1. Generate CSV Export

Generate a daily CSV export by selecting from two approaches for your CSV exports:

1. Maintain a single CSV that receives updates daily with new data entries.

2. Create a new CSV file for each day's telemetry data.

We recommend opting for the second option, which is creating a distinct CSV file for each day's telemetry data.

2. Prepare CSV Format

When preparing your telemetry data for Finout, ensure your CSV files follow the required format, which includes three main column types: Date, Metadata, and Telemetry.

Type of column	How many columns can be used?	Mandatory/Optional	What is this used for?	Format / Notes
Date	Single	Mandatory	Each data row input represents the telemetry for a specific date, accompanied by their metadata values, indicating the day on which the telemetry was recorded.	YYYY-MM-DD “date” column
Metadata (multiple)	0-10	Optional	Any tagging attributes can be utilized to group and filter the telemetry data based on metadata such as team, group, or feature.	Optional
Telemetry	1	Mandatory	Represent the total of the telemetry for the given date and metadata.	Positive number

Important:

1. The CSV must include at least two specific columns: date and <telemetry_name>.

2. The Telemetry column name will be used as the unique identifier for this telemetry within Finout.

3. Column headers must not include spaces.

Example:

In this example, we will supply the mandatory columns date (“date") and telemetry (“bytes") together with an optional metadata column “teamname”.

"date","teamname","bytes"

"2022-06-07","Servers","12.12"

"2022-06-07","Storage","6.22"

"2022-06-08","Network","0.05"

Date = date

teamname = metadata

Bytes = telemetry

3. Create a Folder in the Bucket

Create a separate folder within the bucket for each telemetry integration. This setup allows you to easily reuse permissions for sending other telemetries using the same integration.

4. Set up an S3 Bucket

Set up an S3 bucket. Ensure the bucket is properly configured to store and manage your data securely.

I want to use an S3 bucket already being accessed by Finout

If an S3 bucket is already linked to Finout for the AWS Cost and Usage Report (CUR) or a previous telemetry integration, you can utilize the same S3 bucket and ARN role. The role and bucket details will auto-populate in the Finout console (in the cost center integrations page). In this case, proceed directly to step 5.

I want to set up a new S3 bucket for telemetry-based export

For setting up a new S3 bucket or if you haven't configured one yet, follow the instructions below to grant Finout access to your S3 bucket.

Grant Finout access to an S3 bucket

After creating the daily export, you need to grant Finout access to your export bucket by creating an IAM role that will have read access to the daily export.

1. Copy your “external-id” from the Finout console, or use a random one, preferably in the format of finout-XXXXXXX.

2. Click on create a new cross-account role in IAM to create a role for another AWS account.

3. Enter the AWS Account ID associated with the S3 bucket in the Account ID field.

4. Choose the option to Require external ID and input the “external-id” you got from the Finout console.

   
5. Click Next until you reach the review screen. ​

6. Configure the review as shown below, with one exception: the role name should be: FinoutMetricsReadOnlyRole (unlike in the screenshot below).

   
7. Go to your newly created role. ​

8. Copy the Role ARN and paste it into the Finout console.

9. Click Add permissions and select Create inline policy.

   
10. Select JSON, and insert the following. Replace the <BUCKET_NAME> with the name of your newly created bucket or your existing CUR bucket:

    "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "tag:GetTagKeys"
                ],
                "Resource": "*"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:Get*",
                    "s3:List*"
                ],
                "Resource": "arn:aws:s3:::<BUCKET_NAME>/*"
            },
            {
                "Effect": "Allow",
                "Action": [
                    "s3:Get*",
                    "s3:List*"
                ],
                "Resource": "arn:aws:s3:::<BUCKET_NAME>"
            }
        ]
    }

11. Click Next until you reach the review screen, and name the policy finest-access-policy_telemetry_export (or choose your own name).

5. Provide Data to Finout

Please provide Finout with the following information:

• External ID

• ARN role

• Bucket name

• Folder name within the bucket

• Optional: region