Anomalies

Anomalies Detection Overview

Finout's advanced algorithms analyze historical data to pinpoint cost anomalies within your MegaBill. Finout identifies both cost increases and decreases, allowing you to quickly investigate the reason for any deviations from your regular spending. In addition, you can monitor these anomalies within Finout or receive anomaly updates directly via Slack, MS Teams, ServiceNow (coming soon), or email.

For comprehensive tracking, Finout scans your most frequently used tags, services, cost centers, and virtual tags. Each newly created virtual tag includes an anomaly scan to ensure you get a holistic view of your data.

The following anomalies are tracked automatically:

• AWS: Regions, Tags, Sub Service, Account Name, Entity Name, Charge Type, Instance Type

• Azure: Service, Meter Region, Meter Sub Category, Service Family, Consumed Service

• GCP: Project ID, Labels

• Global: Cost Center

• Kubernetes: deployment, demonset, k8s_namespace, cronjob, Pod Labels

• SnowFlake: query_tag, table_name, cost type, warehouse_name, user name, account

• DataDog: Product, organization, Sub-Product

• All Virtual Tags

Differentiating between anomaly types:

Pre-defined anomalies: These are anomalies identified by Finout for significant cost groups and filters. You have the option to modify these to better fit your specific requirements. For guidance on customizing these anomalies, please see the manage anomalies section.

Custom anomalies: Customize your cloud cost anomaly detection to meet your team's unique needs. You can set your own rules, thresholds, and patterns to align with your specific cost management strategies. Custom anomalies offer the flexibility to pinpoint and tackle cost inconsistencies that are most pertinent to your team. For instructions on creating custom anomalies, refer to the create custom anomalies secton.

There are three main functions in Anomalies:

• Anomalies Feed

• Create a Custom Anomalies Alert

• Manage your Anomalies

Anomalies Feed

In the Anomalies Feed, you can see all of your anomalies and filter, investigate, and manage them.

1. Filters:

   • Timeframe

   • Anomaly threshold: Filter anomalies based on specific thresholds. For example: Set a threshold of over 20% will display all anomalies exceeding this limit.

     
   • Anomaly type: Select from pre-defined or custom anomaly types.

   • Cost center: Select a cost center.

   • Key: Select a Key.

   • Value: Select a value.

2. Search Anomalies: Use free text search to find anomalies based on various terms or descriptions.

3. Create Anomaly Alert: To create an Anomaly Alert, see Create Custom Anomalies.

4. Anomaly Settings: Add a default endpoint:

   1. In the Anomalies Feed, click and then click Anomalies Settings. The Anomalies Settings pop-up appears.

      
   2. Choose a default endpoint and click Save.

      Note: If a default endpoint has been created, all anomaly alerts will automatically be directed to that channel. Should you require a specific alert to be sent to an alternate endpoint, you can customize this preference by adjusting the settings of that particular alert in the edit anomaly section.

   To delete a default endpoint:

   1. In the Anomalies Feed, click and then click Anomalies Settings. The Anomalies Settings pop-up appears.

      
   2. Click X on the default endpoint you would like to delete.

5. Clear Anomalies Feed:

   Important: Proceed with caution, this action will clear the whole anomalies feed. Clearing the feed will remove these notifications, and they won't be available for future reference.

   In the Anomalies Feed, click and then click Clear anomalies feed.
6. Anomaly information: Information regarding a single Anomaly Alert.

7. Investigate: Clicking Investigate opens MegaBill in a new tab with the anomaly configuration (filters) already populated.

8. Delete an Anomaly:

   1. In the Anomalies Feed, click Delete in a select Anomaly.
   2. Click Yes.

9. Add a Comment:

   1. In the Anomalies Feed, click Add Comment in a select Anomaly.

      
   2. Write a comment and click Save.

10. Create a Jira issue.

Create Custom Anomalies

When setting up a custom anomaly alert in Finout, you have two choices: create a single anomaly for a particular cost dimension or generate multiple anomalies simultaneously with a single action.

1. Navigate to Anomalies.

   
2. Select Create Anomaly Alert.

   
3. Assign an Alert Name to your custom anomaly.

   
4. Under Alert Values, select a group of values and apply a cost filter to define the parameters of the anomaly detection scan.

   • Select View: Choose a view from a list of your created views.

     
   • Filters: Specify the anomaly using cost filters. For example, setting a filter for ‘us-east-1’ will create an anomaly for AWS services only within that region.

     
   • Group by: Select a MegaBill key, creating anomalies for all items within that group. For instance, selecting ‘AWS Services’ will generate anomalies for each AWS service.

     
5. Under Alert Thresholds, select the Sensitivity Threshold to specify the alert trigger based on either a percentage or a specific dollar amount.

Sensitivity Threshold: The sensitivity threshold in anomaly detection is designed to help you fine-tune when alerts are triggered. This threshold enables the definition of anomaly parameters based on your operational norms. Default settings include a $20 minimum for cost changes and a 20% deviation from average costs. To detect smaller fluctuations, you could reduce the dollar threshold below $20 or the percentage below 20%. Alternatively, to focus on more substantial anomalies, increase these thresholds above the default values. Adjust these settings to match the level of sensitivity that aligns with your monitoring needs and cost management strategy.

• After defining your group and filters, the associated values will appear. You have the option to activate or deactivate each value, allowing you to refine the anomaly alert parameters, making sure it matches exactly what you're looking for.

  
• A reference point indicating the average daily cost over the last 30 days is provided to guide your threshold setting, reflecting recent spending trends.

• An explanatory sentence will provide clarity on the chosen anomaly alerting conditions.

• Set an anomaly threshold for every value in the group.

  

  When selecting to create a threshold for each value, you'll have the opportunity to specify the cost change and sensitivity threshold individually for each value.

  

1. Alert Endpoints - Easily integrate notifications via Slack, MS Teams, ServiceNow (coming soon), or Email. Select your desired endpoint, ensuring its configuration is completed beforehand, to start receiving anomaly alerts.

   

   1. Set a default endpoint for this anomaly. If no endpoint is defined for a group-by value, the alert for that value will be sent to this default endpoint. If no default endpoint is set, the alert will be sent to the endpoint specified in the anomalies settings.

      
   2. Enable sending alerts to endpoints based on group-by values:

      1. Select an endpoint :

         1. Default endpoint - When the toggle is off, all alerts will be sent to the default endpoint you chose in step a.

         2. Selected endpoints and Metadata endpoints -

            1. Click Select Endpoint and add any additional endpoints to send the alert.

            2. If you group by a virtual tag, it will automatically send Alerts to its associated Metadata endpoints.

               • Click if you would like to disable the Metadata endpoint for this value.
2. Time Interval (Coming soon) - Select the time interval to monitor the anomaly.

   Note:

   • Daily - Choosing a day means that an alert is triggered daily.

   • Weekly - A Week is a weekly alert that notifies you every Tuesday (when you have a full week).

   • The difference between "last week" and "last 7 days" lies in calculating the timeframes. "Last week" refers to the calendar week immediately before the current one. On the other hand, "last 7 days" refers to the prior 7 days from the selected date, regardless of the day of the week.

   

a. Define Evaluation Period - Set your preferred time period (Days or Weeks) to check for anomalies. For example, the last 5 days.

b. Set Comparison Period - This compares the total cost of the current period to the average total cost of several previous periods. For example: You choose 20 days (4 periods). This is compared to the current 5-day total cost to the average total cost of the previous 20 days. Use case -

You want to evaluate anomalies over a 2-day period compared to the previous 6 days:

• Date: Assume today is August 22nd.

• Evaluation Period: Calculates the total cost for the chosen evaluation period: The last 2 days (August 19-20).

• Comparison Period: Calculates the average of the evaluation period (2 days) over the chosen comparison period: the preceding 6 days (August 12-18).

• Alert: An alert is triggered if the time period cost of the evaluation period exceeds the average cost of the comparison period's total costs from the defined thresholds.

1. Seasonality Check (Coming soon) - The Seasonality Check helps reduce false-positive anomaly alerts by recognizing recurring cost patterns. Instead of flagging every cost spike as an anomaly, it checks whether the increase follows a regular weekly or monthly trend before triggering an alert.

   Note: Currently, daily seasonality is supported for an evaluation period of 1 to 6 days.

   Weekday Seasonality

   • Compares the cost on a specific weekday (e.g., Monday) to the average cost of the same weekday over the past few weeks (e.g., the last 4 Mondays).

   • An alert is triggered only if the cost exceeds the expected range based on your alert settings.

   Monthly Seasonality

   • Compares the cost on a specific date (e.g., the 1st of each month) to the average cost on the same date over the past few months (e.g., the last 4 months).

   • An alert is sent if the cost surpasses the historical trend beyond the defined threshold.

   This feature ensures that anomalies are detected more accurately, minimizing noise from predictable fluctuations. What Happens When You Enable Seasonality? When the Seasonality Check is enabled, Finout automatically filters out alerts identified as seasonal anomalies: - They won’t appear in your anomaly feed. - They won’t be sent to your configured endpoint. This ensures that your alerts focus only on unexpected anomalies, helping you cut through the noise of predictable cost fluctuations.
2. Click Save to create the anomaly.

After saving the anomaly, your anomaly will appear under the Manage Anomaly tab. This tab displays a comprehensive table of both custom-created and pre-defined anomalies generated by Finout.

Within the table, each anomaly entry provides:

• Type: Custom anomaly or pre-defined

• Threshold

• Interval

• Endpoint

• Activated or deactivated: Indicating if the anomaly is activated or deactivated.

Manage Anomalies

Deleting, Editing, or Duplicating an Anomaly

1. Navigate to Anomalies.

2. Select the Manage Anomalies tab.

3. Search for the relevant anomaly: Use the search bar for a direct query or apply filters to narrow down results.

4. Select (⋮) beside the relevant anomaly and choose either Edit alert, Delete alert, or Duplicate alert.

5. If you choose to duplicate, set a name for the duplicated anomaly and adjust all fields accordingly.

Note: Pre-defined anomalies can be customized to suit your needs. You have the flexibility to edit the group values by toggling them on or off, ensuring they meet your specific requirements.

When you modify a predefined anomaly, a new custom anomaly is created with the revised settings, and the original predefined anomaly is deactivated.

FAQs

Why does the comparison period need to be a multiple of the evaluation period? The comparison period must be a multiple of the evaluation period to ensure consistency in calculations. This allows Finout to calculate the total cost of each evaluation period within the comparison period and determine an accurate average. For example, if your evaluation period is 3 days, the comparison period could be 9 days (3 evaluation periods) but not 10 days, ensuring reliable and consistent anomaly detection.

Why can’t I choose arbitrary intervals like 3 weeks compared to 4 weeks? To maintain accurate comparisons, the comparison period must align with the evaluation period to ensure equal, consistent time intervals. This alignment ensures anomalies are detected based on reliable averages derived from comparable time intervals.