Anomaly Detection- Introduction
Finout's advanced machine-learning algorithms analyze historical data to pinpoint cost anomalies within your MegaBill. Finout identifies both cost increases and decreases, allowing you to quickly investigate the reason for any deviations from your regular spending. In addition, you can monitor these anomalies within Finout or receive anomaly updates directly via Slack.
For comprehensive tracking, Finout scans your most frequently used tags, services, cost centers, and virtual tags. Each newly created virtual tag includes an anomaly scan to ensure you get a holistic view of your data.
The following anomalies are tracked automatically:
AWS: Regions, Tags, Sub Service, Account Name, Entity Name, Charge Type, Instance Type
Azure: Service, Meter Region, Meter Sub Category, Service Family, Consumed Service
GCP: Project ID, Labels
Global: Cost Center
Kubernetes: deployment, demonset, k8s_namespace, cronjob, Pod Labels
SnowFlake: query_tag, table_name, cost type, warehouse_name, user name, account
DataDog: Product, organization, Sub-Product
All Virtual Tags
Differentiating between anomaly types
Pre-defined anomalies: These are anomalies identified by Finout for significant cost groups and filters. You have the option to modify these to better fit your specific requirements. For guidance on customizing these anomalies, please see the manage anomalies section.
Custom anomalies: Customize your cloud cost anomaly detection to meet your team's unique needs. You can set your own rules, thresholds, and patterns to align with your specific cost management strategies. Custom anomalies offer the flexibility to pinpoint and tackle cost inconsistencies that are most pertinent to your team. For instructions on creating custom anomalies, refer to the relevant section here.
Anomalies Feed- Filtering Anomalies
On the Anomalies Feed screen shown above, you can filter anomalies using several criteria:
Date
Anomaly threshold: Filter anomalies based on specific thresholds. For example, setting a threshold of over 20% will display all anomalies exceeding this limit.
Anomaly type: Pre-defined or custom.
Cost center
Key
Value
Free text: Use free text search to find anomalies based on various terms or descriptions.
Create Custom Anomalies
When setting up a custom anomaly alert in Finout, you have two choices: you can create a single anomaly for a particular cost dimension, or generate multiple anomalies simultaneously with a single action.
Navigate to Anomalies.
Select Add Anomaly Alert.
Assign an Anomaly Name to your custom anomaly.
Select the cost dimension for the anomaly by either choosing Group by or by defining cost filters.
Group by: Select a MegaBill group, creating anomalies for all items within that group. For instance, selecting ‘AWS Services’ will generate anomalies for each AWS service.
Cost filters: Specify the anomaly using cost filters. For example, setting a filter for ‘us-east-1’ will create an anomaly for AWS services only within that region.
Please note: You can create a single anomaly by simply selecting the filters for that anomaly. For example - SubService -> Ec2-Compute.
Select the Cost Anomaly and Sensitivity Threshold to specify the alert trigger based on either a percentage or a specific dollar amount.
Sensitivity Threshold: The sensitivity threshold in anomaly detection is designed to help you fine-tune when alerts are triggered. This threshold enables the definition of anomaly parameters based on your operational norms. Default settings include a $20 minimum for cost changes and a 20% deviation from average costs. To detect smaller fluctuations, you could reduce the dollar threshold below $20 or the percentage below 20%. Alternatively, to focus on more substantial anomalies, increase these thresholds above the default values. Adjust these settings to match the level of sensitivity that aligns with your monitoring needs and cost management strategy.
Please note:
A reference point indicating the average daily cost over the last 30 days is provided to guide your threshold setting, reflecting recent spending trends.
An explanatory sentence will provide clarity on the chosen anomaly alerting conditions.
You have the flexibility to set different anomaly thresholds for every value in the group by.
When selecting to create a threshold for each value, you'll have the opportunity to specify the cost change and sensitivity threshold individually for each value.
After defining your group and filters, the associated values will appear. You have the option to activate or deactivate each value, allowing you to refine the anomaly alert parameters, making sure it matches exactly what you're looking for.
Select the Time Interval for monitoring the anomaly.
(Optional) Set Slack Notification- Integrate Slack notifications by selecting a Slack endpoint. Ensure you have completed the configuration for sending anomalies to Slack.
Click Save to create the anomaly.
After saving the anomaly, your anomaly will appear under the Manage Anomaly tab. This tab displays a comprehensive table of both custom-created and pre-defined anomalies generated by Finout.
Within the table, each anomaly entry provides:
Type: Custom anomaly or pre-defined
Threshold
Interval
End Point
Activated or deactivated: Indicating if the anomaly is activated or deactivated.
Review Anomalies
Navigate to Anomalies.
Search for the relevant anomaly: Use the search bar for a direct query or apply filters to narrow down results.
To continue the investigation on the MegaBill page, select investigate. Following this, the MegaBill page will open in a new tab with the anomaly configuration (filters) already populated.
To remove an anomaly, choose Delete and confirm with Yes.
To leave a comment on an anomaly, click Add comment, enter a comment, and click Save.
Manage Anomalies- Deleting or Editing an Anomaly
Navigate to Anomalies.
Select the Manage Anomalies tab.
Locate the desired anomaly from the list.
Select (⋮) beside the relevant anomaly and choose either Edit alert or Delete alert.
Note: Pre-defined anomalies can be customized to suit your needs. You have the flexibility to edit the group values by toggling them on or off, ensuring they meet your specific requirements.
When you modify a predefined anomaly, a new custom anomaly is created with the revised settings, and the original predefined anomaly is deactivated.
Anomalies Default Slack Endpoint
Anomalies can be posted as messages to Slack.
Create a Slack endpoint (see Create an Endpoint).
Go to the Anomalies section and click on the three dots (:) to access Anomalies Settings.
Choose a Default Endpoint.
If you haven't set up an endpoint yet:
Choose the option create new endpoint.
After creation, the new Slack endpoint will be added as the default endpoint.
Note: If a default endpoint has been created, all anomaly alerts will automatically be directed to that channel. Should you require a specific alert to be sent to an alternate endpoint, you can customize this preference by adjusting the settings of that particular alert in the edit anomaly section.
Removing a default endpoint
Within the Anomalies Settings, select the endpoint you wish to remove and deselect it.
Still need help? Please feel free to reach out to our team at [email protected].