Amazon Web Service (AWS)

OpenSearch

Database

Amazon OpenSearch is the successor to Elasticsearch service. It is an open-source, distributed search and analytics suite based on Elasticsearch, commonly used for log analytics, full-text search, security intelligence, business analytics, and operational intelligence use cases.

Costs

The cost of using OpenSearch depends on several factors, including the number of search queries, the amount of storage used for indexing data, and the amount of data stored in S3. For search queries, you are charged based on the number of queries made per month. The cost of search queries varies depending on the region you are using. For data storage for indexing, you are charged for the amount of data stored and the number of read and write requests made to the data. The cost of data storage varies depending on the region you are using. For data storage in S3, you are charged for the amount of data stored and the number of requests made to access the data. The cost of data storage varies depending on the region you are using.

Direct Cost

ESInstance:<Instance_Type>

ES:GP2-Storage

ES:GP3-Storage

DataTransfer-Regional-Bytes

AWS-Out-Bytes

Indirect Cost

No items found.

Terraform Name

aws_opensearch_domain

OpenSearch

attributes:

The following arguments are required:

domain_name - (Required) Name of the domain.

The following arguments are optional:

access_policies - (Optional) IAM policy document specifying the access policies for the domain.
advanced_options - (Optional) Key-value string pairs to specify advanced configuration options. Note that the values for these configuration options must be strings (wrapped in quotes) or they may be wrong and cause a perpetual diff, causing Terraform to want to recreate your OpenSearch domain on every apply.
advanced_security_options - (Optional) Configuration block for fine-grained access control. Detailed below.
auto_tune_options - (Optional) Configuration block for the Auto-Tune options of the domain. Detailed below.
cluster_config - (Optional) Configuration block for the cluster of the domain. Detailed below.
cognito_options - (Optional) Configuration block for authenticating Kibana with Cognito. Detailed below.
domain_endpoint_options - (Optional) Configuration block for domain endpoint HTTP(S) related options. Detailed below.
ebs_options - (Optional) Configuration block for EBS related options, may be required based on chosen instance size. Detailed below.
engine_version - (Optional) Either Elasticsearch_X.Y or OpenSearch_X.Y to specify the engine version for the Amazon OpenSearch Service domain. For example, OpenSearch_1.0 or Elasticsearch_7.9. See Creating and managing Amazon OpenSearch Service domains. Defaults to OpenSearch_1.1.
encrypt_at_rest - (Optional) Configuration block for encrypt at rest options. Only available for certain instance types. Detailed below.
log_publishing_options - (Optional) Configuration block for publishing slow and application logs to CloudWatch Logs. This block can be declared multiple times, for each log_type, within the same resource. Detailed below.
node_to_node_encryption - (Optional) Configuration block for node-to-node encryption options. Detailed below.
snapshot_options - (Optional) Configuration block for snapshot related options. Detailed below. DEPRECATED. For domains running OpenSearch 5.3 and later, Amazon OpenSearch takes hourly automated snapshots, making this setting irrelevant. For domains running earlier versions, OpenSearch takes daily automated snapshots.
tags - (Optional) Map of tags to assign to the resource. If configured with a provider default_tags configuration block present, tags with matching keys will overwrite those defined at the provider-level.
vpc_options - (Optional) Configuration block for VPC related options. Adding or removing this configuration forces a new resource (documentation). Detailed below.

advanced_security_options

anonymous_auth_enabled - (Optional) Whether Anonymous auth is enabled. Enables fine-grained access control on an existing domain. Ignored unless advanced_security_options are enabled. Can only be enabled on an existing domain.
enabled - (Required, Forces new resource when changing from true to false) Whether advanced security is enabled.
internal_user_database_enabled - (Optional) Whether the internal user database is enabled. Default is false.
master_user_options - (Optional) Configuration block for the main user. Detailed below.

master_user_options

master_user_arn - (Optional) ARN for the main user. Only specify if internal_user_database_enabled is not set or set to false.
master_user_name - (Optional) Main user's username, which is stored in the Amazon OpenSearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.
master_user_password - (Optional) Main user's password, which is stored in the Amazon OpenSearch Service domain's internal database. Only specify if internal_user_database_enabled is set to true.

auto_tune_options

desired_state - (Required) Auto-Tune desired state for the domain. Valid values: ENABLED or DISABLED.
maintenance_schedule - (Required if rollback_on_disable is set to DEFAULT_ROLLBACK) Configuration block for Auto-Tune maintenance windows. Can be specified multiple times for each maintenance window. Detailed below.
rollback_on_disable - (Optional) Whether to roll back to default Auto-Tune settings when disabling Auto-Tune. Valid values: DEFAULT_ROLLBACK or NO_ROLLBACK.

maintenance_schedule

start_at - (Required) Date and time at which to start the Auto-Tune maintenance schedule in RFC3339 format.
duration - (Required) Configuration block for the duration of the Auto-Tune maintenance window. Detailed below.
cron_expression_for_recurrence - (Required) A cron expression specifying the recurrence pattern for an Auto-Tune maintenance schedule.

duration

value - (Required) An integer specifying the value of the duration of an Auto-Tune maintenance window.
unit - (Required) Unit of time specifying the duration of an Auto-Tune maintenance window. Valid values: HOURS.

cluster_config

cold_storage_options - (Optional) Configuration block containing cold storage configuration. Detailed below.
dedicated_master_count - (Optional) Number of dedicated main nodes in the cluster.
dedicated_master_enabled - (Optional) Whether dedicated main nodes are enabled for the cluster.
dedicated_master_type - (Optional) Instance type of the dedicated main nodes in the cluster.
instance_count - (Optional) Number of instances in the cluster.
instance_type - (Optional) Instance type of data nodes in the cluster.
warm_count - (Optional) Number of warm nodes in the cluster. Valid values are between 2 and 150. warm_count can be only and must be set when warm_enabled is set to true.
warm_enabled - (Optional) Whether to enable warm storage.
warm_type - (Optional) Instance type for the OpenSearch cluster's warm nodes. Valid values are ultrawarm1.medium.search, ultrawarm1.large.search and ultrawarm1.xlarge.search. warm_type can be only and must be set when warm_enabled is set to true.
zone_awareness_config - (Optional) Configuration block containing zone awareness settings. Detailed below.
zone_awareness_enabled - (Optional) Whether zone awareness is enabled, set to true for multi-az deployment. To enable awareness with three Availability Zones, the availability_zone_count within the zone_awareness_config must be set to 3.

cold_storage_options

enabled - (Optional) Boolean to enable cold storage for an OpenSearch domain. Defaults to false. Master and ultrawarm nodes must be enabled for cold storage.

zone_awareness_config

availability_zone_count - (Optional) Number of Availability Zones for the domain to use with zone_awareness_enabled. Defaults to 2. Valid values: 2 or 3.

cognito_options

AWS documentation: Amazon Cognito Authentication for Kibana

enabled - (Optional) Whether Amazon Cognito authentication with Kibana is enabled or not. Default is false.
identity_pool_id - (Required) ID of the Cognito Identity Pool to use.
role_arn - (Required) ARN of the IAM role that has the AmazonOpenSearchServiceCognitoAccess policy attached.
user_pool_id - (Required) ID of the Cognito User Pool to use.

domain_endpoint_options

custom_endpoint_certificate_arn - (Optional) ACM certificate ARN for your custom endpoint.
custom_endpoint_enabled - (Optional) Whether to enable custom endpoint for the OpenSearch domain.
custom_endpoint - (Optional) Fully qualified domain for your custom endpoint.
enforce_https - (Optional) Whether or not to require HTTPS. Defaults to true.
tls_security_policy - (Optional) Name of the TLS security policy that needs to be applied to the HTTPS endpoint. Valid values: Policy-Min-TLS-1-0-2019-07 and Policy-Min-TLS-1-2-2019-07. Terraform will only perform drift detection if a configuration value is provided.

ebs_options

ebs_enabled - (Required) Whether EBS volumes are attached to data nodes in the domain.
iops - (Optional) Baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the GP3 and Provisioned IOPS EBS volume types.
throughput - (Required if volume_type is set to gp3) Specifies the throughput (in MiB/s) of the EBS volumes attached to data nodes. Applicable only for the gp3 volume type. Valid values are between 125 and 1000.
volume_size - (Required if ebs_enabled is set to true.) Size of EBS volumes attached to data nodes (in GiB).
volume_type - (Optional) Type of EBS volumes attached to data nodes.

encrypt_at_rest

enabled - (Required) Whether to enable encryption at rest. If the encrypt_at_rest block is not provided then this defaults to false. Enabling encryption on new domains requires an engine_version of OpenSearch_X.Y or Elasticsearch_5.1 or greater.
kms_key_id - (Optional) KMS key ARN to encrypt the Elasticsearch domain with. If not specified then it defaults to using the aws/es service KMS key. Note that KMS will accept a KMS key ID but will return the key ARN. To prevent Terraform detecting unwanted changes, use the key ARN instead.

log_publishing_options

cloudwatch_log_group_arn - (Required) ARN of the Cloudwatch log group to which log needs to be published.
enabled - (Optional, Default: true) Whether given log publishing option is enabled or not.
log_type - (Required) Type of OpenSearch log. Valid values: INDEX_SLOW_LOGS, SEARCH_SLOW_LOGS, ES_APPLICATION_LOGS, AUDIT_LOGS.

node_to_node_encryption

enabled - (Required) Whether to enable node-to-node encryption. If the node_to_node_encryption block is not provided then this defaults to false. Enabling node-to-node encryption of a new domain requires an engine_version of OpenSearch_X.Y or Elasticsearch_6.0 or greater.

snapshot_options

automated_snapshot_start_hour - (Required) Hour during which the service takes an automated daily snapshot of the indices in the domain.

vpc_options

AWS documentation: VPC Support for Amazon OpenSearch Service Domains

security_group_ids - (Optional) List of VPC Security Group IDs to be applied to the OpenSearch domain endpoints. If omitted, the default Security Group for the VPC will be used.
subnet_ids - (Required) List of VPC Subnet IDs for the OpenSearch domain endpoints to be created in.

‍

Associating resources with a

OpenSearch

Resources do not "belong" to a

OpenSearch

Rather, one or more Security Groups are associated to a resource.

Create

OpenSearch

via Terraform:

The following HCL creates an OpenSearch domain

Syntax:

resource "aws_opensearch_domain" "example" {
domain_name = "example"
engine_version = "Elasticsearch_7.10"

cluster_config {
instance_type = "r4.large.search"
}

tags = {
Domain = "TestDomain"
}
}

‍

Create

OpenSearch

via CLI:

Parametres:

create-domain
--domain-name <value>
[--engine-version <value>]
[--cluster-config <value>]
[--ebs-options <value>]
[--access-policies <value>]
[--snapshot-options <value>]
[--vpc-options <value>]
[--cognito-options <value>]
[--encryption-at-rest-options <value>]
[--node-to-node-encryption-options <value>]
[--advanced-options <value>]
[--log-publishing-options <value>]
[--domain-endpoint-options <value>]
[--advanced-security-options <value>]
[--tag-list <value>]
[--auto-tune-options <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]
[--debug]
[--endpoint-url <value>]
[--no-verify-ssl]
[--no-paginate]
[--output <value>]
[--query <value>]
[--profile <value>]
[--region <value>]
[--version <value>]
[--color <value>]
[--no-sign-request]
[--ca-bundle <value>]
[--cli-read-timeout <value>]
[--cli-connect-timeout <value>]
[--cli-binary-format <value>]
[--no-cli-pager]
[--cli-auto-prompt]
[--no-cli-auto-prompt]

Example:

aws opensearch create-domain \
--domain-name 'opensearch-domain' \
--engine-version 'OpenSearch_1.1' \
--cluster-config '{"DedicatedMasterEnabled":true,"InstanceCount":3,"InstanceType":"r6g.large.search","DedicatedMasterCount":3,"DedicatedMasterType":"r6g.large.search","ZoneAwarenessEnabled":false}' \
--encryption-at-rest-options '{"Enabled":false}' \
--log-publishing-options '{"SEARCH_SLOW_LOGS":{"Enabled":false},"ES_APPLICATION_LOGS":{"Enabled":false},"INDEX_SLOW_LOGS":{"Enabled":false},"AUDIT_LOGS":{"Enabled":false}}' \
--node-to-node-encryption-options '{"Enabled":false}' \
--ebs-options '{"EBSEnabled":true,"VolumeSize":50}'

Best Practices for

OpenSearch

Categorized by Availability, Security & Compliance and Cost

Low

Ensure Auto-Tune feature is enabled in OpenSearch clusters

Other

No items found.

Medium

Ensure OpenSearch Service Domain AdvancedSecurityOptions are enabled

Security & Compliance

No items found.

Critical