聚合在Elasticsearch中的使用及示例验证

聚合在Elasticsearch中的使用

系统中使用的ES环境不一定每篇文章都有,但是可以在合集中找到,关注《醉鱼Java》一起进步

环境

  • elasticsearch 8.1

搭建

version: '3.8'
services:
cerebro:
image: lmenezes/cerebro:0.8.3
container_name: cerebro
ports:
- "9000:9000"
command:
- -Dhosts.0.host=http://eshot:9200
networks:
- elastic
kibana:
image: docker.elastic.co/kibana/kibana:8.1.3
container_name: kibana
environment:
- I18N_LOCALE=zh-CN
- XPACK_GRAPH_ENABLED=true
- TIMELION_ENABLED=true
- XPACK_MONITORING_COLLECTION_ENABLED="true"
- ELASTICSEARCH_HOSTS=http://eshot:9200
- server.publicBaseUrl=http://192.168.160.234:5601
ports:
- "5601:5601"
networks:
- elastic
eshot:
image: elasticsearch:8.1.3
container_name: eshot
environment:
- node.name=eshot
- cluster.name=es-docker-cluster
- discovery.seed_hosts=eshot,eswarm,escold
- cluster.initial_master_nodes=eshot,eswarm,escold
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.security.enabled=false
- node.attr.node_type=hot
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- D:zuiyuftpdockeres8.1eshotdata:/usr/share/elasticsearch/data
- D:zuiyuftpdockeres8.1eshotlogs:/usr/share/elasticsearch/logs
- D:zuiyuftpdockeres8.1eshotplugins:/usr/share/elasticsearch/plugins
ports:
- 9200:9200
networks:
- elastic
eswarm:
image: elasticsearch:8.1.3
container_name: eswarm
environment:
- node.name=eswarm
- cluster.name=es-docker-cluster
- discovery.seed_hosts=eshot,eswarm,escold
- cluster.initial_master_nodes=eshot,eswarm,escold
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.security.enabled=false
- node.attr.node_type=warm
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- D:zuiyuftpdockeres8.1eswarmdata:/usr/share/elasticsearch/data
- D:zuiyuftpdockeres8.1eswarmlogs:/usr/share/elasticsearch/logs
- D:zuiyuftpdockeres8.1eshotplugins:/usr/share/elasticsearch/plugins
networks:
- elastic
escold:
image: elasticsearch:8.1.3
container_name: escold
environment:
- node.name=escold
- cluster.name=es-docker-cluster
- discovery.seed_hosts=eshot,eswarm,escold
- cluster.initial_master_nodes=eshot,eswarm,escold
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.security.enabled=false
- node.attr.node_type=cold
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- D:zuiyuftpdockeres8.1escolddata:/usr/share/elasticsearch/data
- D:zuiyuftpdockeres8.1escoldlogs:/usr/share/elasticsearch/logs
- D:zuiyuftpdockeres8.1eshotplugins:/usr/share/elasticsearch/plugins
networks:
- elastic

# volumes:
# eshotdata:
# driver: local
# eswarmdata:
# driver: local
# escolddata:
# driver: local

networks:
elastic:
driver: bridge

什么是聚合?

在Elasticsearch中,聚合是一种功能强大的数据处理技术,它允许我们对索引中的数据进行多种计算和分析操作。聚合可以理解为对数据集进行分组,并在每个分组上执行各种指标计算,类似于SQL中的GROUP BY和聚合函数。

示例数据

为了验证聚合功能,我们将使用一个示例数据集,假设我们有一个存储了商品信息的索引,包含以下字段:

  1. product_name:商品名称
  2. category:商品分类
  3. price:商品价格
  4. quantity:商品数量
  5. manufacturer:制造商
  6. timestamp:记录时间戳

下面我们导入测试数据

创建索引

PUT /zfc-doc-000001
{
"settings": {
"index":{
"number_of_shards":3,
"number_of_replicas":2
}
},
"mappings": {
"properties": {
"product_name":{
"type":"keyword"
},
"category":{
"type":"keyword"
},
"price":{
"type": "integer"
},
"quantity":{
"type": "integer"
},
"manufacturer":{
"type": "keyword"
},
"timestamp":{
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
}
}
}
}

添加数据

PUT _bulk
{"index":{"_index":"zfc-doc-000002","_id":"1"}}
{"product_name": "iPhone 12","category": "Electronics","price": 999,"quantity": 50,"manufacturer": "Apple","timestamp": "2023-07-24 10:00:00"}
{"index":{"_index":"zfc-doc-000002","_id":"2"}}
{"product_name": "Samsung Galaxy S21","category": "Electronics","price": 799,"quantity": 30,"manufacturer": "Samsung","timestamp": "2023-07-24 11:30:00"}
{"index":{"_index":"zfc-doc-000002","_id":"3"}}
{"product_name": "Sony Bravia 65-inch TV","category": "Electronics","price": 1499,"quantity": 20,"manufacturer": "Sony","timestamp": "2023-07-24 13:15:00"}
{"index":{"_index":"zfc-doc-000002","_id":"4"}}
{"product_name": "HP Spectre x360","category": "Electronics","price": 1299,"quantity": 25,"manufacturer": "HP","timestamp": "2023-07-24 15:45:00"}
{"index":{"_index":"zfc-doc-000002","_id":"5"}}
{"product_name": "Dell XPS 15", "category": "Electronics","price": 1399,"quantity": 15,"manufacturer": "Dell","timestamp": "2023-07-24 17:20:00"}
{"index":{"_index":"zfc-doc-000002","_id":"6"}}
{"product_name": "Nike Air Zoom Pegasus 38", "category": "Sports","price": 119,"quantity": 100,"manufacturer": "Nike","timestamp": "2023-07-24 09:30:00"}
{"index":{"_index":"zfc-doc-000002","_id":"7"}}
{"product_name": "Adidas Ultraboost 21","category": "Sports","price": 129,"quantity": 80,"manufacturer": "Adidas","timestamp": "2023-07-24 10:45:00"}
{"index":{"_index":"zfc-doc-000002","_id":"8"}}
{"product_name": "Canon EOS Rebel T7i","category": "Electronics","price": 699,"quantity": 10,"manufacturer": "Canon","timestamp": "2023-07-24 14:05:00"}
{"index":{"_index":"zfc-doc-000002","_id":"9"}}
{"product_name": "LG 55-inch 4K TV", "category": "Electronics","price": 899,"quantity": 30,"manufacturer": "LG","timestamp": "2023-07-24 16:30:00"}
{"index":{"_index":"zfc-doc-000002","_id":"10"}}
{"product_name": "Lenovo ThinkPad X1 Carbon", "category": "Electronics","price": 1599,"quantity": 18,"manufacturer": "Lenovo","timestamp": "2023-07-24 18:10:00"}

聚合示例

1. 词条聚合(Terms Aggregation)

词条聚合是一种用于对文本字段进行分组的聚合方式,它会将相同值的文档分到同一个桶(Bucket)中,并计算每个桶中文档的数量。

示例查询:

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"category_count": {
"terms": {
"field": "category",
"size": 10
}
}
}
}

解释:

  • "size": 0:表示只返回聚合结果,不返回实际文档数据。

  • "aggs":定义聚合操作。

  • "category_count":自定义的聚合名称,用于标识结果。

  • "terms":指定使用词条聚合。

  • "field": "category":指定要进行聚合的字段。


2. 嵌套聚合(Nested Aggregation)

嵌套聚合允许在一个桶内进行更深层次的聚合操作。例如,我们可以先按分类分组,然后在每个分类内再按制造商进行分组,并计算每个分类下的平均价格。

示例查询:

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"category_group": {
"terms": {
"field": "category",
"size": 10
},
"aggs": {
"avg_price": {
"avg": {
"field": "price"
}
}
}
}
}
}

解释:

  • "aggs":定义聚合操作。
  • "category_group":自定义的聚合名称,用于标识结果。
  • "terms":指定使用词条聚合。
  • "field": "category":指定要进行聚合的字段。
  • "avg_price":自定义的聚合名称,用于标识结果。
  • "avg":指定使用平均值聚合。
  • "field": "price":指定要进行聚合的数值字段。

3.直方图聚合示例(Histogram)

假设我们希望根据商品价格(price字段)创建一个价格区间的直方图,将商品按照价格范围进行分组,并统计每个价格区间内的商品数量。

示例查询:

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"price_histogram": {
"histogram": {
"field": "price",
"interval": 200
}
}
}
}

解释:

  • "aggs":定义聚合操作。
  • "price_histogram":自定义的聚合名称,用于标识结果。
  • "histogram":指定使用直方图聚合。
  • "field": "price":指定要进行聚合的数值字段,即商品价格。
  • "interval": 200:指定直方图的间隔大小,这里设置为200表示将价格范围划分为200的区间,例如:0-200、200-400、400-600等。

4.范围聚合示例(Range)

范围聚合允许我们根据指定的范围条件将文档分组,例如:按价格范围进行分组并统计每个价格范围内的商品数量。

示例查询:

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"price_ranges": {
"range": {
"field": "price",
"ranges": [
{ "from": 0, "to": 200 },
{ "from": 200, "to": 500 },
{ "from": 500, "to": 1000 },
{ "from": 1000 }
]
}
}
}
}

解释:

  • "aggs":定义聚合操作。
  • "price_ranges":自定义的聚合名称,用于标识结果。
  • "range":指定使用范围聚合。
  • "field": "price":指定要进行聚合的数值字段,即商品价格。
  • "ranges":指定价格范围的条件数组。
    • { "from": 0, "to": 200 }:表示价格从0到200之间的商品。
    • { "from": 200, "to": 500 }:表示价格从200到500之间的商品。
    • { "from": 500, "to": 1000 }:表示价格从500到1000之间的商品。
    • { "from": 1000 }:表示价格大于等于1000的商品。

5. 统计聚合(Stats Aggregation)

统计聚合可以对数值字段进行计算,包括最小值、最大值、平均值、总和和文档数量。

示例查询:

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"price_stats": {
"stats": {
"field": "price"
}
}
}
}

解释:

  • "aggs":定义聚合操作。
  • "price_stats":自定义的聚合名称,用于标识结果。
  • "stats":指定使用统计聚合。
  • "field": "price":指定要进行聚合的数值字段。

我们上面在统计聚合中可以获取很多值,那么我们也可以细化单独获取某一个的聚合结果。

6. 平均值聚合(Avg Aggregation)


GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"avg_price": {
"avg": {
"field": "price"
}
}
}
}

7. 总和聚合(Sum Aggregation)

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"total_price": {
"sum": {
"field": "price"
}
}
}
}

8. 最小值聚合(Min Aggregation)

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"min_price": {
"min": {
"field": "price"
}
}
}
}

9. 最大值聚合(Max Aggregation)

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"max_price": {
"max": {
"field": "price"
}
}
}
}

10. 扩展统计聚合(Extended Stats Aggregation)

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"price_stats_extended": {
"extended_stats": {
"field": "price"
}
}
}
}

11. 百分位数聚合(Percentiles Aggregation)

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"price_percentiles": {
"percentiles": {
"field": "price",
"percents": [25, 50, 75, 90]
}
}
}
}

12. 日期直方图聚合(Date Histogram Aggregation)

假设有一个名为timestamp的日期字段,我们可以进行日期直方图聚合,按照日期进行分组并统计每个时间段内的文档数量。

GET zfc-doc-000002/_search
{
"size": 0,
"aggs": {
"date_histogram_agg": {
"date_histogram": {
"field": "timestamp",
"fixed_interval": "1h"
}
}
}
}


原文始发于微信公众号(醉鱼Java):聚合在Elasticsearch中的使用及示例验证

版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。

文章由极客之音整理,本文链接:https://www.bmabk.com/index.php/post/231514.html

(0)
小半的头像小半

相关推荐

发表回复

登录后才能评论
极客之音——专业性很强的中文编程技术网站,欢迎收藏到浏览器,订阅我们!