基于 Python3 的 ES 压力测试命令行工具，功能丰富支持自动创建、运行、销毁 ES 集群，以及不同数据集的测试结果比较。
功能特性
-
用于基准测试的 Elasticsearch 集群的设置和拆卸 -
管理基准数据和规范，甚至跨 Elasticsearch 版本 -
运行基准测试并记录结果 -
通过附加所谓的遥测设备来发现性能问题 -
比较性能结果
快速入门
Rally 是为 Unix 开发的，并在 Linux 和 macOS 上进行了测试。Rally 支持对 Windows 上运行的 Elasticsearch 集群进行基准测试，但 Rally 本身需要安装在运行 Unix 的计算机上。
首次测试
esrally race --distribution-version=6.5.3 --track=geonames
这将下载 Elasticsearch 6.5.3 并对其运行 geonames 跟踪。
------------------------------------------------------
_______ __ _____
/ ____(_)___ ____ _/ / / ___/_________ ________
/ /_ / / __ / __ `/ / __ / ___/ __ / ___/ _
/ __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
/_/ /_/_/ /_/__,_/_/ /____/___/____/_/ ___/
------------------------------------------------------
| Lap | Metric | Task | Value | Unit |
|------:|----------------------------------------------------------------:|-----------------------:|----------:|--------:|
| All | Cumulative indexing time of primary shards | | 54.5878 | min |
| All | Min cumulative indexing time across primary shards | | 10.7519 | min |
| All | Median cumulative indexing time across primary shards | | 10.9219 | min |
| All | Max cumulative indexing time across primary shards | | 11.1754 | min |
| All | Cumulative indexing throttle time of primary shards | | 0 | min |
| All | Min cumulative indexing throttle time across primary shards | | 0 | min |
| All | Median cumulative indexing throttle time across primary shards | | 0 | min |
| All | Max cumulative indexing throttle time across primary shards | | 0 | min |
| All | Cumulative merge time of primary shards | | 20.4128 | min |
| All | Cumulative merge count of primary shards | | 136 | |
| All | Min cumulative merge time across primary shards | | 3.82548 | min |
| All | Median cumulative merge time across primary shards | | 4.1088 | min |
| All | Max cumulative merge time across primary shards | | 4.38148 | min |
| All | Cumulative merge throttle time of primary shards | | 1.17975 | min |
| All | Min cumulative merge throttle time across primary shards | | 0.1169 | min |
| All | Median cumulative merge throttle time across primary shards | | 0.26585 | min |
| All | Max cumulative merge throttle time across primary shards | | 0.291033 | min |
| All | Cumulative refresh time of primary shards | | 7.0317 | min |
| All | Cumulative refresh count of primary shards | | 420 | |
| All | Min cumulative refresh time across primary shards | | 1.37088 | min |
| All | Median cumulative refresh time across primary shards | | 1.4076 | min |
| All | Max cumulative refresh time across primary shards | | 1.43343 | min |
| All | Cumulative flush time of primary shards | | 0.599417 | min |
| All | Cumulative flush count of primary shards | | 10 | |
| All | Min cumulative flush time across primary shards | | 0.0946333 | min |
| All | Median cumulative flush time across primary shards | | 0.118767 | min |
| All | Max cumulative flush time across primary shards | | 0.14145 | min |
| All | Median CPU usage | | 284.4 | % |
| All | Total Young Gen GC time | | 12.868 | s |
| All | Total Young Gen GC count | | 17 | |
| All | Total Old Gen GC time | | 3.803 | s |
| All | Total Old Gen GC count | | 2 | |
| All | Store size | | 3.17241 | GB |
| All | Translog size | | 2.62736 | GB |
| All | Index size | | 5.79977 | GB |
| All | Total written | | 22.8536 | GB |
| All | Heap used for segments | | 18.8885 | MB |
| All | Heap used for doc values | | 0.0322647 | MB |
| All | Heap used for terms | | 17.7184 | MB |
| All | Heap used for norms | | 0.0723877 | MB |
| All | Heap used for points | | 0.277171 | MB |
| All | Heap used for stored fields | | 0.788307 | MB |
| All | Segment count | | 94 | |
| All | Min Throughput | index-append | 38089.5 | docs/s |
| All | Mean Throughput | index-append | 38325.2 | docs/s |
| All | Median Throughput | index-append | 38613.9 | docs/s |
| All | Max Throughput | index-append | 40693.3 | docs/s |
| All | 50th percentile latency | index-append | 803.417 | ms |
| All | 90th percentile latency | index-append | 1913.7 | ms |
| All | 99th percentile latency | index-append | 3591.23 | ms |
| All | 99.9th percentile latency | index-append | 6176.23 | ms |
| All | 100th percentile latency | index-append | 6642.97 | ms |
| All | 50th percentile service time | index-append | 803.417 | ms |
| All | 90th percentile service time | index-append | 1913.7 | ms |
| All | 99th percentile service time | index-append | 3591.23 | ms |
| All | 99.9th percentile service time | index-append | 6176.23 | ms |
| All | 100th percentile service time | index-append | 6642.97 | ms |
| All | error rate | index-append | 0 | % |
| All | ... | ... | ... | ... |
| All | ... | ... | ... | ... |
| All | Min Throughput | large_prohibited_terms | 2 | ops/s |
| All | Mean Throughput | large_prohibited_terms | 2 | ops/s |
| All | Median Throughput | large_prohibited_terms | 2 | ops/s |
| All | Max Throughput | large_prohibited_terms | 2 | ops/s |
| All | 50th percentile latency | large_prohibited_terms | 344.429 | ms |
| All | 90th percentile latency | large_prohibited_terms | 353.187 | ms |
| All | 99th percentile latency | large_prohibited_terms | 377.22 | ms |
| All | 100th percentile latency | large_prohibited_terms | 392.918 | ms |
| All | 50th percentile service time | large_prohibited_terms | 341.177 | ms |
| All | 90th percentile service time | large_prohibited_terms | 349.979 | ms |
| All | 99th percentile service time | large_prohibited_terms | 374.958 | ms |
| All | 100th percentile service time | large_prohibited_terms | 388.62 | ms |
| All | error rate | large_prohibited_terms | 0 | % |
----------------------------------
[INFO] SUCCESS (took 1862 seconds)
----------------------------------
使用 Docker 运行 Rally
使用 Docker 映像时不支持以下 Rally 功能：
-
分发负载测试驱动程序以从多台计算机应用负载。 -
使用除「benchmark-only. _」
快速入门
可以通过首先发出一个简单的命令来列出可用轨道来测试 Rally Docker 映像：
$ docker run elastic/rally list tracks
下一步假设 Elasticsearch 正在 es01:9200 运行 Rally Docker 映像的主机上运行并且可以从该主机访问。使用以下方式运行 nyc_taxis 轨道 test-mode：
$ docker run elastic/rally race --track=nyc_taxis --test-mode --pipeline=benchmark-only --target-hosts=es01:9200
配置
Docker 镜像附带了一个默认配置文件，位于/rally/.rally/rally.ini. 要自定义 Rally，可以创建自己的 Rallyrally.ini 并使用以下命令绑定安装它：
docker run -v /home/<myuser>/custom_rally.ini:/rally/.rally/rally.ini elastic/rally ...
扩展 Docker 镜像
还可以在现有镜像的基础上创建自己的自定义 Docker 镜像。下面的示例展示了如何开始：
FROM elastic/rally:2.9.0.dev0
COPY --chown=1000:0 rally.ini /rally/.rally/
然后，可以使用以下命令构建和测试图像：
docker build --tag=custom-rally .
docker run -ti custom-rally list tracks
高级主题
使用自定义参数源控制操作参数
考虑以下操作定义：
{
"name": "term",
"operation-type": "search",
"body": {
"query": {
"term": {
"body": "physician"
}
}
}
}
该查询是静态定义的，但如果想改变参数，例如还搜索“mechanic”或“nurse”，可以在 Python 中编写自己的“参数源”。首先，在操作定义中定义参数源的名称：
{
"name": "term",
"operation-type": "search",
"param-source": "my-custom-term-param-source"
"professions": ["mechanic", "physician", "nurse"]
}
Rally 识别参数源并 track.py 查找 track.json. 该文件包含参数源的实现：
import random
def random_profession(track, params, **kwargs):
# choose a suitable index: if there is only one defined for this track
# choose that one, but let the user always override index and type.
if len(track.indices) == 1:
default_index = track.indices[0].name
if len(track.indices[0].types) == 1:
default_type = track.indices[0].types[0].name
else:
default_type = None
else:
default_index = "_all"
default_type = None
index_name = params.get("index", default_index)
type_name = params.get("type", default_type)
# you must provide all parameters that the runner expects
return {
"body": {
"query": {
"term": {
"body": "%s" % random.choice(params["professions"])
}
}
},
"index": index_name,
"type": type_name,
"cache": params.get("cache", False)
}
def register(registry):
registry.register_param_source("my-custom-term-param-source", random_profession)
传送门
开源协议：Apache-2.0 license
开源地址：https://github.com/elastic/rally
项目合集：https://github.com/OpenTechCol/OpenTechCol
-END-
原文始发于微信公众号（开源技术专栏）：基于 Python3 的 ES 压力测试命令行工具
版权声明：本文内容由互联网用户自发贡献，该文观点仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 举报，一经查实，本站将立刻删除。
文章由极客之音整理，本文链接：https://www.bmabk.com/index.php/post/155095.html
