开源压测工具Locust

locust是流行的开源性能测试工具,其压测脚本就是python代码,所以特点是简单,灵活.

之前使用的场合比较多,汇总些笔记供参考.

安装及依赖

pip install locust

注: 可以在Windows做开发调试,不建议做实际运行,性能较差

Locust主要基于以下库实现:

  • gevent/greenlet: 轻量级异步线程
  • msgpack-python: master和slave通讯的一种高效消息格式
  • flask/jinja2/werkzeug: web界面的实现

Windows安装

  1. 安装gevent错误修复：

修复Python库 Windows安装错误:error Microsoft Visual C++ 14.0 is required 安装Microsoft Build Tools for Visual Studio

下载vs_buildtools.exe并选择Workloads → C++ build tools -> Windows 10 SDK安装。

  1. 启动时web页面打不开问题解决：需指定–web-host参数

locust –web-host=127.0.0.1

使用

默认的压测脚本为本地目录的locustfile.py

具体的用法见locust -h

locust -H <my_url> -f <my_locust_file> -c 120 -n 10000 --no-web --only-summary

Web管理页面方式

执行完命令后,访问本地或Master的IP地址, http://127.0.0.1:8089 就可以看到web控制台了.

常用选项

示例：

locust --tags 'tag1'

执行相关:

 -t RUN_TIME, --run-time RUN_TIME
     Stop after the specified amount of time, e.g. (300s, 20m, 3h, 1h30m, etc.). Only used together with --headless or --autostart. Defaults to run forever.

--autoquit AUTOQUIT   
    Quits Locust entirely, X seconds after the run is finished. Only used together with --autostart. The default is to keep Locust running until you shut it down using CTRL+C

任务过滤:

Tag options:
  Locust tasks can be tagged using the @tag decorator. These options let specify which tasks to include or exclude during a test.

  -T [TAG [TAG ...]], --tags [TAG [TAG ...]]
                        List of tags to include in the test, so only tasks with any matching tags will be executed
  -E [TAG [TAG ...]], --exclude-tags [TAG [TAG ...]]
                        List of tags to exclude from the test, so only tasks with no matching tags will be executed


User classes:
  UserClass             Optionally specify which User classes that should be used (available User classes can be listed with -l or --list)

日志:

  --loglevel LOGLEVEL, -L LOGLEVEL
                        Choose between DEBUG/INFO/WARNING/ERROR/CRITICAL. Default is INFO.

命令行执行

也支持不使用web页面, 需要指定几个参数值

locust --no-web -c 10 -r 5
# -c == --clients
# -r == --hatch-rate

压测脚本开发

基本功能

# 记录日志，默认INFO
import logging
# 常用模块和类
from locust import HttpUser, TaskSet, task, between

class UserTasks(TaskSet):
    def on_start(self):
        self.client.header = {'Content-Type':'application/json'}
        
    def on_stop(self):
        pass
    
    @task(1)  # 比例/权重
    def get_api(self):
        self.client.get("/api/get")
    
    # 支持捕获异常并自定义失败
    @task
    def post_api(self):
                with self.client.post("/api/post",json={"mykey": "myvalue"},headers=self.client.header, verify=False, catch_response=True) as response:
                    logging.debug(response.text)
                    if not response.headers['errorCode'] == '0':
                        response.failure(response.headers['errorMessage'])

class WebsiteUser(HttpUser):
    host = "https://127.0.0.1:8000"
    wait_time = between(1, 3)  # 随机延迟
    tasks = [UserTasks]

增加集合点

from gevent.lock import Semaphore
all_users_spawned = Semaphore()
all_users_spawned.acquire()

@events.init.add_listener
def _(environment, **kw):
    @environment.events.spawning_complete.add_listener
    def on_spawning_complete(**kw):
        # 释放锁
        all_users_spawned.release()


class UserTasks(TaskSet):
    def on_start(self):
        all_users_spawned.wait()

使用预生成数据

# 获取随机
global USERS
USERS = get_user_list()  # 自定义实现

class UserTasks(TaskSet):
    def on_start(self):
        self.client.user = USERS.pop()  # 选取某模拟用户

禁用 InsecureRequestWarning: Unverified HTTPS request is being made to host

import urllib3
urllib3.disable_warnings()

高级用法

使用固定的RPS

from locust import constant_pacing

class WebsiteUser(HttpUser):
    wait_time = constant_pacing(1)  # fixed RPS rate

集成prometheus

主要步骤：

  1. 配置prometheus/prometheus.yml，之后启动
scrape_configs:
  - job_name: locust
    metrics_path: '/export/prometheus'
    static_configs:
      - targets: ['127.0.0.1:8089']  # 地址修改为实际地址

  1. locust启动后(参考我的fork示例)，访问exporter URL做简单验证: http://127.0.0.1:8089/export/prometheus

  2. 访问http://127.0.0.1:9090/，在Expression输入 locust_stats_current_rps并点击Execute，就可以图形报告了。

  3. 如果需要Grafana展现的话，配置数据源之后导入Dashboard ID: 12081就可以了。

Go扩展 Boomer

以examples/main.go为例

go get -u github.com/myzhan/boomer@master
# 注：默认master端口为5557，不要修改

go build main.go


# 调试，只运行一次
./main --run-tasks foo,bar  # 或者 go run --run-tasks foo,bar
# 其他选项
--master-host=127.0.0.1   # 默认为127.0.0.1
--max-rps=0
--request-increase-rate

// 限制最大RPS
ratelimiter := boomer.NewStableRateLimiter(100, time.Second)
log.Println("the max rps is limited to 100/s.")
// 限制RampUp生成速度
ratelimiter, _ := boomer.NewRampUpRateLimiter(1000, "100/1s", time.Second)
log.Println("the max rps is limited to 1000/s, with a rampup rate 100/1s.")

globalBoomer.SetRateLimiter(ratelimiter)
globalBoomer.Run(task1)

扩展阅读

官方必读

其他