Linux调优

Linux调优

Linux作为服务器针对使用场景, 有许多需要调优的地方, 本文记录常用优化项.

性能基本调优

通常涉及到/etc/sysctl.conf和/etc/security/limits.conf配置文件的修改.
也可使用命令修改,使用sysctl -p 立即生效.

内核参数设置

sysctl命令可以用来实时的读取/修改内核参数

# 显示所有可用内核参数
sysctl -a

# 加载/etc/sysctl.conf的参数
sysctl -p

提高文件描述符限制

soft limit类似于warning, hard limit是真实的最大值限制.
默认的1024偏小, 有两种方式修改:

临时设置

# 查看当前值
ulimit -n
# 临时增加
ulimit -n 65535
# 单个进程的限制为soft limit
# hard limit应小于当前系统打开的文件描述符
# 相应增加nr_open
echo 2000000 > /proc/sys/fs/nr_open
# 系统级限制, 上限为nr_open
echo 1000000 > /proc/sys/fs/file-max

永久设置

修改 /etc/security/limits.conf,重启以生效.

*               soft    nofile           65535
*               hard    nofile           65535
root            soft    nofile           65535
root            hard    nofile           65535
# 系统级内核句柄限制
fs.file-max = 1000000

查看当前描述符情况

# 列出打开/占用的文件描述符
cat /proc/sys/fs/file-nr
# 三个值分别代表 占用/未使用/最大可用值
# 注: lsof只会列出进程占用
lsof | wc -l
# 要得到线程占用,需要使用
ps -eLf

# 查看某进程限制
cat /proc/[Process ID]/limits

增加可用端口数

默认28000, net.ipv4.ip_local_port_range
如果Nginx作代理,需要增加端口范围,否则会出现错误: Cannot assign requested address .
IPv4端口可用数:端口号是16位无符号整数,即65535

# 实时生效
echo 12000 64000 > /proc/sys/net/ipv4/ip_local_port_range
# 永久生效
sysctl -w net.ipv4.ip_local_port_range="12000 64000"

可选:最大线程数

一般不需要设置

# 默认31299
echo 100000 > /proc/sys/kernel/threads-max

查看当前线程数:
* top, then hit H to view threads
* top -H
* htop

网络调优

通用网络参数

/etc/sysctl.conf
# 系统网络设置
# 生效值取系统和下面TCP设置值的最大值
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216

TCP/IP调优

Backlog Queue

最大连接数队列. 可选, 查看kernel日志决定是否需要调整

net.core.somaxconn
net.core.netdev_max_backlog = 300000

网络缓冲区大小

# TCP读取缓冲区
# 格式: 最小值/默认值/最大值 字节数
# cat /proc/sys/net/ipv4/tcp_rmem
net.ipv4.tcp_rmem = 4096 87380 16777216
# 发送缓冲区
net.ipv4.tcp_wmem = 4096 65536 16777216
# TCP内存, 对应 low/pressure/high 页大小(4K)
net.ipv4.tcp_mem = 786432 2097152 3145728

UDP调优

默认比较受限

#改成8M
sysctl -w net.core.rmem_max=8388608

主要参数

Receive-Side Scaling (RSS)

also known as multi-queue receive, distributes network receive processing across several hardware-based receive queues, allowing inbound network traffic to be processed by multiple CPUs.

cat /sys/class/net/eth1/queues/<rx-0>/
ethtool --show-rxfh-indir eth1

CLOSE_WAIT 和 TIME_WAIT 解释

TCP是全双工的,任何一端可以是source或destination.
Due to the way TCP/IP works, connections can not be closed immediately. Packets may arrive out of order or be retransmitted after the connection has been closed.
CLOSE_WAIT indicates that the remote endpoint (other side of the connection) has closed the connection.
TIME_WAIT indicates that local endpoint (this side) has closed the connection. The connection is being kept around so that any delayed packets can be matched to the connection and handled appropriately.
The connections will be removed when they time out within four minutes.

tcp_tw_reuse和tcp_tw_recycle

不用开启 net.ipv4.tcp_tw_recycle, 最新内核4.12已结去掉该参数.
连接有incoming和outgoing之分,tcp_tw_reuse仅仅对outgoing有效.
设计协议时,尽量不用让客户端先关闭连接,应该让服务端控制.

TCP/UDP参数

  • Socket receive buffer size: Socket send and receive sizes are dynamically adjusted, so they rarely need to be manually edited.
  • rmem_default : A kernel parameter that controls the default size of receive buffers used by sockets.

调优常用指标

  • Ping 100以下
  • 网络延迟50ms以下
  • Dns解析尽量快
  • 尽量少丢包
  • 反向代理优化

调优辅助工具

perf-tools

开源的性能分析工具,基于perf和ftrace.

SystemTap

扩展阅读

发表评论

电子邮件地址不会被公开。 必填项已用*标注