redis replication问题一解

发表于 2015-04-11 更新于 2024-07-13 分类于樱桃沟本文字数： 3.3k 阅读时长 ≈ 3 分钟

公司有个redis比较大，同时又是跨IDC同步，但是最近发现一旦连接断了之后，好久都不能完全同步。查看了一下log。 c \[12826\] 01 Apr 16:54:37.675 # I/O error trying to sync with MASTER: connection lost \[12826\] 01 Apr 16:54:38.555 * Connecting to MASTER 10.x.x.x:6379 \[12826\] 01 Apr 16:54:38.555 * MASTER <-> SLAVE sync started \[12826\] 01 Apr 16:54:38.621 * Non blocking connect for SYNC fired the event. \[12826\] 01 Apr 16:54:38.692 * Master replied to PING, replication can continue... \[12826\] 01 Apr 16:54:45.229 * MASTER <-> SLAVE sync: receiving 390598473 bytes from master 通过info观察也是一直去master上同步，有时候看着马上就完成了，就报以上的错误。以前使用redis很老的版本2.4一直都没有这样的问题，自从换了2.6以后才出现这样的问题的。放狗查了下原来是 client-output-buffer-limit 这个参数导致的。默认是： ```c
client-output-buffer-limit slave 256mb 64mb 60

soft limit是一种依赖于时间的。 比如一个soft limit被设置为32MB 10s, 那就意味着当client的output buffer超过32MB，并且持续10秒钟，那这个连接就会被断开。 默认值就是hard 为256M， soft为 32M 60秒 普通客户端的默认limit为0, 就是任何时候都没有limit，因为普通的client使用阻塞来实现发送命令和接收完整的返回，在发送下一个命令之前，所以在普通的client的情况下关闭连接是不合适的。 但是要特别注意pub/sub客户端，这种方式一次处理和输出的数据都会特别大。 我们可以使用config set 来进行设置，但是注意使用config set的时候不支持MB，GB 这样的单位。   下面这个是redis官网的一个具体说明： [http://redis.io/topics/clients](http://redis.io/topics/clients "http://redis.io/topics/clients")

Output buffers limits
---------------------

Redis needs to handle a variable-length output buffer for every client, since a command can produce a big amount of data that needs to be transferred to the client. However it is possible that a client sends more commands producing more output to serve at a faster rate at which Redis can send the existing output to the client. This is especially true with Pub/Sub clients in case a client is not able to process new messages fast enough. Both the conditions will cause the client output buffer to grow and consume more and more memory. For this reason by default Redis sets limits to the output buffer size for different kind of clients. When the limit is reached the client connection is closed and the event logged in the Redis log file. There are two kind of limits Redis uses:

*   The **hard limit** is a fixed limit that when reached will make Redis closing the client connection as soon as possible.
*   The **soft limit** instead is a limit that depends on the time, for instance a soft limit of 32 megabytes per 10 seconds means that if the client has an output buffer bigger than 32 megabytes for, continuously, 10 seconds, the connection gets closed.

Different kind of clients have different default limits:

*   **Normal clients** have a default limit of 0, that means, no limit at all, because most normal clients use blocking implementations sending a single command and waiting for the reply to be completely read before sending the next command, so it is always not desirable to close the connection in case of a normal client.
*   **Pub/Sub clients** have a default hard limit of 32 megabytes and a soft limit of 8 megabytes per 60 seconds.
*   **Slaves** have a default hard limit of 256 megabytes and a soft limit of 64 megabyte per 60 second.

It is possible to change the limit at runtime using the [CONFIG SET](http://redis.io/commands/config-set) command or in a permanent way using the Redis configuration file `redis.conf`. See the example `redis.conf` in the Redis distribution for more information about how to set the limit.