在做最后一个MySQL NBU备份的时候，发现从库有问题，好奇的是怎么主从状态异常没有告警呢？先不管这么多了，处理了这个问题再完善告警内容。
一、错误信息
从库show slave status \G看到的错误信息如下：
Slave_IO_Running: No
Slave_SQL_Running: Yes
Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position; the first event 'mysql-bin.000081' at 480141113, the last event read from './mysql-bin.000081' at 4, the last byte read from './mysql-bin.000081' at 4.'
二、错误原因
这里看到从库的io_thread已经终止，错误编号是1236，具体是由于读取主库的binlog日志位置（the first event 'mysql-bin.000081' at 480141113, the last event read from './mysql-bin.000081' at 4）不对导致主从失败建立失败。
三、解决方案
1.检查从库状态以及读取、执行的binlog信息
mysql> show slave status \G
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: xx.xx.xx.xx
Master_User: username
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000081
Read_Master_Log_Pos: 480141113
Relay_Log_File: mysql9017-relay-bin.000163
Relay_Log_Pos: 480141259
Relay_Master_Log_File: mysql-bin.000081
Slave_IO_Running: No
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 480141113
Relay_Log_Space: 480141462
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Client requested master to start replication from impossible position; the first event 'mysql-bin.000081' at 480141113, the last event read from './mysql-bin.000081' at 4, the last byte read from './mysql-bin.000081' at 4.'
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 17
1 row in set (0.00 sec)
2.查看主库的binlog内容
[backup]# mysqlbinlog mysql-bin.000081 >mysql-bin.log
看到主库binlog日志mysql-bin.000081最大的pos为480140557，但从库要读取的是'mysql-bin.000081' at 480141113，显然从库要读的pos值比主库本身存在的pos值大，导致读取不到，进而失败。
可通过下面语句查看binlog的pos信息和日志内容
mysql> show binlog events in 'mysql-bin.000081' from 480140557 limit 10;
Empty set (0.04 sec)
3.更改从库的同步位置，完成数据重新同步
主库：
mysqlbinlog mysql-bin.000082 |more
从库：
change master to master_host='xx.xx.xx.xx',master_user='username',master_port=3306,master_password='password',master_log_file='mysql-bin.000082',master_log_pos=4;
start slave;
show slave status \G
主从同步正常
4.主库参数改进