binlog2sql操作注意事项

  sre

5fe0ad80791208687.jpg_fo742.jpg

简介

github

binlog2sql用于解析操作sql、生成恢复sql
需要连上mysqld,不支持离线解析
mysqld需要binlog_format = row
参数 binlog_row_image 必须为FULL,暂不支持MINIMAL
操作的用户需要select, super/replication client, replication slave权限:
    * select:需要读取server端information_schema.COLUMNS表,获取表结构的元信息,拼接成可视化的sql语句
    * super/replication client:两个权限都可以,需要执行'SHOW MASTER STATUS', 获取server端的binlog列表
    * replication slave:通过BINLOG_DUMP协议获取binlog内容的权限

安装

yum install -y git python-pip
git clone https://github.com/danfengcao/binlog2sql.git && cd binlog2sql
pip install -r requirements.txt

使用


获取操作SQL

#不能直接操作活动binlog,需要拷到别处
python binlog2sql.py -h127.0.0.1 -P23306 -uroot -p'123456' -dtest_uc -t uc_user uc_app --start-file='mysql-bin.000002' --start-datetime='2018-11-21 09:50:00' --stop-datetime='2018-11-21 09:55:00'

输出:

#正确地记录了我改了jzsuperadmin12为jzsuperadmin1245
UPDATE `test_uc`.`uc_user` SET `native_code`=NULL, `home_address`=NULL, `app_name`='rrt_jinzhou', `creator`='jzsuperadmin', `user_password`='c7a8a670aebd891ade59f5d2b58f463e710be878579245962fed625f', `user_photo_size`=NULL, `pinyin_name`=NULL, `modificator`='2390000229000000001', `political_state`=NULL, `del_flag`='0', `create_time`='2018-01-31 14:27:12', `im`=NULL, `used_name`=NULL, `id_card_no`=NULL, `birth_place_code`=NULL, `reserved1`=0, `id`=2390000229000000001, `valid_id_card`=NULL, `reserved2`=NULL, `user_status`='0', `im_type`=NULL, `position`=NULL, `home_page`=NULL, `census_register`=NULL, `forbidden`='0', `user_name`='锦州超管', `email`=NULL, `id_card_type`=NULL, `update_time`='2018-10-19 08:46:46', `is_binded`='0', `forbidden_reason`=NULL, `nation`=NULL, `phone`=NULL, `user_code`=NULL, `live_addr`=NULL, `nationality`=NULL, `census_nature`=NULL, `user_photo`=NULL, `census_register_code`=NULL, `remark`=NULL, `postal_addr`=NULL, `mobile`=NULL, `gender`=NULL, `hong_kong_kiu`=NULL, `native_place`=NULL, `birth_date`=NULL, `birth_place`=NULL, `zip_code`=NULL, `login_name`='jzsuperadmin1245' WHERE `native_code` IS NULL AND `home_address` IS NULL AND `app_name`='rrt_jinzhou' AND `creator`='jzsuperadmin' AND `user_password`='c7a8a670aebd891ade59f5d2b58f463e710be878579245962fed625f' AND `user_photo_size` IS NULL AND `pinyin_name` IS NULL AND `modificator`='2390000229000000001' AND `political_state` IS NULL AND `del_flag`='0' AND `create_time`='2018-01-31 14:27:12' AND `im` IS NULL AND `used_name` IS NULL AND `id_card_no` IS NULL AND `birth_place_code` IS NULL AND `reserved1`=0 AND `id`=2390000229000000001 AND `valid_id_card` IS NULL AND `reserved2` IS NULL AND `user_status`='0' AND `im_type` IS NULL AND `position` IS NULL AND `home_page` IS NULL AND `census_register` IS NULL AND `forbidden`='0' AND `user_name`='锦州超管' AND `email` IS NULL AND `id_card_type` IS NULL AND `update_time`='2018-10-19 08:46:46' AND `is_binded`='0' AND `forbidden_reason` IS NULL AND `nation` IS NULL AND `phone` IS NULL AND `user_code` IS NULL AND `live_addr` IS NULL AND `nationality` IS NULL AND `census_nature` IS NULL AND `user_photo` IS NULL AND `census_register_code` IS NULL AND `remark` IS NULL AND `postal_addr` IS NULL AND `mobile` IS NULL AND `gender` IS NULL AND `hong_kong_kiu` IS NULL AND `native_place` IS NULL AND `birth_date` IS NULL AND `birth_place` IS NULL AND `zip_code` IS NULL AND `login_name`='jzsuperadmin12' LIMIT 1; #start 568852516 end 568853181 time 2018-11-21 09:53:41

获取回滚SQL

#加了参数--flashback
python binlog2sql.py --flashback -h127.0.0.1 -P23306 -uroot -p'123456' -dtest_uc -t uc_user uc_app --start-file='mysql-bin.000002' --start-datetime='2018-11-21 09:50:00' --stop-datetime='2018-11-21 09:55:00'

输出:

#生成回滚sql非常慢,请不要终止
#能恢复jzsuperadmin1245为jzsuperadmin12
UPDATE `test_uc`.`uc_user` SET `native_code`=NULL, `home_address`=NULL, `app_name`='rrt_jinzhou', `creator`='jzsuperadmin', `user_password`='c7a8a670aebd891ade59f5d2b58f463e710be878579245962fed625f', `user_photo_size`=NULL, `pinyin_name`=NULL, `modificator`='2390000229000000001', `political_state`=NULL, `del_flag`='0', `create_time`='2018-01-31 14:27:12', `im`=NULL, `used_name`=NULL, `id_card_no`=NULL, `birth_place_code`=NULL, `reserved1`=0, `id`=2390000229000000001, `valid_id_card`=NULL, `reserved2`=NULL, `user_status`='0', `im_type`=NULL, `position`=NULL, `home_page`=NULL, `census_register`=NULL, `forbidden`='0', `user_name`='锦州超管', `email`=NULL, `id_card_type`=NULL, `update_time`='2018-10-19 08:46:46', `is_binded`='0', `forbidden_reason`=NULL, `nation`=NULL, `phone`=NULL, `user_code`=NULL, `live_addr`=NULL, `nationality`=NULL, `census_nature`=NULL, `user_photo`=NULL, `census_register_code`=NULL, `remark`=NULL, `postal_addr`=NULL, `mobile`=NULL, `gender`=NULL, `hong_kong_kiu`=NULL, `native_place`=NULL, `birth_date`=NULL, `birth_place`=NULL, `zip_code`=NULL, `login_name`='jzsuperadmin12' WHERE `native_code` IS NULL AND `home_address` IS NULL AND `app_name`='rrt_jinzhou' AND `creator`='jzsuperadmin' AND `user_password`='c7a8a670aebd891ade59f5d2b58f463e710be878579245962fed625f' AND `user_photo_size` IS NULL AND `pinyin_name` IS NULL AND `modificator`='2390000229000000001' AND `political_state` IS NULL AND `del_flag`='0' AND `create_time`='2018-01-31 14:27:12' AND `im` IS NULL AND `used_name` IS NULL AND `id_card_no` IS NULL AND `birth_place_code` IS NULL AND `reserved1`=0 AND `id`=2390000229000000001 AND `valid_id_card` IS NULL AND `reserved2` IS NULL AND `user_status`='0' AND `im_type` IS NULL AND `position` IS NULL AND `home_page` IS NULL AND `census_register` IS NULL AND `forbidden`='0' AND `user_name`='锦州超管' AND `email` IS NULL AND `id_card_type` IS NULL AND `update_time`='2018-10-19 08:46:46' AND `is_binded`='0' AND `forbidden_reason` IS NULL AND `nation` IS NULL AND `phone` IS NULL AND `user_code` IS NULL AND `live_addr` IS NULL AND `nationality` IS NULL AND `census_nature` IS NULL AND `user_photo` IS NULL AND `census_register_code` IS NULL AND `remark` IS NULL AND `postal_addr` IS NULL AND `mobile` IS NULL AND `gender` IS NULL AND `hong_kong_kiu` IS NULL AND `native_place` IS NULL AND `birth_date` IS NULL AND `birth_place` IS NULL AND `zip_code` IS NULL AND `login_name`='jzsuperadmin1245' LIMIT 1; #start 568852516 end 568853181 time 2018-11-21 09:53:41
#恢复sql执行未报错

语法说明


mysql连接配置

-h host; -P port; -u user; -p password

解析模式

–stop-never 持续解析binlog。可选。默认False,同步至执行命令时最新的binlog位置。

-K, –no-primary-key 对INSERT语句去除主键。可选。默认False

-B, –flashback 生成回滚SQL,可解析大文件,不受内存限制。可选。默认False。与stop-never或no-primary-key不能同时添加。

–back-interval -B模式下,每打印一千行回滚SQL,加一句SLEEP多少秒,如不想加SLEEP,请设为0。可选。默认1.0。

解析范围控制

–start-file 起始解析文件,只需文件名,无需全路径 。必须。

–start-position/–start-pos 起始解析位置。可选。默认为start-file的起始位置。

–stop-file/–end-file 终止解析文件。可选。默认为start-file同一个文件。若解析模式为stop-never,此选项失效。

–stop-position/–end-pos 终止解析位置。可选。默认为stop-file的最末位置;若解析模式为stop-never,此选项失效。

–start-datetime 起始解析时间,格式’%Y-%m-%d %H:%M:%S’。可选。默认不过滤。

–stop-datetime 终止解析时间,格式’%Y-%m-%d %H:%M:%S’。可选。默认不过滤。

对象过滤

-d, –databases 只解析目标db的sql,多个库用空格隔开,如-d db1 db2。可选。默认为空。

-t, –tables 只解析目标table的sql,多张表用空格隔开,如-t tbl1 tbl2。可选。默认为空。

–only-dml 只解析dml,忽略ddl。可选。默认False。

–sql-type 只解析指定类型,支持INSERT, UPDATE, DELETE。多个类型用空格隔开,如–sql-type INSERT DELETE。可选。默认为增删改都解析。用了此参数但没填任何类型,则三者都不解析。

LEAVE A COMMENT

Captcha Code