Friday, September 22, 2006

Master Master Replication Ring Recovery

In an Active Active Master Master setup there is a special procedure to do when recovering a down server.

Assumptions:

  • Your application can handle 1 of the servers being down.

  • You’re a good DBA so you take hot backups at least once a night.

  • You never PURGE MASTER LOGS to a point where a backup can't recover.

  • I'm assuming that you already know how to set up a master master config. If not comment and I will post how to, and things to watch out for.



Recover from backup, lets assume your using ibbackup


%> ibbackup --apply-log --use-memory 2000 my.cnf.restore


my.cnf.restore contains the innodb definitions and the location of the backup files.

Once restored INNODB will actually print out the last slave position it was at.

Start the mySQL instance

Perform a change master to, to that last slave position.
VERY IMPORTANT DO NOT and I REPEAT DO NOT set the Active master's replication on the 1st position of the recovered box.

Why?

If you’re using an Active Active, Master Master set up then that other box will log slave updates to its binary log and send all events back to the master that is still up. Additionally mySQL will not append the server-id of the last master to process the event, so there is a way to produce infinite loops of SQL statements.

Below is how to do so.


#060922 9:29:31 server id 14221485 log_pos 263789 Query thread_id=101364349 exec_time=0 error_code=0


Above is a dump of some header information that is put into a binary log tracking the server id of the original event. Now let’s say the server with the server id above died, and you brought up a new server with a new server id. If this event is put into a replication ring, prior to all server-id 14221485 finishing on the new Master, then ALL these events are sent back to the master that did not go down, the active Master. Then the active master sends this same event with the same server-id of the original event back to the passive master producing an infinite loop.

Really mySQL should append all server ids that processed the event to this header to prevent loops. But, this does not exist so... just follow the procedure below and you don't have to worry about it.
The procedure

  1. Recover from backup

  2. Turn the recovered box into a slave of the master that is still up

  3. Let it catch up

  4. Once catched up on the passive active master, FLUSH MASTER LOGS, then PURGE MASTER LOGS TO the new file

  5. On the active master CHANGE MASTER TO the new MASTER HOST, and the NEW MASTER Log, and the NEW MASTER Position

  6. Perform an event that will replicate on the active master, make sure the event took and is logged to the passive masters binary log.

  7. Perform an event that will replicate on the passive master, make sure the event took and is logged in the active masters binary log.

  8. Put the new MASTER in production.

3 comments:

Anonymous said...

Hi,
Please post the steps for doing a Master-Master replication. And the precaution to be taken so that a recovery can be done, when one of the DB fails critically.

PS: The link is broken

http://www.testingtheweb.com/archives/1001

Thanks

sexy said...

情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣,情趣,情趣,情趣,情趣,情趣,情趣,情趣,按摩棒,跳蛋,充氣娃娃,情境坊歡愉用品,情趣用品,情人節禮物,情惑用品性易購

免費A片,AV女優,美女視訊,情色交友,免費AV,色情網站,辣妹視訊,美女交友,色情影片,成人影片,成人網站,A片,H漫,18成人,成人圖片,成人漫畫,情色網,日本A片,免費A片下載,性愛

A片,色情,成人,做愛,情色文學,A片下載,色情遊戲,色情影片,色情聊天室,情色電影,免費視訊,免費視訊聊天,免費視訊聊天室,一葉情貼圖片區,情色,情色視訊,免費成人影片,視訊交友,視訊聊天,視訊聊天室,言情小說,愛情小說,AIO,AV片,A漫,av dvd,聊天室,自拍,情色論壇,視訊美女,AV成人網,色情A片,SEX

情趣用品,A片,免費A片,AV女優,美女視訊,情色交友,色情網站,免費AV,辣妹視訊,美女交友,色情影片,成人網站,H漫,18成人,成人圖片,成人漫畫,成人影片,情色網


情趣用品,A片,免費A片,日本A片,A片下載,線上A片,成人電影,嘟嘟成人網,成人,成人貼圖,成人交友,成人圖片,18成人,成人小說,成人圖片區,微風成人區,成人文章,成人影城,情色,情色貼圖,色情聊天室,情色視訊,情色文學,色情小說,情色小說,臺灣情色網,色情,情色電影,色情遊戲,嘟嘟情人色網,麗的色遊戲,情色論壇,色情網站,一葉情貼圖片區,做愛,性愛,美女視訊,辣妹視訊,視訊聊天室,視訊交友網,免費視訊聊天,美女交友,做愛影片

av,情趣用品,a片,成人電影,微風成人,嘟嘟成人網,成人,成人貼圖,成人交友,成人圖片,18成人,成人小說,成人圖片區,成人文章,成人影城,愛情公寓,情色,情色貼圖,色情聊天室,情色視訊,情色文學,色情小說,情色小說,色情,寄情築園小遊戲,情色電影,aio,av女優,AV,免費A片,日本a片,美女視訊,辣妹視訊,聊天室,美女交友,成人光碟

情趣用品.A片,情色,情色貼圖,色情聊天室,情色視訊,情色文學,色情小說,情色小說,色情,寄情築園小遊戲,情色電影,色情遊戲,色情網站,聊天室,ut聊天室,豆豆聊天室,美女視訊,辣妹視訊,視訊聊天室,視訊交友網,免費視訊聊天,免費A片,日本a片,a片下載,線上a片,av女優,av,成人電影,成人,成人貼圖,成人交友,成人圖片,18成人,成人小說,成人圖片區,成人文章,成人影城,成人網站,自拍,尋夢園聊天室

sex said...

徵信社
情趣用品
情趣用品
情趣用品
情趣
情趣


SM
充氣娃娃


SM
性感睡衣


免費視訊聊天室
aio交友愛情館
愛情公寓
情色貼圖
情色文學
情色小說
情色電影
情色論壇
成人論壇
辣妹視訊
視訊聊天室
情色視訊
免費視訊

免費視訊聊天
視訊交友網
視訊聊天室
視訊美女
視訊交友
ut聊天室
聊天室
豆豆聊天室
尋夢園聊天室
聊天室尋夢園
080聊天室
080苗栗人聊天室
女同志聊天室

上班族聊天室
小高聊天室






免費視訊聊天
免費視訊聊天室
免費視訊
ut聊天室
聊天室
豆豆聊天室 聊天室尋夢園
影音視訊聊天室


色情遊戲
寄情築園小遊戲
情人視訊網
辣妹視訊
情色交友

成人論壇
情色論壇
愛情公寓
情色
色情聊天室
色情小說
做愛
做愛影片
性愛


一葉情貼圖片區
情趣用品


辣妹視訊
美女視訊
視訊交友網
視訊聊天室
視訊交友
視訊美女