Monday, April 21, 2008

Common Steps to Scale Linearly

Whenever I work at a place I do the following.

Get a rundown of what the application is, what its demands are, what does the company expect the application to be a year from now - like how many users are going to use the application. 10 million, 20 million, 100 million?

Then I find all the slowdowns:
- What are the my.cnf settings?
- What are the most active tables?
- What type of SQL is being used?
- How is the data accessed?
- Who/What owns the data?
- What is the Read-Write Ratio?
- How many servers are used now to handle the site load, and how many are needed within a few months.
- What is the reads per second, connections per second, writes per second
- How does the data grow? MxN, MxNxO, N^4 etc.

Once I get this down (a few days) then I change everything :)

If the data is small and doesn't change often I don't bother federating that at 1st. I go for the meat of the product. My goal is to run mean, lean, cheap, fast, and easy to maintain. I love my sleep.

So steps on federating:

What is the main object?
What are the mappings to this main object?
Spread data out by this main object
Cache lookups to the pointer where the main object data is.
Build everything around the main object(s).
Use a versioning system
Document a global view of how things work, and make cookbooks-so someone else can wake up in the middle of the night. I love my sleep :)


At the same time get

dsh working
nagios working
ganglia working
custom tools working



What you have is an easy to use, maintained system that scales linearly as long as that main object is being referenced.

Above is the easy stuff. The time-consuming part of the procedure is rewriting all the code to work for old and new, and migrate the new. This is needed to make sure no one if affected by upgrading to the new system.

3 comments:

mike said...

I love dsh. I don't know how long ago I discovered it but man. I use it daily now. Combined with ssh keys...

suhaib said...

Good points to start optimizing. I'll also check for column index, where clauses, tmp tables.

Mike can you point me to 'dsh' tool. I am not sure how to locate it.

Anonymous said...

A片,A片,成人網站,成人漫畫,色情,情色網,情色,AV,AV女優,成人影城,成人,色情A片,日本AV,免費成人影片,成人影片,SEX,免費A片,A片下載,免費A片下載,做愛,情色A片,色情影片,H漫,A漫,18成人

a片,色情影片,情色電影,a片,色情,情色網,情色,av,av女優,成人影城,成人,色情a片,日本av,免費成人影片,成人影片,情色a片,sex,免費a片,a片下載,免費a片下載

情趣用品,情趣用品,情趣,情趣,情趣用品,情趣用品,情趣,情趣,情趣用品,情趣用品,情趣,情趣

A片,A片,A片下載,做愛,成人電影,.18成人,日本A片,情色小說,情色電影,成人影城,自拍,情色論壇,成人論壇,情色貼圖,情色,免費A片,成人,成人網站,成人圖片,AV女優,成人光碟,色情,色情影片,免費A片下載,SEX,AV,色情網站,本土自拍,性愛,成人影片,情色文學,成人文章,成人圖片區,成人貼圖

情色視訊,美女視訊,辣妹視訊,視訊聊天室,視訊交友網,免費視訊聊天,視訊交友90739,視訊,免費視訊,情人視訊網,視訊辣妹,影音視訊聊天室,視訊交友,視訊聊天,免費視訊聊天室,成人視訊,UT聊天室,聊天室,豆豆聊天室,色情聊天室,尋夢園聊天室,聊天室尋夢園,080聊天室,080苗栗人聊天室,上班族聊天室,小高聊天室

6K聊天室,080中部人聊天室,聊天室交友,成人聊天室,中部人聊天室,情色聊天室,AV女優,AV,A片,情人薇珍妮,愛情公寓,情色,情色貼圖