Thursday, August 23, 2007

InnoDB: Number of pending reads 128, pending pread calls 0

InnoDB: Number of pending reads 128, pending pread calls 0
InnoDB: Error: InnoDB has waited for 50 seconds for pending
InnoDB: reads to the buffer pool to be finished.


What does this mean? Well searching yahoo search I couldn't find anything other then pointers to source code where this message generated. I assumed it was a disk problem because performance was super crappy.

My disk monitoring software which probes MegaRaidCli didn't report a failed disk. Linux didn't report a failed disk or any errors. So, what is happening? Was I wrong?

Well, in fact it was a failed disk, but not in a state that could be classified as failed by the controller or any other component including the OS. With the help of my bud we found that a disk in the array was in a critical state-the disk itself just had a bunch of sector errors, yet the controller, os, everything would not mark it as dead - so the end result was to remove the drive and swap it with another.

So in essence this message means something is wrong with your disks and or filesystem.

3 comments:

Maxim Veksler said...

See how SMART can be useful to you http://www.linuxjournal.com/article/6983

Unknown said...

I found myself in a situation with this error after a user tried to query a table while it was being repaired and mysql crashed itself because of a semaphore timeout.

In an act of desperation I was able to clear this with a reboot of the server. I'd be curious to see if switching to single user mode remounting the /usr filesystem wold be sufficient.

Anonymous said...

I had the same thing happen, but restarting didn't work. I had to delete all my ibdata files and re-create them. This is easy, you just stop, delete, start. Of course, you lose all your data, but them's the breaks. Hopefully you've got backups.