MSExchangeGuru.com

Learn Exchange the Guru way !!!

 

Exchange database repair – The eseutil /p process

Recently, a reader of the website came to me with a question- “You explained in detail on how to perform various operations to recover data from an exchange database “which is great”. But, what exactly repair (eseutil /p) processes do?    

Exchange Database Recovery – Using eseutil commands: http://msexchangeguru.com/2009/07/12/exchange-database-recovery-using-eseutil-commands/

Well first I thought of just answering his questions and then got a second thought on why not write an article with what I know about ESEUTIL and the repair process? So, here we go.

Believe me guys, if you are an Exchange pro, you got to spend some time reading an understanding this article.

ESEUTIL, ESEUTIL and ESEUTIL!!!

I interviewed a candidate recently, “What is eseutil” he started off by explaining the eseutil commands and different switches. Well, that aint my question… My question was, “What is eseutil”?

Well, my answer would be “ESEUTIL.exe or the Exchange Storage Engine Utility is an executable file residing in the bin directory which is used to perform various operations like recovery, repair, integrity check on an Exchange database while it is offline”.

I would like to stress on “while it is offline” because a database will be locked by the store.exe process when it is mounted/online. Therefore, to run eseutil on a database, the database should be taken offline. Eseutil can only be used on one database at a time.

The file’s which backs eseutil.exe to perform various operations are Ese.dll, jcb.dll, and Exchmem.dll. These files reside in the bin directory too. This means that if you need to run eseutil on a database all you need is these 4 files and it will work fine even on an XP machine.

The Repair process

Question: When do we end up having to repair an exchange database?

Answer: Your database is in a dirty shutdown state and the required log files are missing.

To get an overview about transaction log files and the process in which they commit to the database, refer:

The Operative synopsis of Exchange database and ESE: http://msexchangeguru.com/2011/02/26/ese/

When you database is in a “dirty shutdown”, it will look like this when you dump the database using the eseutil /mh command:


So, long story short à your database is missing some log files and there is no way we can create in manually. At this point, if you don’t have a backup, you need to repair the database to get it to a “clean shutdown” state so that you can mount it and retrieve data.

Repairing databases involves the following three stages, in this order:

########################################################################

Run Eseutil in /P mode to perform a database page-level and table-level repair.

Run Eseutil in /D mode to fully rebuild indexes and defragment the database. eseutil /d does is removes empty pages in the database file and rebuilds its indices

Run Isinteg only on the database to repair the database at the application level and fix the logical corruption of the database.

########################################################################

Before proceeding, look at the figure below:


I mentioned before that the exchange database structure is called “B-tree structure”. The B-tree structure consists of pages, rows, tables etc. The repair process could discards rows from tables or even entire tables based on how “bad shaped” the database is right now. After completing the ESE-level repairs, it is necessary to perform an application-level repair to correct problems that may now exist at the application level because of missing data. Use Isinteg or “Information store integrity checker” to perform this application-level analysis and repair on databases. The following example illustrates how the repair mode in Eseutil works.

For example, a table in the database stores messages for all mailboxes. A separate table is used for each user’s Inbox folder. Suppose that a message is lost when using Eseutil to repair the message table. Eseutil does not correlate the message with the reference to it in each Inbox folder because Eseutil does not have the information about the cross-table schema of the application. Isinteg is needed to compare the repaired message table with each Inbox to remove a lost message from the Inbox folder.

Eseutil looks at each Exchange database page and table and ensures consistency and integrity within each table. Isinteg repairs a database at the application level and ensures the integrity of the relationships between tables.

So, what did you understand?

Let’s study the figure together.

Fig1:

This shows the current state the database is in right now. You can see there are several corrupted pages and link broken between rows and tables.

Fig2:

This shows the state of the database after running the repair process (eseutil /p). As you can see, the process got rid of the corrupt pages, rows from tables or an entire table and filled the spaces with empty space called White spaces.

Fig3:

Here we ran eseutil /d, the offline defragmentation process to reduce the physical size of the database and fully rebuild indexes. This process removes empty pages (white spaces) in the database file and rebuilds its indices.

Fig4:

If you look at the figure you can see I have mentioned “Physical numbering” and “Logical numbering”. It is pretty self explanatory. When there is a 5MB JPEG image stored in a users inbox, it is not stored as a “5MB file inside the database”. Rather, this file is spilt and allocated spaces within the database and there are pointers to the location of the file. Well, did you ever have an issue where you opened an email message and entire body was filled with “garbled” images? Yeah… these pointers would have got misplaced somehow.

When the offline defrag process ran, it got rid of the white spaces in the database and full rebuild the indices. This leaved the database in a situation where 3 should point to 3, but 3 is now pointing to 7. Look at the figure closely one more time. I am talking about the Physical number” and “Logical number”. ISINTEG basically fixes this.

So now you know what happens when an exchange database is “repaired”. The process is lot time consuming and runs @ 4-5 GB/hr. Again, if you have proper backup in place you will never end up doing a repair of you exchange database.

Ratish Nair
MVP Exchange
Team@ MSExchangeGuru

Keywords: Exchange database repair, eseutil /p, what does repair on database do, Repairing exchange database, eseutil for exchange database

43 Responses to “Exchange database repair – The eseutil /p process”

  1. Tim Says:

    Seriously awesome man. I think Microsoft is paying you Lol

    Tim

  2. Manny Says:

    This is great. Thanks for taking the time to lay this out for us.

  3. Nandhitha Says:

    It is vry helpful… Thanks a lot

  4. Shaptoni Says:

    Brilliant

  5. Exchange Database Repair – eseutil /p « Rodrigo Rodrigues .:. www.andersonpatricio.org Says:

    […] Entenda como funciona e como usar aqui: http://msexchangeguru.com/2011/03/02/databse-repair/ […]

  6. Shyam Seegu Says:

    It is very Helpful…Thankyou.

  7. Deb Says:

    This is the type of explanation MS should be displaying in kb’s. People with experience might find easy to tackle these complicated process by reading kb’s, not all.
    Thanks Guru and team.

  8. Penn Says:

    I love your blog! It was a very nice read.. keep up the good work you are very talented!…

  9. Jagjit Says:

    I always wondered who it worked and you have explained it beautifully. Thanks mate. Good work

  10. Anand Kumar Deva Says:

    Great man…simply superb…

  11. Sivakumar Says:

    All the articles are being posted in this msexchangeguru websits are awesome and very much helpful to understand every nook and corner part of exchange domain . I was expecting this sort of technical explanation everywhere and finally I got it … Thanks for your great effort and time ..

  12. jeevan Says:

    thanks , its a very good article

  13. htreutler Says:

    Firstly: Thanks for this really useful explanation!!

    But one remark though – one will need esoxal.dll additionally to the 4 files mentioned above in the article to run eseutil on another machine than the exchange server. It’s also located in the …\exchsrvr\bin directory.

  14. Ran Says:

    Gud one..

  15. Sandeep78610 Says:

    Thanks for explaining important things to us.

  16. Lenin Says:

    Thanks a lot for this step by step helpful guide to solve the Dirty shutdown problem. Although I got a repaired database, I am experiencing some important data loss issue. Is there any way to prevent this data loss problem.

    I have a backup copy but it is 3 months old.

    Please help !

  17. Luis Bob Says:

    @Lenin,

    You are experiencing this data loss issue because of eseutil /p repair process. Actually, this command line utility removes the corrupt database pages ( As Ratish Nair has also mentioned in above post) during the repair process to get the database into a consistent state or “clean shutdown”.

    If you have a copy of offline database, then you may use a third party application like Stellar Phoenix Exchange server recovery software. I would suggest to try demo version which is free of cost to test the efficiency of the tool. You can download trial version from here : http://exchange.stellarservertools.com/

    Let me know if it works for you !

    Luis Bob

  18. Ratish Sekhar Says:

    I would agree Stellar Exchange recovery tool is the best in market…

  19. JoeC Says:

    Wondering if anyone has any input on a situation I had. I work for a small company with 60 Exchange users. The EDB file itself grew from 20GB in December, to 45GB. Well above the expected growth rate base on the previous year’s growth.

    On the 45GB DB, whitespace was reported at about 4GB, the mailboxes themselves totaled about 20GB. So there’s roughly 21GB in unexplained data in the EDB. We migrated all mailboxes to a new DB. After all mailboxes were removed, the whitespace was reported at 44GB or almost all available space within the DB. Ran eseutil /d and DB dropped to about 1GB in size.

    The new DB size we migrated mailboxes to is roughly 24GB, with maybe 200MB in whitespace, so pretty clean.

    My question is, do you have any idea what could have been taking up the extra 20 to 25GB space in the old DB prior to moving the mailboxes out? Indexes possibly?

    We played with cleaning out recoverableitems and got back about 1-2GB. There are no litigation holds, or odd policies retain deleted items, in place.

    Any input is appreciated.

  20. Ratish Sekhar Says:

    @JoeC

    Well – first of all good job that you are in the right direction.
    Before checking event 1221 to know the whitespace, you need to ensure online defrag is completing on the DB…
    Event series 700 shows you that…

    If you really need to know what consuming extra space on the database, you gotta pass it to MS but the question is – Why to take that much pain when you have already taken care of the issue… 🙂

  21. JoeC Says:

    Thanks for a quick response! It’s one of those things where people are asking me ‘why the growth?’ and ‘what was taking up all of that space?’ and ‘why?’. Sometimes fixing the problem is not enough…

    I do have event ID 700’s, however upon further investigation, it doesn’t appear that it was running against the mailbox database itself. It’s running against C:\…\Exchange Server\V14\TransportRoles\data\Queue\mail.que’ and ‘D:\…\GFI\MailEssentials\Quarantine\quarantine.edb’ (spam filter)

    I inherited this Exchange server, but I assume it was a pretty vanilla install…

  22. JoeC Says:

    I’ll add, on the properties of the DB itself, I do have ‘Enable background database maintenance (24 x 7 ESE scanning)’ enabled. Which I thought was the proper way to ensure that took place…

  23. Ronnie Says:

    Pretty! This was an extremely wonderful post. Thanks
    for providing this information.

  24. Exchange Database Recovery – Using eseutil commands « MSExchangeGuru.com Says:

    […] Exchange database repair – The eseutil /p process: http://msexchangeguru.com/2011/03/02/databse-re… […]

  25. Shivappa Hosamani Says:

    Thank you very much for providing these steps………

  26. nashim khan Says:

    You have explained it in very simple and easy to understand for every one.
    Please cover the SCC,SCR,LCR and CCR with DAIG 2010.

    Thanks
    Nashim Khan

  27. Antonio Alfau Says:

    I did all the step but when i try to do the ISInteg and confirme the database that appears offline i received the followinr error:

    Isinteg cannot initiate verification process. Please review the log file for more information.

    any Help?

  28. Manu Says:

    Very good presentation on ESEUTIL vs ISINTEG

  29. Adrian PK Says:

    I’ve followed first 2 steps eseutil /p and eseutil /d and got that damn database priv1.edb clear shutdown state. I must confess that I’ve managed to format before the partition that has had that .edb database. Well all good and well I’ve restaurated .edb with get data back and put it in it’s place. Eseutil /md showed me that was a dirty shutdown and I’ve spend cca 8 h fixing tha table and indexes wint swithces /p an /d. Although I’ve stopped all exchanges services inclusive IIS I didn’t manage to start step 3 command isinteg /fix so then I’ve tried to mount the database it returned me with hideous horror that same error : c 1041724.
    So do you think you can help me?!

  30. Adi Pk Says:

    Isinteg doesn’t functioned on my database. I coldn’t make it run. Isinteg “e:>program files\exchsrv\ pivr1.edb” doesn’t return any result – just automated help… I’m in a state of disperation, confused. Please help me. The other command worked and brought the database in clean sutdown but I couldn’t mount it because isinteg isn’t running. Help please.

  31. Adi Pk Says:

    Isinteg doesn’t functioned on my database. I coldn’t make it run. Isinteg /fix “e:>program files\exchsrv\ pivr1.edb” doesn’t return any result – just automated help… I’m in a state of disperation, confused. Please help me. The other command worked and brought the database in clean sutdown but I couldn’t mount it because isinteg isn’t running. Help please.

  32. Jack C. Leire Says:

    Thanks Ratish for putting the useful information about ESEUTIL tools. It’ll help the users to run the utility correctly and fix the Exchange database files. Recently, we’re also faced the database dirty shutdown problem which corrupted our database and hanged us from using any email in our inbox. Then, with the help of a third party program, we successfully rebuild and mount the database to MS Exchange server. here is the link of that tool: http://www.serversdatarecovery.com/exchange.html

    Although eseutil /p repair command is a useful inbuilt utility but always does not work accurately and gives the data less than our expectation. In such cases, when inbuilt utilities do not work, it is good to go with a third party program.

    Note: I would suggest to make backup of your databases regularly !

    Jack

  33. Aj Says:

    What a explaination.. Thanks a lot. Really appreaciate your help..

  34. Curtis Says:

    Thanks for sharing a valuable information. For performing such task user needs some technical knowledge and sufficient time. When i performing the search on Google found Many companies offering Exchange data repair software.
    Some company offering Free Demo version also. By the use of these tools you perform this task with few clicks. From there one such tool is here: http://www.undeletepcfiles.com/exchange-edb-recovery-tool.html

    I hope this tool would help you visitors.

    Thanks

  35. Akash Rajput Says:

    Its really helpful. First time i could clear my concept regarding database repair.

  36. Hidden Admirar Says:

    Seriously well explained.
    Perhaps you can add a link to a site which explains the switches..

  37. Geraldo Says:

    Great ƿost.

  38. Vijay Says:

    Simple and more clarity. Thanks Rathish

  39. Bob Says:

    I created a new database, now i want my old database logs to be replayed on new database using the othr database logs, can i achive this using /r?

  40. Prabhat Nigam Says:

    No it is not possible. There will be signature mismatch.

  41. KOBO Says:

    Seriously awesome man… this command was brilliant.!

  42. Ayman Says:

    Thank you for this great article

  43. Ryan O. Says:

    Awesome article. I came across a corrupt mailbox database for the first time for one of my clients this week, and my team did not do a very good job of helping me understand the repair process. This sheds a great deal of light on it, and helps me become much more knowledgeable. Thanks for sharing this great info.

Leave a Reply

ad

Categories

Archives