Event id: 9667 – When database reaches maximum limit of named properties or replica identifiers
UPDATE 30/07/2010: See Enhancements in Exchange 2007 SP2 for Named Props Quota
So what is this Named Property anyways ???
Most objects for MAPI specification are represented as properties which are PropIDs.
Range of these PropIDs can vary from 1~65535 (0xFFFF). There are 3 ranges which differentiates a PropID and anything which falls within range 32768 (0x8000 or 8K)is called a standard property.
Now comes our hero, NAMED PROPERTY.
Classification:-
- Properties that have numbers for names.
- Properties that have string values for names.
Named properties are basically properties that are not well known. MAPI developers (as I told 3rd party vendors) can use them to store values that only pertain to them or in simple terms extend the standard MAPI property by adding their application involved properties..,for ex: blackberry,fax applications etc. The definition of this properties are stored in the named props table. This is where the problem starts, the table that stores these values only allows for 32k entries. So if let’s say you sent 32k messages to a company, each with a unique X- header or a single message with 32k X-headers, you could fill up this table, effectively causing the store to become inoperable. To provide a buffer, we set a soft limit of 16k, and further ratcheted it down to 8k. So if you do hit this limit, it provides an administrator the opportunity to take action.
This happens if a store has been running for a really long time, an MAPI application in the environment creating a lot of named properties, or have some malicious or strange message patterns coming into the Exchange Server.
The second use of the named props table is related to incoming SMTP messages. When mail arrives from the internet, it can contain any number of customer “X-“ headers and their associated value. The headers must be preserved in some fashion. When a message is fully converted from MIME to MAPI format, we must save these X- headers off somewhere inside the EDB file. Each unique X- header encountered becomes a separate entry in the named props table. For example if the following headers were encountered they would become 2 unique entries in the named props table. If they are encountered again in a later message, no big deal, we already have a definition for them.
X-MyCoolCompanySCLLevel
X-ULConvertState
Now the fix most do for exhaustion of named property is to increase the limit!!! Upto what? No more than 32768 as the table can accommodate only that. This is temporary fix and at some point in a down the line in future problem is going to hit again, you have no other option than to move all your users to a new database where that gives you a fresh table (32K) for named property entries. (big pain if lot of users exists in your environment)
Best way is to troubleshoot this in E2K3 is you can use MFCMapi to dump the named props table.
Help on MFCMapi below:
Run MFCMapi and then open your mailbox and run “Property Pane”/”Find All Named Props (SLOW)”, it should dump all
Now you have the job to sort which application (spam software, archiving, Disclaimer) does eat most of the property quota and delete them. Idea behind dumping is to look at the props and figure out what program they are for and fix it.
CopyLargeNamedPropertyToDebugOutput – Demonstrates how to read a large named MAPI property by using IStream
Best way is to troubleshoot this in E2K7 is codeplex (Not my finding, but have seen users with positive output)
How do you delete this unwanted or piled up named properties?
Download Codeplex and run it on the server which is responsible for receiving internet emails (gateway server).
Help on Codeplex below:
HeaderFilterAgent for Exchange 2007: http://www.codeplex.com/HeaderFilterAgent
Got bored ?? Lets see how to identify and resolve this issue !!
This event id appears when database reaches maximum limit of named properties or replica identifiers and should be treated with priority. Yes we have some more events 9666, 9668, and 9669… They all tell you nothing different !!!
Event ID : 9667
Raw Event ID: 9667
Record Nr. : 164428077
Category : General
Source : MSExchangeIS
Type : Error
Generated : 2/28/2009 11:55:04 AM
Written : 2/28/2009 11:55:04 AM
Machine : EXCHANGE2K3
Message : Failed to create a new named property for database “First Storage Group\County Mail” because the number of named properties reached the quota limit (8192). User attempting to create the named property: “SYSTEM” Named property GUID: 00020386-0000-0000-c000-000000000046 Named property name/id: “x-vitals”
Suggested Action:
1. Change the Registry value of Named Props Quota to 16384 by going to following location in registry
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSExchangeIS\<Server Name>\Private-00000000-0000-0000-0000-00000000
2. Increase the Registry value of NonMAPI Named Props Quota to 16384
3. Dismounted and remount the store
PFA Screenshot:
Path to follow in registry
I’ve done this but issue still persists!!!
Dont take a second opinion… get rid of the database before users start getting NDR’s
To split the database:
- Create 2 new mailbox stores on the same server or a different Exchange Server.
- Move 50 % of mailboxes from the database to the new mailbox store and the remaining ones to the second one.
- Make sure you right click on Mailboxes under Mailbox Store and RUN CLEANUP AGENT.
- Bottom line, we should not allow a particular mailbox store database to grow more than 50 – 60 GB : http://technet.microsoft.com/en-us/library/aa996891.aspx
######UPDATE######
As per:
Events 9666, 9667, 9668, and 9669 Received When Named Properties or Replica Identifiers Are Depleted for An Exchange Database: http://technet.microsoft.com/en-us/library/bb851495(EXCHG.80).aspx
To use Performance Monitor to monitor the rate at which the named properties and replica identifiers are created in your environment:
- Enable additional Microsoft Exchange Information Store service logging on the Exchange server that you need to monitor. For detailed steps about enabling additional Microsoft Exchange Information Store logging, see the Microsoft Knowledge Base article 254606, XADM: How to Enable Additional Information Store Logging.
- Use Performance Monitor to capture performance counter values. For more information about using Performance Monitor, see Monitoring server performance. Capture the values for the following performance counters:
- MSExchangeIS Mailbox\Rows in NamedProps Table
- MSExchangeIS Mailbox\Rows in ReplidMap Table
- MSExchangeIS Public\Rows in NamedProps Table
- MSExchangeIS Public\Rows in ReplidMap Table
Analyze the performance data you captured to identify trends and try to find a correlation between the increases in these counters over time with the activity on your network.
I’ve done this too !!! Im still getting these !!! help…..
I wish I knew the answer…. 🙁
You’ll have to consider opening a support incident with Microsoft PSS and they will dump the Prop Limit and advise further…
Ratish
September 10th, 2009 at 7:11 am
I did this and it worked. Thanks for the info. Was really helpful.
November 2nd, 2009 at 6:38 pm
Good stuff… but can you identify how to solve the ‘new stores’ issue in Exchange 2003 SP2 on SBS? (Apparently limits to one public and one provate store?)
March 5th, 2010 at 7:48 pm
Comeon Paul… Thats by Design for SBS Exchange STD…. go for a E2K3 Ent or jump to E2K7 STD/ENT!!! 🙂
April 18th, 2010 at 3:25 am
Paul,
SBS is ideally considered for small companies with few Users and the limit which named props has should be more than enough for a long long time to be consumed by the store
May 12th, 2010 at 11:20 am
Hi ratish.I salute your knowledge ! great job
Brian Mason
MS Exchange consultant
June 20th, 2010 at 8:36 am
A similar issue and a way to resolve it.
The official document can be found here for anyone who’s interested.
http://technet.microsoft.com/en-us/library/bb851493%28EXCHG.80%29.aspx
Hope this helps!!
June 20th, 2010 at 8:38 am
Hi DIFFMEISTER, Its the same registry I have mentioned in this article…
Thanks anyways….
June 26th, 2010 at 8:14 am
[…] followed the KB, but the errors continued. later I found this on MSExchangeGuru.com blog:https://msexchangeguru.com/2009/09/04/event-id-9667/. So far no more […]
July 30th, 2010 at 6:13 am
[…] Ratish post here has already detailed about Named Props Quota and the trouble it creates for the Admins in Exchange 2003 and 2007 and so, I am not going to dive into it now. But the good news is that Exchange 2007 SP1 RU7 and Exchange 2007 SP2 has got in some improvements, but at the same time, not with problems. […]
September 1st, 2010 at 12:10 pm
Hi just wanted to express my gratitude for all of your hard work keep it up. I bookmarked your websiteand will come back soon.
September 28th, 2010 at 2:54 am
First off, wonderful article. I am experiencing this problem with my environment and have been researching it for the past week. Recently MS released a hotfix, KB972077. Is this more of a fix than just increasing the quota? Using MFCMAPI, could I just delete named properties to free up some space and get rid of this error? I know the ultimate solution is upgrading from 2k3 std but I won’t be able to accomplish that until next summer. Thanks again for the wonderful article!
October 13th, 2010 at 12:12 pm
Great information and the best I have seen on named property. Thank you
October 22nd, 2010 at 12:43 am
MSExchangeguru team – Thank you so much for this information. My boss asked me to give me an explanation before increasing the registry and your blog helped me with that.
October 23rd, 2010 at 10:41 pm
Excellent article. Thank you Ratish
October 29th, 2010 at 5:30 am
[…] https://msexchangeguru.com/2009/09/04/event-id-9667/ http://msexchangeteam.com/archive/2009/04/06/451003.aspx […]
November 1st, 2010 at 8:03 am
Thanks for this excellent blog post. I am keen to see more on this topic in the future. Thanks again
December 14th, 2010 at 1:59 am
Excellent article. You are really a Guru!!!!!!!
Thank you.
December 20th, 2010 at 4:26 am
Thank you for explaining this. Really appreciated and the best I could find on this event out there.
-Kate
January 13th, 2011 at 12:00 pm
Can you explain why this error occurs only for 1-2 users receiving email from one particular outside person?
January 13th, 2011 at 12:17 pm
Well… I just want to ensure you are not getting confused 9667 with 9646… kb 830836 talks about 9646.
Assuming its 9667 you need an explanation for, all I can think of is that the 2 users in question are expecting emails from an “X” application which need to write its named property on the database to display the messages properly. Since the DB on which their mailbox reside has ran out of named props quota, it is throwing an event.
The best way to check this is to:
1. Create a new/blank database
2. Move these 2 users to that database and have the “outside person” send them an email.
3. 9667 shouldnt pop up.
If you dont have an option to create a new store, move them to a store which has a small db (assuming there is free named props quota available in it.)
You can use MFCMapi to dump the names props of the DB to see if it has exceeded…
Should work…
Ratish Nair
MVP Exchange
January 13th, 2011 at 12:56 pm
So, the 9667 error only occurs if the DB has reached the limit AND an incoming email includes an X-Header that is not already in the database? How come emails from that person going to other people inside our company, on the same storage database, go through?
Also, further testing showed that emails from this particular person are now being delivered to the same person that was getting it rejected before. Are there things an individual recipient can do to “free up space”?
January 21st, 2011 at 10:04 am
So if I dump out the table and empty it then will any message saved to the user with the problem in there outlook disappear?
Thanks,
RC
January 21st, 2011 at 10:05 am
Let me better ask my above question. The user that is throwing the event code in Event Viewer on Exchagne 2003 is using Outlook 2007 and have messages saved. If I dump out the table that has reached its quota…will they loose any message.
Thanks
rC
January 21st, 2011 at 2:01 pm
All we are doing is dumping the props quota using MFCMapi… you wont use any messages…
Again, how can a user throw this error? Remember that 9646 is different from 9667…
Send me an email with the event id and description and I’ll check to see if I can help….
The ONLY scenario in which I had to dump the DB props quota was once an SAP system processing automated emails was trying to add named props again and again “one a new database”(coz of some reason) and as soon as I identified it, asked them to work on their system and they fixed it somehow….
99% times, I move the user to another store and eventually re-create the database…. This has improved a lot in E2K7….
March 15th, 2011 at 10:01 am
I’m running a 2003 SBS server that’s getting Event ID 9667. How can I create a new database when SBS allows only one?
Thanks
Paul
March 15th, 2011 at 11:20 am
@Paul – Are you getting this event often? I suggest you open a case with MS PSS to see what/which application is consuming your database’s props id…
March 15th, 2011 at 12:11 pm
Hi Ratish,
Yes, it’s pretty much constant – always from one machine which just happens to be the last machine we’ve added and the only Windows 7 machine we have and the only machine running Office 2010.
She’s unable to reply to messages or add conacts.
I’m preparing to apply the patch from Microsoft KB 972077. Do you have any thoughts about whether that will help my issue?
Thanks,
Paul
March 17th, 2011 at 3:19 pm
Thanks all. one would think doubling the value to 16K with a mount/dismount would do the trick. Why then is KB972077 a 40MB download to change one registry value? Methinks there must be more undocumented features in this patch.
Apparently this is the cost of maintaining legacy hardware…users outgrow the default values of 4GB mailboxes and 8192K table entries.
March 17th, 2011 at 4:22 pm
FWIW the registry fix did it for me – the store is still under 20GB. The Exchange store default value was previously raised to 32GB. Is there a reason not to raise the x-header limit to 32K instead of 16K?
I did not run the KB972077 patch and unless there other reasons to do so I think I’ll pass on it.
March 31st, 2011 at 9:42 am
Great Found ! nice work, works for me running sbs.
Highly appreciated !
Thanks
Nain Agha
nagha@hilltopconsultants.com
Hilltop Consultants, Inc
2010 P st NW 20036
October 10th, 2011 at 4:48 am
Hi- What is the maximum count of email ids, that MAPI can support. Please advise asap.
March 20th, 2012 at 2:24 pm
The registry change did the job on my exchange 2003 server, thank you. There were no more 9667 events being logged after I changed the value for the named props in the registry.
March 21st, 2012 at 10:05 pm
Wow, amazing weblog structure! How long have you been blogging for? you made blogging glance easy. The entire glance of your website is fantastic, as well as the content! LE69 Escorts, 8, 50 Horseferry Rd, London, SW1P 2AF, 020 3011 2842
July 31st, 2012 at 3:07 am
youre the best. Now I got an idea on how to approach this issue.
December 14th, 2014 at 12:55 pm
I have exchange 2003 STD, increased named props quota to 16k, the vent 9667 doesn’t appear anymore, but how does it take to fill up again?
I guess this named props filled after we upgraded outlook from 2003 to 2007, someone above mentioned that, it just looks like it might be the one of things to be thought about. I don’t know… Outlook 2007 have some habit populating more named props because 2007 2010 have resolved this issue???
Cause of named props being filled:
1.
Also, recently I was getting lots of spams which populates named properties like;
Sushi:24324233223423
Tallis:23432432432423432
….
The spams randomly generates these named prop pairs and create 9667 events. Even thought the spams were filtered and moved to junk folders, the attempt to create named props are not gone away, which I think it’s because named props are created no matter the email is moved to Junk folder or not, which it makes sense because named props are on EDB database level..
2. I also see that my spam filer fills up named props as well, but I see most of them are repeated named props, but very frequent. However, as far as I understand reading this article, named props are only filled with named prop name, not string value of the named prop. So, I should not concern.
Question:
I downloaded MFCMapi and looked at named props, but it only examines a single mailbox, not mailbox store. If named props is a property of mailbox store database, why do we concern only one mailbox to analyze? Is there a way to visually look the table and what named properties are in there and total size of it in mailbox store database???
January 7th, 2016 at 4:45 pm
Thank you for sharing your knowledge. It worked after step #3 in suggestion was applied (Dismount and Mount DB) that I had missed trying to rush thru it.
March 21st, 2016 at 2:52 pm
In reply to han (dec 14, 2014), it took me about 2 yrs for the named props to get filled up again after i increased to 16k.