Event ID 629 and CRM Outlook Client-Collection of common programming errors

So after creating a new database and moving all users my Event ID 629’s are back, at least this time I have an idea of the cause. We have a MS CRM  V4 server here and our Outlook 2007 users have the CRM Outlook Plugin Installed. Ocasionally when our CRM Admin publishes changes to CRM, it hangs up. The fix for CRM is to do an IISReset. Once this is done CRM is fine. However, I noticed my log drive on the Exchange 2010 SP1 server begins to fill rapidly. I then ran EXmon to see who was hitting the server. I find one user who has two sessons, one for Outlook, one for the CRM client Plugin. I end task his Outlook.exe and the log growth stops. Of course this was after it dropped 5GB of junk into the database. The next day I start getting the 629’s in my app log. Anyone know what the CRM client is dropping in the database that can cause logs to grow that fast? I have no idea. I cant keep moving users to a new database every time this happens. What are Unversioned Deletes? Is there some retension period I can change to purge these? Or is there some way to determine what user they are associated with so I can delete the account. I am desperate for an answer here.

Thanks!

18 Answers

Well, we opened a case with MS Tech support as well. Here is what he said. Basically there is a bug in CRM/ CRM Outlook Client calendaring that will query forever, essentially in a loop which generates the massive amounts of transaction logs. He said that CRM Rollup 17 due out in May will address this issue. Until then we are making sure that when our CRM Admin published changes we are monitoring Exchange closely. Not the best answer but untill Rollup 17 is installed I guess it will have to do.

Our setup is the following:

Exchange 2010 SP1 HF1 on Server 2008 R2

Clients are Outlook 2007, CRM 4, Outlook plugin version 4 Rollup 13

Do you mean event id 623 or 629? 623’s indicate you are experiencing log stalls and your version store is getting backed up. Third party tools and clients can do this causing looping action and generating the massive logs. I don’t have experience with CRM plug in, but I would at least try to uninstall it from these two clients if it’s always occuring with them and reinstall it.

Definite 629. Source ESE  Warning, Category: Performance

Quote: “The database may benefit from widening the online maintenance window during off-peak hours in order to purge such nodes and reclaim their space. If this message persists, offline defragmentation may be run to remove all nodes which have been marked for deletion but have yet to be purged from the database”

For the event, please increase the online maintenance window

Maintain Mailbox Databases

James Luo

TechNet Subscriber Support in forum

If you have any feedback on our support, please contact [email protected]  

I have already made the maintenance window as big as possible. including all day Sat and Sunday and 8 or more hours each weekday. The last time this happened I moved everyone to a new database and it worked until CRM crashed last week. Now that we are in the CRM forum can someone tell me what this might be or where to look?

I just found our Exch 2010 (no SP) server experiencing exactly the same symptoms as Aszyman, but adjusting the Maintenance Window on the Exchange store was not our fix.  Our Database disk drive ran out off space (and records show some 25GB of higher than normal disk growth for the past 2 weeks on a volume that only contains the mailbox database).  Then our Exchange logfile volume ran out of space next.  After extending both Database and Log volumes in a pinch, the symptoms continued and we could see logfiles getting created 10-20 per minute.  Looking back in Explorer at the rest of the day’s Exchange mailbox store transaction logs, I was seeing up to 3.5GB of logfiles generated per hour over the course of a day.  After finding this article, I installed the EXmon utility and found one user connected using Outlook and CRM client plugin and putting a heavy load on CPU and generating a large amount of bytes in and out.  After killing Outlook on the user’s PC, the logs immediately stop filling up.  Come to find out, the CRM client installed on this user’s laptop was a much older version while everyone else using the CRM client plugin had a more current version.  Unistalled the CRM client from that PC and restarting Outlook produced no more problems on the Exchange Server.  Going to let this site for a while before installing the updated CRM client on that user’s PC.

Not sure about what to do with all the “junk” that was dumped into the Database and whether the maintenance will clear it out eventually or an offline defrag would, but am going to just create a new mailbox database today, migrate mailboxes, and purge the original DB store.

Well, we opened a case with MS Tech support as well. Here is what he said. Basically there is a bug in CRM/ CRM Outlook Client calendaring that will query forever, essentially in a loop which generates the massive amounts of transaction logs. He said that CRM Rollup 17 due out in May will address this issue. Until then we are making sure that when our CRM Admin published changes we are monitoring Exchange closely. Not the best answer but untill Rollup 17 is installed I guess it will have to do.

Our setup is the following:

Exchange 2010 SP1 HF1 on Server 2008 R2

Clients are Outlook 2007, CRM 4, Outlook plugin version 4 Rollup 13

We are having the exact sames issues as listed above… I am at least glad to see others are having the same issue and MS is aware of it. Any updates on this issue?

I have just been babysitting the log drive space usage. Some real time trending app for space usage would be great. Our CRM guy lets us know before he publishes so we can watch EXmon. No issues since. No patch has been released as far as we have heard. I have already moved all users to a new database so I can get rid of that freaking 629 error.

So it just happens when when the CRM guy publishes the changes? If that is true I can let my CRM guy to give me some heads up before he publishes.

So far as we have seen it is after a publish, typically when it times out and we do an IISReset to get CRM working again.

we are having these issues as well has anyone seen a change after applying roll up 17 ?

Bump. We just installed the rollup on the server here and we are going to start re-installing the Outlook Client today. Fingers crossed. Anyone have any updates?

the rollup 17 applied to the outlook Plugin stoped the extras from generating also cleared up the sync errors on the users

I have a simliar issue with garbage getting dumped in to my exchange log folder and making my database grow very large.  But we are currently on CRM 2011 and exchange 2010 sp1.  All the clients are office 2010 with CRM 2011 client for outlook.  When the problem is happening I can watch log files being added by 1-4 logs per second or more, having the users close outlook will make the log folder only grow by 1 log every minute or so, which seems like more normal traffic.  I have already moved my users to a new database with a total db size of about 30gb after about 1 to 2 months after the move my database is already 130gb.  Does anyone have and answer or update?  I beleive that I am on all the latest roll updates for exchange and CRM.  We did upgrade from CRM 4.0 and were experiencing the same issues when we were on 4.0. 

I dont have an answer, the rollup to CRM (v4) seemed to make it go away here. In the mean time to protect your Exchange server I would do the following.

Set up an alert of some sort when drive space on the log drive decreases quicker than normal.

Second, use EXmon to see what user is causing it. Once you find that user, kill all Outlook.exe sessions for the user. That will stop the growth from continuing.

Thats what I did until the rollup was released.

I’m seeing the 629 warnings in my Exchange 2010 RTM system as well, and i’m now wondering if it’s related to CRM, since we run that as well (I’m checking on the version/rollup status). I’m wondering what everyone here has for their Exchange environment – do you have all Exchange server roles installed on one box and using the CRM E-mail router?

In my environment, we have 2 combined Hub/CAS servers and 2 mailbox servers. CRM is on another machine entirely. We had seen the log files on the Hub/CAS servers increase significantly in size when we installed the CRM router, and eventually changed CRM to NOT use the router in Exchange – the log files on the CAS servers went back to normal.

But, I’m still seeing 629 warnings on my mailbox databases. Changing the online maintenance window didn’t seem to alleviate the problem, so I’m wondering if it has something to do with our CRM system.

Hi,

the same situation here. We have CRM in our customer environment and on the mailbox server hosting active copy of CRM mailbox database I can see ESE warning 629.

Widening the maintenance window didnt help.

Regards