iManage WCS webinar series – Part 3

This is the transcript of 3rd part of iManage WCS webinar series (webinar recording date May 23^rd, 2013)\n\nPart 3 – Troubleshooting\n

\n
1. I.                    Introduction
   \n
   1. Functional Review

\n

2. II.                  Email Filing Server
   \n
   1. Monitoring queues

   \n

   2. Logging

   \n

   3. Configuration files

   \n

   4. Failures/Errors

\n

3. III.                Mailbox Sync
   \n
   1. Logging

   \n

   2. SQL Compact databases

   \n

   3. Control Panel/Simulator

\n

4. IV.                Summary

Introduction – Functional review\n\n \n\nEmail Filing Server\n\n \n\n-          FilingWorker.exe (process) – Processes server-side filed emails. Reads queued jobs from the EM_REQUEST table\n\n-          MarkingWorker.exe – Processes marked folders. Reads folders from the EM_PROJECTS table\n\n-          EmailMgmtSvc.exe (service) – Manages startup and shutdown of the FilingWorker and MarkingWorker processes\n\n-          EmailMgmtAdmin.exe – EFS Management Console. Validates users in a single server configuration\n\n-          LoadBalancer.exe (service) – Validates and distributes users in a clustered configuration\n\n-          imFmaSvc.exe (service) – WorkSite Cluster Manager service used for communication between nodes by LoadBalancer\n\nMailbox Sync\n\n \n\n-          Sweeper.exe (service) – Adds filing information to Inbox emails which have already been filed to WorkSite.\n\nEmail Filing Server – Monitoring Queues\n\n \n\nThis may be useful both for troubleshooting and general monitoring\n\n \n\nFiling Request Management\n\n-          Displays filing jobs queued in the EM_REQUESTS table\n\n-          Shows job details including type (email or folder) and submitted time\n\n-          Allows admin to reset or cancel jobs\n\nMarked Folder Management\n\n-          Displays marked folders stored in the EM_PROJECTS table\n\n-          Shows synchronization status and last sync time\n\n-          Allows admin to reset or disable folders\n\nEmail Filing Server – Configuration Files\n\n \n\nEFS Configuration Files\n\n \n\nAll EFS configuration files are written to …AutonomyWorkSiteServerConfig\n\n-          DmsSettingsX.config – Contains DMS connection information. One file will exist for each DMS connection made by EFS (i.e. DmsSettings0.config, DmsSettings1.config)\n\n-          ExchangeSettings.config – Contains Exchange connection information, including service account credentials used to access mailboxes.\n\n-          FilingOptions.config – Contains settings which define how profile metadata is applied to filed items\n\n-          SvcSettings.config – Contains settings which define how EFS processes behave, including number of threads and polling interval\n\nEFS ValidUsers Files\n\n \n\n-          ValidUsers.urs (contains extended information including some details on users’ mailboxes) and ValidUsers.txt (stripped down copy of .urs file) – Contains list of users which were validated for processing (created upon automated/manual load balancing)\n\n \n\n* all these config files can be backed, for example before you make any changes during troubleshooting process\n\nEFS Log Information\n\n \n\n-          All EFS log are written to …AutonomyWorkSiteServerLogs\n\n-          Individual logs are created for each process – FilingWorker.exe, MarkingWorker.exe, EmailMgmtSvc.exe, EmailMgmtAdmin.exe, LoadBalancer.exe, imFmaSvc.exe\n\n-          For each process, a new log will be created on the next startup after the existing log reaches 2MB. If the active log exceeds 40MB, a new log will be created immediately\n\n-          Each process will retain the 50 most recent logs before overwriting\n\n-          Log Filter level is set to Verbose by default\n\n-          Log monitor can be used to view log writes in real-time\n\nEmail Filing Server – Log Information\n\n \n\nWhere to Look for Logging\n\n \n\nFilingWorker.log – Contains information about filing job processing, including downloading the item(s) from Exchange and importing them into WorkSite.\n\nContains filing error messages – server side filing requests from EM_REQUESTS table, e.g. profile error etc. In additional to this log you usually see detailed error information in a filing status column on a client side.\n\nMarkingWorker.log – Contains information about marked folder processing, including downloading the folder contents from Exchange and importing item(s) into WorkSite. E.g. errors with filing of individual message within the marked folder.\n\nEmailMgmtSvc – Contains information on service startup and continuous cycling of the worker processes.\n\nContains information about general server issues, startup failures. This service control worker processes, so it’s your destination if you need to check why marking or filing worker not starting or not happening as quickly as they should be.\n\nEmailMgmtAdmin – Contains information on what is displayed in the EFS management Console. Also displays user validation information in a single server configuration.\n\nIf you are not using clustering component  with automatic load balancing then all the logic for the user validation is within this executable. When you click on a Load Balancing button on a non clustered system all the user validation happens in this console and all the output for failures and successes during that validation will be written to EmailMgmtAdmin.log\n\nLoadBalancer – Contains information on user validation and distribution in a clustered environment.\n\nContains data about users validation on clustered systems only, plus about users distribution and number of cluster nodes.\n\nimFmaSvc – Contains information on the communication between nodes in a clustered environment.\n\nMay contain errors related with joining nodes to the cluster and communication between nodes.\n\nEmail Filing Server – Reading Logs\n\n \n\nUnderstanding the FilingWorker Logs\n\n \n\nBy default, FilingWorker will launch 1 main thread and 10 worker threads (configurable). Each line in the FilingWorker log indicates the thread number which is writing the line. Each log will include the following information:\n

\n
1. The main thread (thread #1) will write startup information, load and validate queued jobs from the EM_REQUESTS table for valid users, and initialize worker threads.

\n

2. Each worker thread (thread #5-14) will connect to a user mailbox, attempt to process the user’s queued items (500 max)

\n

3. As worker threads complete work for each user, the main thread will check the EM_REQUESTS table for more work

\n

4. Once no additional work is available, the main thread will update filing statuses, remove completed jobs from the EM_REQUESTS table, shutdown all worker threads and kill FilingWorker.exe

\nIt is convenient to use as a log viewer something like Notepad++, which allow you to highlight thread numbers to more easily follow them and has good search capabilities.\n\nUnderstanding the MarkingWorker Logs\n\n \n\nBy default, MarkingWorker will launch 1 main thread and 10 worker threads (configurable). Each line in the MarkingWorker log indicates the thread number which is writing the line.\n\nEach log will include the following information:\n

\n
1. The main thread (thread #1) will write startup information, load and validate marked folder links from the EM_PROJECTS table for valid users, and initialize worker threads.

\n

2. Each worker thread (thread #5-14) will connect to a user mailbox, attempt to process the user’s marked folders. The status of each folder will be updated during processing.

\n

3. Once all users are complete, the main thread will shutdown all worker threads and kill MarkingWorker.exe

Understanding the LoadBalancer Logs\n\n \n\nThe LoadBalancer logs will capture information on user validation process. For users which are successfully added to the ValidUsers file for processing, minimal information is captured. For failures in validation, more detail will be recorded. Each log will show following:\n

\n
1. Write startup information

\n

2. Retrieve all enabled users with properly configured email address in WorkSite.

\n

3. Validate all users against AD or Address Book (by default Address Book validation is turned on)

\n

4. Report user distribution

Email Filing Server – Failures/Errors\n\n \n\nFilingWorker Processing\n\n \n\nSome issues which can cause filing failures include:\n\n-          Profile errors – Refer to FilingWorker log for details. Example:  Cannot Import the message: \n\nProfile Error: AttID=imProfileCustom1, Value=TEST, Desc=The field is disabled\n\n-          Filing user not validated – Refer to LoadBalancer log for clustered environments, EmailMgmtAdmin log for single server environments. Example:\n\nThe user account with the Email Address: USER@DOMAIN.COM failed agains Active Directory\n\n-          Filed item cannot be located using ENTRY_ID or PR_SEARCH_KEY – Refer to Filing Worker log for details. Example:\n\nFiling Error…System error. <spMsgStore.OpenEntry: EID=123456789> NotFound at Interwoven.WorkSite.Ems.ThFilingProcessor.ThEfsConroller.ProcessFilingRequest(ThConfigData&ConfigData,ThDMS& Dms, ThRequest& tfr, IMsgStore, IMAPISession spSession, Boolean bDoNotCheck, List’1& vDouble, ThSlot& Slot)\n\n \n\nMarkingWorker Processing\n\n \n\nSome issues which can cause marked folder failures include:\n\n-          Profile errors – Refer to MarkingWorker log for details. Example:\n\nCannot Import the message: Profile Error: AttID=imProfileClass, Value=, Desc=Field is required\n\n-          Filing user not validated – Refer to LoadBalancer log for clustered environments, EmailMgmtAdmin log for single server environments. Example:\n\nTESTUSER does not have an E-Mail address\n\n-          Folder cannot be located using ENTRY_ID – Refer to FilingWorker log for details. Example:\n\nSystem error.<spFolder.GetProps> NotFound sAddInfo: <> at Interwoven.WorkSite.Ems.ThFilingProcessor.ThEfsController.GetLatestMessages(IMAPISession spSession, IMsgStore spMsgStore, ENTRYID FolderEID, MarkedFolder& mf, List’1& vEID, ENTRYID& ParentFolderEID, String sAddMsgClass)\n\nNOTE: Marked folder status is only updated to Successful or Failed based on whether the process was able to locate the folder and retrieve the contents. Individual item filing successfully or not does not reflect the folder status.\n\nMailbox Synchronization – Log Information\n\n \n\nMailbox Synchronization Logging Information\n\n \n\n-          Synchronization Log Monitor displays Mailbox Sync log activity in real-time.\n\n-          Log files are written to the following location:\n\nWindows 2008 Server:\n\nC:ProgramDataAutonomyWorkSiteMailboxAgentLogsMirrorLogs\n\nWindows 2003 Server:\n\nC:Documents and SettingsAll UsersAutonomyWorkSiteMailboxAgentLogsMirrorLogs\n\n-          MirrorLogs directory is only written after a successful initial synchronization\n\n-          During processing, all log data is written to SweeperLog.sdf\n\n-          SweeperLog.sdf data is written to log files in MirrorLogs upon each cycle completion\n\nMailbox Synchronization – Data\n\n \n\nMailbox Synchronization SQL Compact Files\n\n \n\n-          Mailbox Synchronization maintains separate SQL Compact data files for each user in the following location:\n\nWindows 2008 Server:\n\nC:ProgramDataAutonomyWorkSiteMailboxAgentDataStorage\n\nWindows 2003 Server:\n\nC:Documents and SettingsAll UsersAutonomyWorkSiteMailboxAgentDataStorage\n\n-          SQL Compact user files maintain a MSG ID database of the Inbox contents\n\n-          Service logging is written to SweeperLog.sdf during processing\n\n \n\nResetting Sync Data\n\n \n\n-          All synch data can be reset by stopping EFS and Mailbox Synchronization services and renaming.\n\nC:ProgramDataAutonomyWorkSiteMailboxAgent. This will force the initial synch to take place again.\n\n-          Individual users can be reset by renaming individual user SDF files.\n\n \n\nMailbox Synchronization – Control Panel\n\n \n\nControl Panel Functions\n\n \n\n-          Synch – Force an immediate synchronization without waiting for the polling interval (default, out of box polling interval equals to 1 hour)\n\n-          Validate Users – Validates the users in the WorkSite group used for Mailbox Synchronization (single server configuration only, in clustered configurations by default Mailbox Synchronization ignores group and perform processing for all users each node is balanced for)\n\n-          Select Mailbox – Lists all validated mailboxes for which Mailbox Synchronization is currently processing\n\n \n\nMailbox Synchronization – Simulator\n\n \n\nMailbox Operations Simulator\n\n 

–          The Mailbox Operations Simulator can be used to simulate individual Mailbox Synchronization activities against specific users

–          Helpful for troubleshooting problems with specific users as well as general issues with Mailbox Synchronization

–          Does not update data, only simulates activity and displays output

Summary\n\nTroubleshooting Email Filing Server\n\n \n\n-          Check the queue – Make sure the queued job or marked folder shows in queue\n\n-          Check the user – Verify that the filing user is validated on the server\n\n-          Check the logs – Look in the appropriate service log for errors\n\n \n\nTroubleshooting Mailbox Synchronization\n\n-          Check the user – Make sure user mailbox is visible\n\n-          Check the logs – Check the logs for errors\n\n-          Test – Use the Mailbox Operations Simulator