This is the transcript of 2nd part of iManage WCS webinar series.\n\nPart 2. Email Filing Server – Best Practices Design and Deployment\n\nSizing & deployment strategies.\n\nAgenda.\n
- \n
- Introduction\n
- \n
- EFS Review
- Performance Defined
- Performance Variables
\n
\n
- Filing Time\n
- \n
- EFS Proximity to Exchange
- EFS Proximity to WorkSite DMS
- DMS Proximity to SQL
\n
\n
- Scaling for Improved Performance
- Summary
- Sizing scenarios\n
- \n
- Case Study for Central Exchange Architecture
- Case Study for Distributed Exchange Architecture
\n
\n
\n
\n
\n
Introduction – EFS Review
Email Filing (Filing Worker) is asynchronous (i.e. filing is not real time)
User flags a message to be filed in Outlook
On the Email Filing Server, the Filing Worker process attempts to file the message on its next iteration. All messages are filed in batches.
The amount of time that Filing Worker spends on each iteration is a product of:
– The number of messages that need to be processed
– The amount of time it takes to process each message
An Admin will have no direct control over the number of messages that his/her users are filing.
Marked Folder Processing (Marking Worker) is asynchronous
– User maps (marks) an Outlook folder to a WorkSite folder.
– MarkingWorker attempts to process the folder on next iteration. The MarkingWorker processes folders in batches. All folders are checked each iteration.
– For each folder, MarkingWorker determines the messages that haven’t been filed and processes these one at a time.
– The amount of time that MarkingWorker spends on each iteration is a product of
\n\n
- \n
- The number of marked folders that need to be processed
- The number of messages that need to be processed
- The amount of time it takes to process each message
\n
\n
– An admin will have no direct control over the number of folders that his/her users are marking, nor does he/she have control over the number of messages that are in each Outlook folder that get’s processed
Introduction – Performance Defined
Q: What is the single most important measure of performance for Email Filing Server?
A: The amount of time it takes EFS to process a batch.
– Messages or Marked Folders are processed in batches. The faster the batch completes, the faster the messages in the batch complete. The faster the next batch can be processed.
– The amount of time that a message takes to file after a user has flagged the message as a WorkSite message (either by placing it in a WorkSite folder, or by putting it in a Marked Outlook folder)
Introduction – Performance Variables\n\n- Email Filing\n
- \n
- Number of messages being processed in this batch
- Single message filing time
\n
\n- Marked Folder Processing\n
- \n
- Number of Marked Folders
- Number of messages that need to be filed from each Outlook folder
- Single Message filing time
\n
\n
\nThere are methods to modify each of these variable in order to have a positive impact on message filing time.\n\nFiling Time – Proximity of EFS to Exchange\n\nEFS proximity to Exchange has a significant impact on performance\n\n- The time it takes EFS to file one or more queued items to WorkSite (appear in the WorkSite filing location and/or see updated filing information on the original item) after they’ve been successfully queued.\n\n- The time it takes EFS to file items to WorkSite after being added to a Marked Folder\n\nThe performance of the following filing functions will improve with EFS located geographically close to the user’s Exchange mailbox:\n\n- Connect to user mailboxes in Exchange\n\n- Locate message or folder in Exchange for Filing or Marking process\n\n- Search mapped Exchange folders to retrieve new content added to marked folders\n\n- Copy message from Exchange to EFS server for import into WorkSite\n\n- Update filing information values on Emails to reflect the filing status (Filed, Queued, Failed…)\n\nFiling Time – EFS proximity to WorkSite\n\nThe perfromance of the following filing functions will improve with EFS located geogrpaphically close to the DMS server:\n\n- Load and validate filing jobs and marked folders\n\n- Import downloaded items from EFS to WorkSite (WorkSite import process)\n\n- Update EM_REQUESTS and EM_PROJECTS to reflect filing status\n\n- Load and validate users\n\nFiling Time – WorkSite Proximity to SQL\n\nThe performance of the following filing functions will improve with the DMS located geographically close to the SQL server:\n\n- Import downloaded items into WorkSite\n\n- Update EM_REQUESTS and EM_PROJECTS to reflect filing status\n\n- Perform duplicate detection checks (if enabled)\n\nArchitectural Tie Breakers\n\nShould EFS be geographically closer to Exchange or WorkSite DMS?\n\n- EFS should be located geographically closer to Exchange (as MAPI is much more “chattier” protocol by comparison with WorkSite communications traffic which are optimised for WAN; ideally both Exchange & WorkSite should be close to EFS, in other cases Exchange is preferable)\n\nShould WorkSite DMS be geographically closer to EFS or SQL?\n\n- DMS should be geographically closer to SQL.\n\n* It is not recommended to use caching/proxy for EFS connectivity.\n\nScaling – EFS Threads
– Each EFS server is configured by default to have 10 threads for Email Filing, 10 threads for Marked Folders, and 10 threads for Mailbox Synchronization. This means that there could be 30+ concurrent connections to the DMS at any given time (The EFS Management console and Legacy EFS service also make separate connections)
– Each thread will generate multiple transactions for the various calls the services will need to make to the DMS, including downloading queued requests, importing items, and searching the database.
– Some transactions, such as selecting filing requests, are very lightweight when compared to imorting items.
\n* This is not out of ordinary for WorkSite servers which provide connectivity to EFS to have high transaction per minute values, but you should keep in mind that a lot of these transactions (generated by EFS) are lightweight.\n\nScaling – Add Email Filing Servers
While you have no control over the number of messages submitted by your users, you can reduce the amount of time that EFS spends on each batch by spreading this work out across additional Email Filing Servers (scaling out).
\nMethods for adding EFS servers:\n
– Automatic Load Balancing with Clustering (preferred)
– Single server mode (using WorkSite security groups to divide data among servers and/or database/mailbox filters)
Note: WorkSite database filters and Exchange mailbox server filters can be applied in conjunction with either of the above methods. However WorkSite security groups can be used only with the single server mode at this point. If you do enable Automatic Load Balancing with Clustering you can’t use WorkSite groups and apply those on top of clustered servers.
\nAutomatic Load Balancing\n\nBenefits of using Automatic Load Balancing with Clustering rather than single server mode\n\n- Loads new WorkSite users (daily) automatically without admin intervention (no need to perform manual load balancing each time when new user was added)\n\n- Provides failover protection at the EFS level\n\n- Less administrative overhead when adding additional Email Filing Servers (no need to mess with groups as in multiple non-clustered WCS servers scenario)\n\nScaling – Add DMS Servers
EFS will increase the workload on the WorkSite system as we have made it very easy to file a large number of emails to WorkSite.
Medium and Large firms in particular may want to segregate the asynchronous work of EFS from the real time work of production DMS. This will decrease the likelihood of a negative impact to production with the increased workload.
Best practice is to have a dedicated DMS server/cluster for EFS.
Some benefits of this approach include:
– Protect production users from the additional transactions introduced by EFS
– Simplify maintenance and troubleshooting (isolation of EFS traffic/log events from clients traffic/log events)
It may take a few tries to determine the best number of additional DMS servers to use for this “back office” DMS server/cluster.
In medium and large environments typical ration is 2 EFS servers to 1 DMS server. Though this is very difficult to size it upfront as it largely depends on actual workload.
Summary – Exchange Architecture
Centralized vs. Distributed Exchange
– For optimal performance, EFS should always be local to the service account mailbox.
– For centralized Exchange environments, all EFS servers should be local to central Exchange
– For distributed Exchange environments, EFS servers should be placed near in each Exchange location (separate service account with local mailboxes in each location for the EFS service processing locally)
– Mailbox Server filters can be applied to each EFS server. Each server will process only the mailboxes which are located on the mailbox servers included in the applied filter.
Summary – WorkSite Architecture
Centralized vs. Distributed WorkSite
– For optimal performance, EFS should be local to the WorkSite DMS server/cluster and SQL database (ideal case)
– When the EFS cannot be local to WorkSite, meshed DMS (eDMS) servers shouldn’t be used. EFS should connect directly to DMS servers/clusters which are local to the SQL database. It’s better to have EFS traffic over the WAN, than ODBC traffic, so you should place WorkSite locally to SQL and connect your EFS to this WorkSite server. Either it’s not recommended to provide EFS connectivity throgh caching/proxy DMS servers.
– WorkSite database filters can be applied to each DMS connection on each EFS server. Each server will process only the databases included in the applied filter for each connection.
We can make connection to individual WorkSite servers and/or individual databases from each EFS server.
Design Scenario A – Exchange Distributed\n\nInitial Design:\n\n
Chicago (Exchange CAS Array, 8 node DMS Cluster, SQL Database, 2000 WorkSite EM clients) – WAN – New York (Exchange, 2 node DMS cluster, SQL database, 500 WorkSite EM Clients)
Design with EFS:\n\n 
Chicago (8 node EFS Cluster, Exchange CAS Array (with added connection to Chicago Back Office DMS), 4 Node Back Office DMS Cluster, 8 Node DMS Cluster (with added connections to NY databases), SQL Database, 2000 WorkSite EM clients) WAN – New York (2 node EFS Cluster (with added connection to back office DMS Cluster in NY), Back Office DMS Server, Exchange, 2 node DMS cluster (with added connections to Chicago databases), SQL database, 500 WorkSite EM clients)
EFS servers were added to each location doing mailbox processing only for local user base. EFS servers to DMS servers ratio is 2 to 1.\n\nDesign Scenario B – Exchange Centralized\n\nInitial Design (the same as a but Exchange is centralized and located in Chicago):\n\n 
\n\nDesign With EFS:\n\n 
\n\nQuestios from last session:
– Q: EFS is an asynchronous process. What value is added by increasing the number of Email Filing Servers (scaling out)?
– A: It deacreases filing time by distributing EFS work across multiple servers.
– Q: We have no ability to limit the amount of email that a user may submit for processing. What challenges does this present to the design, implementation, and administration of the system?
– A: We must scale the system to deliver the functionality and performance required by the business. As we have no control over the amount of email coming in, this process could be iterative and may thake several “tries” before the right balance is found.
Final Thoughts
Here are a couple of questions that we would like you to think about before we meet for the next session.
– Now that you have a good understanding of how EFS works, where do you expect problems to occur?
– What types of problems have you encountered with EFS?
1 Trackback or Pingback