MS20331 Core Solutions of Microsoft SharePoint Server 2013 Training – Day 4

We gradually reaching end of this 5 day course and day 4 was focused on User Profile Service and Search. Those are two big features, especially search. Though from indexing flow and search sub-components architecture it is normal full fledged search and my previous experience with Autonomy IDOL was somewhat handy for understanding this.

Service applications were introduced in Microsoft SharePoint Server 2010, replacing the Shared Service Provider architecture of Microsoft Office SharePointServer 2007. Service applications provide a flexible design for delivering services, such as Managed Metadata or PerformancePoint, to users who need them.Microsoft SharePoint Server 2013 includes more than 20 services, some of which are new to this version, whereas others are enhanced.In planning and configuring service applications, it is important that you understand the dependencies, resource usage, and business requirements for each.

Sharing, or federation, of service applications covered in more detail in course 20332B: Advanced Solutions of Microsoft SharePoint Server 2013.

Key components and topologies for SharePoint Server 2013 service application architecture Provision and manage SharePoint 2013 service applications.

The service application architecture was introduced in Microsoft SharePoint Server 2010, replacing the Shared Service Provider (SSP) model of Microsoft Office SharePoint Server 2007. The architecture remains consistent in SharePoint 2013, with the addition of new service applications.The advantage of service applications over the SSP is that services are more granular and can be deployed only to web applications that require the functionality that individual services offer. This offers greater design flexibility for the management of resources and functionality.Describe the function of service application instances and service application dependencies.

Describe some of the key options for service application topology design.Explain how to map business requirements to service application design.

SP Features work on 4 different levels: subsite, collection, farm, web app. Some basic feature can be enabled independently on any levels, other have dependencies.

you should find “SharePoint Server Publishing Infrastructure” under

“Site settings” > “Site Collection Administration” > “Site collection features”

you wont find it under site feature

or you can access it through this link :

https://{sharepoint server}/_layouts/15/ManageFeatures.aspx?Scope=Site for Site scope features
http://server/site/siteCollection/_layouts/ManageFeatures.aspx for Web scoped features


Architecture Picture

http://sharepoint.stackexchange.com/questions/52248/why-does-my-feature-appear-in-the-site-collection

SharePoint health analyzer – can report on missing service dependencies
State Service Service Application – can be configured with PS only.

#Uncomment the line below if you running this not in SP Management shell
#Add-PSSnapin Microsoft.SharePoint.PowerShell
$stateName = “State Service”
$stateDBName = “State_Service”
$stateDB = New-SPStateServiceDatabase -Name $stateDBName
$state = New-SPStateServiceApplication -Name $stateName -Database $stateDB
New-SPStateServiceApplicationProxy -Name ”$stateName Proxy” -ServiceApplication $state –DefaultProxyGroup
SP User Profile Service (UPS)
Provides the ability to create and administer user profiles that can be accessed from multiple sites and farms.
The User Profile Service is a service application in Microsoft SharePoint Server 2013 that provides a central location for configuring and managing the keyelements of personalization settings. The User Profile Service holds the settings for the following features:

User profiles. A user profile stores detailed information about the user in the form of properties. You can manage and display all of the properties that are related to each user.

Profile synchronization. You can synchronize user profile information that the User Profile Service stores with external directory services such as Active Directory Domain Services. A user profile can incorporate data from more than one source. Youcan schedule synchronization depending on how often you expect the relevant information to change.

Audiences. Audiences enable you to target content to users based on their jobs or tasks. You can define an audience by membership in a SharePoint group ordistribution list, by the organizational reporting structure, or by the public properties in user profiles.

My Site Host. My Site Host is a dedicated site for hosting My Site websites. You must provision a My Site Host before you can deploy the social features ofSharePoint 2013.

My Site website. Each user in your organization who has a synchronized user profile can have a personal site. Users can store documents, manage the contentof their My Site website, and share content with others. The My Site content storage is also referred to as SkyDrive@<companyname>.

Social tags and notes. Users can add social tags to documents, to other SharePoint items, and to other objects, such as external webpages and blog posts. Userscan also create notes on any SharePoint page. Administrators can delete all tags for employees when they leave the company or remove a tag that they do notwant.

User personalization permissions. You can use permissions settings within the User Profile Service application to control which users can edit profiles, usepersonal sites, and use tags and notes.

You cannot access the Manage Profile Service page until an instance of a User Profile Service application exists and the associated services are started. You canuse the SharePoint Central Administration website in addition to Windows PowerShell to create and manage User Profile Service applications and other serviceapplications for non-hosted environments. You can also delegate management of a User Profile Service application to someone who does not have permissionsto manage other services or settings contained in Central Administration.

My Sites. Microsoft push for replacing it with Yammer, but this met certain resistance as Yammer is purely cloud thing whereas My Sites local/on-prem (loads of companies are still not ready to embrace cloud :) ).
People Search.
Audiences.
Org Chart.
SQL Database Profiles.
Start 2 services: User Profile Service, User Profile Synchronization Service.
Create User Profile Service Application and 3 databases
Create separate site collection (optional) – as we need to have separate Web App, because we enabling auto creation of sites for users.
UPS sync with AD DS.
Special account – should have rights to replicate AD data (should be granted BOTH in ADUC and ADSI EDIT) on domain level (ADUC > Domain Properties > Security Tab > Grant Replicating Directory Changes)
miisclient.exe
SP Search Service
Changed significantly in SP 2013. Full text content search and attribute search.
Multiple content/index supported (mirror/stripe)
Configuring search:
Backup search service application:

MS20331 Core Solutions of Microsoft SharePoint Server 2013 Training – Day 3

Third day of the training was mainly about SharePoint cache and configuring SharePoint services.

Documents in libraries an elements in lists.
Libraries and Lists.
Child sites.
Site Collection. Defines web path and creates Root Site. Has own DB (could be separate)
Web Application. SQL DB – one per web app.
Path Based and Host Named site collections (russian):
Block file upload by extension (check extensions only) – CA > Manage Web Applications > Select Application > Click “Blocked File Types” on Ribbon
Big files upload settings. CA > Manage Web Applications > Select Application >  General Settings button on the ribbon > Maximum Upload Size (default 250 MB). For list items web.config
Both of aforementioned settings go to web config?
To create test files fsutil can be used:
fsutil file create new <filename> <lenght>
SP cold start issue. Warm up scripts.
Fiddler to test difference with and without warm up scripts
SP can be configured for anonymous access:
1. Enable anonymous access on Web App level. CA > Manage Web Applications > Authentication Providers button on the ribbon (verify in IIS)
SP Site Permissions Management.
1. Site Permissions.
Around 50 individual permissions for certain actions
Permission levels (groups of individual permissions) – Visitors, Participants, Owners etc. New levels can be created.
Permissins can be assigned to AD user account (or other user account) or AD groups or alternatively SP groups can be created (valod on site collection leve;)
Whichever method, or methods, you use to authorize user access to SharePoint objects in your environment, it is independent of the userauthentication mechanism you use because SharePoint 2013 converts all authenticated users into a SharePoint User object (SPUser).
Permissions inheritance/level: Site Collections, Sub Site, Libraries, Elements
Best practice – move on site collection level all content which should be accessible for all
In case you forgot how to create sites/subsites:
Revoke/deny access for particular user – Policy for Web Application
SP cache to alleviate load on SQL, network and SP app servers. Caching removes/minimizes queries to SQL.
If misconfigured caching can lead to perfrormance degradation. Mostly cach is disabled by default.
4 types of cache:
1. Blob cache (stores JS, CSS and pictures on WFE). Each WFE has its own BLOB cache. Cache penalty – 1st query is slower when using cache. If file accessible anonymously it speed things up as there is no neeed to check permissions. BLOB cache optimized for sites anonymous. Extra RAM consumption – extra 800 bytes per file (for index)
BLOB cache configuration: change web.config, BlobCache settings – location, file types, max size in GB
2. Output cache. Requires activation of SP Publishing Feature. Only Publishing pages go into this cache. Being stored in RAM. If disabled constant stream of F5 page refresh request may lead to significant load on SP server. Cache handling: TTL OR Cache being discarded if content changed on site. Each page takes 2x page size + 32 KB in RAM.
You can create cache profiles. E.g. for static content (like final version of article) set TTL, for changing change check.
Create Publishing site and enable SIte Output cache in Site Settings.
3. Object cache. Enabled by default and can not be disabled (navigation, search query box), but has to be configured (you may see warning event about this upon each IIS startup – event ID 7363). Configured after starting “Share Point Publishing Feature”

Open a shell

Execute the following commands:

 
#1
$wcm = Get-SPWebApplication -Identity http://[webappurl/] 
#2
$wcm.Properties["portalsuperuseraccount"] = “DOMAIN\sp_superuser”
#3
$wcm.Properties["portalsuperreaderaccount"] = “DOMAIN\sp_superreader”
SharePoint 2010: Event ID: 7362: The super user account utilized by the cache is not configured:
4. Distributed cache. Introduced in SP2013. Aka App Fabric Cache. Enabled by default and can be turned off. By default can use 10% of RAM.
#Check distibuted cache status on node:
Use-CacheCluster</div>
Get-CacheHost
#Check cache host config:
Get-CacheHostConfig SERVERNAME 22233</div>
LowWatermark: Percentage of memory usage (from Size) when *expired* items are removed (evicted) from cache if expiration is enabled.
HighWatermark: Percentage of memory usage (from Size) when *all* items may be removed (evicted) from cache if eviction is enabled.
Reference:
You may verify your cache using Performance Console on SP Server (perfmon) – report view and cache counters
SharePoint 2010 cache overview document from MS (no Distributed Cache coverage due to obvious reasons):
Use iisreset /noforce to avoid cache index corruption
Related links:
SharePoint 2013 + Distributed Cache (AppFabric) Troubleshooting
SP Services
CA > Manage Services on server
A few services can be Started and Stopped without extra configuring, those run on WFE
Other – Application Services
What Service?
Where it will be run?
Start?
Service Application
Example – configuring Excel Services
For Excel Services managed account you have to grant SPAccess right on content DB in SQL.
You also have to make sure that service not only created but also binded to site (CA > Application Management > Service Application sections > Service Application Associations)
Proxy Group
When you create a service application in SharePoint 2013, a service application connection is created. A service application connection is also referred to as an application proxy. A service application connection associates the service application to Web applications via membership in a service application connection group (also referred to as application proxy group).
SP Admin Site becomes hidden in IIS if service stopped
Get-SPServiceInstance | Where-Object {$_.TypeName -eq “Central Administration”} | Start-SPServiceInstance -Verbose

MS20331 Core Solutions of Microsoft SharePoint Server 2013 Training – Day 2

To bind this day content to exam objectives it was continuation of discussion of SP topology, but now with focus on design of logical architecture, whereas first day covered physical architecture and touched (just a bit) on information architecture.

Run CA As Administrator (there is some issues because of UAC, as some menu items will be missing, like Manage Services on server in System Settings section) when UAC is enabled.
SP configuration: 1. Create sites. 2. Set up and configure services. You need create sites first, as you need them to test services.
Create Web Application in SP.
Manage Web Application > Create New
Host Header, SSL optional
Authentication
Internal:
NTLM
Kerberos (requires extra setup, allows for delegation, faster in large networks)
Basic (passwords sent in plain text)
External:
Forms Based Authentication FBA (Exchange Web Access, external source for authentication date, e.g. SQL)
Claims Based (mainly for Internet portals – external providers like LiveId etc.)
Public URL (name + port)
Application pool name (just a name, make it nice and descriptive)
By default one DB per SP Web Application, but individual DBs can be created for each site collection (via PS)
It will create new site in IIS (host header, port – default HTTP/HTTPS or custom), setup authentication and create application pool (dedicated w3wp.exe process) and its service account (domain user is required for Kerberos, which won’t work if local account is being used)
With each application SP creates content DB on SQL
Site Collection: defines/contains templates, root site (template, administrators, quotas)
Site Collections can share one web app – managed path is used for differentiation (http://myportal/collection1 etc.)
Register account for portal-pool
CA > Security > Configure managed accounts
Despite the same name it doesn’t use the same Managed Service account functionality built-in into Windows Server, this one is separate thing for the same purpose
Once Web Application created we can start with site collection creation and first one will be top level site. Sub sites can be created (forum etc.)
Application Management > Manage Content Databases
Sites > Lists/Libraries
When you create new empty DB next collection will use it.
Upload file and check in content DB:
SELECT * FROM dbo.AllDocs
WHERE leafname='%docname%'
CA > Application Management > View all site collections – to see database name for sites
SP Management Shell get-pssnapin
PS / PS ISE:
add-pssnapin Microsoft.SharePoint.PoweShell
To manage SP via PS you have to add your account to SP shell administrators (even if you already have farm admin rights)
#this command requires rights on SQL
Add-SPShellAdmin -username domain\user -database %ID%
List all site collections with databases:
Get-SPWebApplication | get-spsite | format-table hostname,url,contentdatabase -AutoSize > C:\1.txt
# 1) Create web application
# 2) Create managed paths
# 3) Create number of site collections
#1
new-spwebapplication -name "Contoso Internet WebApp" -port 80 -HostHeader sharepoint.contoso.com -URL "http://www.contoso.com" -ApplicationPoool "ContosoAppPool" -ApplicationPoolAccount("DOMAIN\jdoe") `
-DatabaseName WSS_Content_Contoso
#2
$sites = 1..10
Foreach ($i in $sites)
{
New-SPManagedPath "site$i" -WebApplication "http://sharepoint.itband.ru" - Explicit
}
#3
Foreach ($i in $sites)
{
New-SPsite -name "Student Portal $i" -URL "http://sharepoint.itband.ru/site$i" `
-Template "STS#0" -owneralias "itband\sp_install"
}
Use klist to see if you have ticket to access SP (should contain user name and servername), if no ticket then kerberos is not in use
Configuring Kerberos:
DC setspn -S HTTP/portal.itband.ru itband\portal-pool
This can be done via editing Attribute of portal-pool account (DSA, user properties Attribute Editor tab then check servicePrincipalName attribute)
On SP server computer object enable delegation
Once done CA > Application Management, select application, click Authentication Providers on ribbon
Use klist command to verify or Even Viewer event ID 4624
SharePoint 2010 Kerberos configuration guide:
SSL Setup
Install CA
Allow enroll/write on Web Server template for Authenticated Users, gpupdate
Request web certificate, fill in Type CN=portal.domain.ru
Alternative Name DNS=portal.domain.ru
Enroll
On IIS add HTTPS binding for site
In CA add HTTPS entry in Alternate Access Mappings
HTTP URL Rewrite
WebApp can have 5 zones/names. Manage WebApp Extend. When you click OK in extend window don’t hurry to click around – it doesn’t give you any indication of work in progress and window also stays active, but you just have to vait. Pool stays the same.
Alternatively you may use DNS + IIS Bindings + SP CA Alternate Access Mappings
Extend allows for different authentication etc. All site settings.

MS20331 Core Solutions of Microsoft SharePoint Server 2013 Training – Day 1

Today was first day of 5 days SharePoint 2013 training for me. As you may expect first day of this training dealt with what SharePoint is (including use cases, planning and architecture) and we also went through 3 tier SharePoint 2013 SP1 farm installation.
I’m posting crude notes from day 1 with intent to trim them later :)
Sites: Internet/Intranet/B2B. Portals = Data + Services’SQL DB actual storage of documentClient: IE/other browser.

Collaboration. Versioning. Approvals. Notifications.

Content storage: Files (any)/ Data in Tables (Calendar events, Tasks, Any Web Tables, Polls, Surveys)

Portals VS Sites

Portal – data + services (e.g. Yandex – search engine, mail, marker, maps, traffic data)

SP Services:
Search (content indexing, including data external to SP like files on file servers) – has to be configured first
User Profiles Service – stores data about users. Sub components: Profiles (photo, details etc.), My Sites (site for each user where he has full access); potentially may be substituted with Yammer
InfoPath Services – for creation of InfoPath forms on SP for data collection etc. (InfoPath is still alive and kicking :) )
Excel Services – BA services, tooks Excel documents stored on SP and use them as a data source on SP pages
MMS – Managed Metadata Services – for building taxonomies across site collections
s
BCS – Business Connectivity Services (DBs as data sources)

EDMS. SP is not a DMS OOB. SP includes workflows (SP designer allow codeless creation of workflows)

SP 2013 introduced forms for collaboration (community sites, forums)

SP could be development platform

Typical SP usecases: 1) Corporate DropBox 2) Requests system (SP sites + InfoPath or other forms) 3) EDMS 4) External/Public portal/site (expensive, requires heavy customization)

SP imlementation development options: 1) No coding, only GUI tools – allows for easy migrations/upgrades to new SP versions. Painless. Supported. 2) Custom development – migration is a pain, potential loss of what you built. Your deployment evolves into “sepatate branch”

SP team (ideal scenario :) ): 1. SA (back end, architecture planning) 2. DBA (SP stores all its data in SQL) 3. Developer (ASP.NET/VS/DOT.NET) – optional if codebased SP development required. 4. Developer/Designer (SP Designer/Infopath Designer/Report Builder) 5. Designer (design site templates) 6. Site Administrator (end-user SP features exert which has company BP)

SP Farm – group of SP servers with shared configuration DB (SharePoint_Config) which serves your sites and runs SP services

SharePoint_Config DB stores SP farm config.

SP farm can comprise out of 1 to N servers

SP 2013 Foundation – no services (only search and BCS), can be downloaded free from MS site
SP 2013 Standard (no BA services – Excel, Performance Point)
SP 2013 Enterprise

Std/Ent – the same distributive, feature set defined by key

Within 1 farm different editions of SP can be used

Editions comparison:

SP servers:
WFE (Web Front End) – IIS – receives/returns clients queries
DB server (content databases, SharePoint_Config) – stores data
App Server – runs SP services – processes client queries
Production minimum: 2 servers WFE + APP & DB
Defining number of servers for SP farm. Factor in:
1) Number of employees
2) RPS (Requests per second) – difficult to quantify exactly in advance, some formulas available
3) Percentage of simultaneously working users
4) Quotien of peak load (typically value of 2 is used)
5) Average number of requests from user per day
RPS:
Evaluation of content DBs size
CDB size = ((DxV)xS) + (10Kbx(L+(VxD)))
Microsoft suggested farm topologies:
1 Server – WFE+APP+SQL on one box, less that 100 users or evaluation
2 tier farm – WFE+APP & SQL up to 10 000 users
3 tier farm – WFE & APP & DB
next different varieties of scaling out
Virtualization consodirations:
- spread extra servers on different hypervisor hosts
- don’t use snapshots in production (performance hit, potential data loss due to corruption of one VHD in the chain of snaphots)
- avoid dynamic memory, SP tends to hog it all/max limit
- multiple CPU settins, N of vCPUs should not be more than twice of N of real cores on virtualization hosts
- no thin provisioning, only fixed disks
SQL clustering (mirroring/AlwaysOn)
Office Web App (OWA) – separate product
IOPS min 0.25 per GB of DB, recommended 2 IOPS per GB
Soft limit 200GB / hard limit 4 TB for content DBs. Soft limit in place because of serviceabilty – back up/restore, maintenance time will be too long for big DBs
For SP farm diagrams package for Visio is available:
sharepoint visio stensils 2013s
Accounts for SP
SQL Server support: Server 2008 SP1 and beyond
.NET 3.5 installation from local source for SQL:
dism /online /enable-feature /all /featurename:netfx3 /source:d:\sources\sxs /limitaccess
Minimum SQL components:
Database Engine and data files, Replication, Full-Text Search, and Data Quality Services
Client Components (other than SQL Server Books Online components and Integration Services tools)
SQL Management Studio
Use named instances for convenience
Collation:
Named instance uses random TCP port (see SQL Server Configuration Manager)
Alias os strongly recommended for flexibility (SQL sever change)
c:\windows\system32\cliconfig on SP server – create alias
3 rules:
Random instance port
Program instance exe
UDP 1434 SQL Server Browser
MAXDOP=1 is required for SP farm install (when you do an install with full rights installer sets it to 1, if no rights – install fails)
SP install:
1. Prerequisites install (IIS + Components)
2. SP installation (extraction)
3. Language packs/updates installation
4. Farm creation wizard
SP prerequisites script (useful when SP server has no access to Internet)
No WCF 5.6 which is required for SP2013 SP1 and App Fabric CU
SP distrib contain folder prerequisiteinstallerfiles
Never choise stand-alone for prod install, as it will install local SQL and you won’t be able to add more servers into farm
SP product configuration wizard
SP DB access account – sp_farm (farm account, used for Windows Timer Service, CA and User Profile service)
passphrase – used when adding servers to domain and also for encryption SP service passwords, could be changed by farm administrator in CA
IE loopback check issue – when you unable to access sites from SP server, solution regedit:
New-ItemProperty HKLM:\System\CurrentControlSet\Control\Lsa -Name "DisableLoopbackCheck" -Value "1" -PropertyType dword

Windows 8.1 ReFS support

Windows 8.1 supports Resilient File System aks ReFS (codename “Protogon”). As WinFS project was cancelled ReFS is what we have as possible successor of NTFS in its early days. This file system become available in Windows Server 2012 and tightly connected with Storage Spaces technology. Window 8/8.1 also supports this file system (as well as other server side things like Hyper-V and Storage Spaces with some limitations).

But if you venture out to disk management on Windows 8.1 you most likely won’t see an option to format any of your drives to ReFS. This is because by default it is allowed on mirrored volumes only. But you may change this through registry settings. First you need to create new registry key “MiniNT” in the following location:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control

Next you should create a new DWORD value called “AllowRefsFormatOverNonmirrorVolume” and set it to 1. Reboot and you will be able to format your volumes as ReFS volumes even if they are not mirrored.

Format to ReFS

How to: quickly check SQL server collation settings

Just a quick note on how to quickly check Microsoft SQL server collation without wading through SQL Server Management Studio GUI. Go to CMD and execute the following:

sqlcmd -q "SELECT CONVERT (varchar, SERVERPROPERTY('collation'));"

Alternatively you may use sp_helpsort stored procedure, though when run via sqlcmd it gives a bit strange output if run with sqlcmd

sqlcmd -q "EXECUTE sp_helpsort;"

On the boxes with SQL server installed path to sqlcmd executable added to PATH variable so you can fire it off without specifying full path. I guess you may also check corresponding key in registry which should exist there I believe (at least for server collation) :)

Also you may view collation settings on databases:

sqlcmd -q "SELECT name, collation_name FROM sys.databases;"

Reference: MSDN: View Collation Information

The Art of Conflict Management by Professor Michael Dues

I recently finished listening to  (at least first pass done, maybe revisit it later – most of Great Courses titles worth revisiting/more than one listen) of 24 lectures series from The Great Courses on conflict management – “The Art of Conflict Management: Achieving Solutions for Life, Work, and Beyond” by Professor Michael Dues.

The Art of Conflict Management

This course uses dramatizations to illustrate conflict situations and ways of handling them, and tries to emphasize practical side (some assignments suggested in the end of each lecture) which is as usual by far more difficult than theory.

My takeaways from this course is number of interesting models and shortcuts to think about conflicts (triangulation, defunct conflict strategies etc.), then science and history behind widespread buzz-word “win-win”. It was interesting to know ideas behind the word which is being thrown around sometimes mindlessly nowadays. We can trace back almost any concept or technology to the initial (in hindsight sometimes plain and simple) idea or scientific paper. For Kerberos technology it was project Athena, based in turn on is on a paper published in 1978 by Needham and Schroeder (Needham–Schroeder protocol) for win-win idea it was 1948 Morton Deutsch’s PhD paper about win-win solutions. Basically he distinguished 2 types of conflicts: competitive conflict, a situation that requires one party to lose in order for the other to win, and pure conflict, a situation in which both parties can fully win. This is important distinction and gives you different point of view on possibilities for conflict resolutions, in addition to point of view which is formed by long standing idea of  adversary system which comes from Ancient Greece.

There also was nice overview lecture on overarching managerial theories – really good summary on each and on movement from one to another. I also liked  the story mentioned at some point there on etymology of the word bureaucracy (which is French in origin, and combines the French word bureau – desk or office – with the Greek word κράτος kratos – rule or political power).

Next I going to start listening to my first audio book in French which is surprisingly enough “Le journal d’un fou” by Nicolas Gogol :) And I also got another title from The Great Courses – “Building a Better Vocabulary” by Professor Kevin Flanigan.