On hotkeys or the most important hotkey for Ubuntu GUI

Whenever I observe a Windows user (sometimes very proficient in IT) struggling with UI to invoke some sort of a tool of immediate necessity and just scrolling through start menu icons instead of relying on “CTRL + R + type exact executable or applet name” or just “WIN + type immediately what you need” I found it really weird. I mean that’s fine for vi proficient Linux command line guru or a Windows user whose needs do not go beyond web browsing and couple of regularly used apps/games, but when I see this type of behavior from full-time IT person touching Windows systems on a daily basis that seems to me a really strange avoidance to learn fundamentals/tools of one’s craft 🙂

As I’m currently using Linux more and more it is quite interesting to observe various levels of the same paradox/pattern among more then proficient Linux (desktop) users. For example, the other day during the training, I was observing instructor trying to guide a student to invoke Linux context menu and use “Open in Terminal” while that stubbornly tried to rely on search to find Terminal icon there 🙂 So use of context menu from empty desktop area and selecting “Open in Terminal” is a LEVEL2 way of accessing Terminal… but honestly what one should do, and what is true LEVEL3 ,is to use CTRL+ALT+T hotkey whenever one needs to jump into terminal from desktop UI, this is as cool as using what was called MCSE hot key in times long forgotten to invoke Task Manager (CTRL+ALT+ESC). Using hotkeys may not or may not impress other people (frequently it does), but you should learn them just because it makes your work so much efficient – after passing through short learning curve you just won’t want to use “long path” of accessing things which is reserved for people who didn’t care to learn shortcuts and trying not to notice how slow and clumsy wading through UI is at times.

I just wanted to share/reiterate that knowing some hotkeys just speed ups you a lot, and I guess the one should remember that being a professional implies both understanding of software use cases and inner workings as well as its usage basics (such as knowledge of hotkeys and UI features) – both things are required to be considered as a professional in the field IMO and it is sad to see that some trying to neglect the basics, let’s do not do that 🙂

Azure DevOps pipeline unable to checkout external repo – remote: TF401019: The Git repository with name or identifier does not exist

Assume the following scenario you trying to run Azure DevOps pipeline from one Azure DevOps repo and it relies on some contents in another Azure DevOps repo which specified in resources section of your YAML pipeline (for example it can be a repo which stores your master templates). You absolutely sure that your resources section is right and that it contains correct values for repository type and name as you were reusing this configuration in some other pipelines in other projects. For example, your pipeline resources section may look as follows:

Sample resources section

Nonetheless your YAML pipeline stored in your new Azure DevOps repository keeps failing with the following error:

remote: TF401019: The Git repository with name or identifier CORRECT_REPOSITORY_NAME does not exist or you do not have permissions for the operation you are attempting.

Here is a sample error message screenshot from pipeline job history:

Failing pipeline checkout stage

So the problem here (as we are sure that resource section repository config is OK) is configuration of a DevOps project which contains failing pipeline. To resolve this you will need to disable the following project settings:

  • Limit job authorization scope to current project for non-release pipelines
  • Limit job authorization scope to referenced Azure DevOps repositories
Azure DevOps project authorization scope settings

Once these setting changed you should be able to re-run pipeline to make sure that it no longer fails due to TF401019 error.

Unable to install SQL Server 2019 – SQL Install Error: The MOF compiler could not connect with the WMI server

The other day I was doing K2 5.5 installation on top of Windows Server 2019 and somehow that install beat all the records in terms of weird issues I run into, and even after re-provisioning server and starting from scratch I had to do resolve quite a few of them. One of those errors (not a directly K2 related one) was inability to install required version of SQL Server. Somehow I got a re-provisioned server with SQL 2014 and, as it is not supported by K2 5.5, I had to install SQL Server 2019 (5.5 supports 2016, 2017, 2019 and Azure SQL Database). But while installing SQL 2019 I run into the following blocking error:

SQL Install Error: The MOF compiler could not connect with the WMI server. This is either because of semantic error such as an incompatibility with existing WMI repository or an actual error such as the failure of WMI server to start.

Error message during installation of SQL Server 2019

Error message is a bit broad and it persisted even after removing all SQL related components from Programs and Features UI as well as deleting some remaining SQL related folders and registry keys. And I was trying to install only database engine services without any extra components and install was keep failing. That required to look into SQL installer logs where it was possible to see more specific/actionable error which looked approximately as follows:

Specified instance {SOME_GUID} via transform :InstID02.mst;:InstName02.mst is already installed. MSINEWINSTANCE requires a new instance that is not installed.

Error message from SQL Server 2019 Install Log

That basically means that some remnants of previous SQL installations are present and block installer from installing new/required versions of some components. And when you already deleted all SQL installer components from Programs and Features UI, you will need to query software list inventory using WMIC (Windows Management Instrumentation command-line) command:

WMIC PRODUCT Where "Caption like '%SQL%'" GET Caption, IdentifyingNumber

That will probably return bunch of SQL components with various GUID, as a minimum you will need to remove components with GUIDs mentioned in SQL installer logs, or if your goal is complete clean up of all SQL components all of them. As those are not visible in Programs and features you may delete them using the following command:

msiexec /x {SOME_GUID}

As in my case I had dozens of components returned and I wanted to do a complete clean up I created a little script for that:

SQL Server Components Clean Up script

With high probability removal of some of the components with the command shown above may fail and it that case you will need to remove them directly from registry looking for GUID value under the following keys:

HKEY_CLASSES_ROOT\Installer\Products\
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products

P.S. There is a blog post on sqlauthority.com which describes dealing with the same issue and helped me to resolve mine.

Validating Azure DevOps YAML pipeline locally

Today my colleague asked me how to validate Azure pipeline without committing changes to repo and trying to run pipeline. There is a VS Code Azure Pipelines YAML Validator plugin, but it does not work for YAML files which reference template files. In this scenario (validation of YAML files which reference template files) you can perform local validation using VSTeam PowerShell module, see details on using this approach below.

Output of Test-VSTeamYamlPipeline returns Name / URL / state / Id and state = unknown with Id = -1 means that no validation errors were found:

If validation errors will be found it will return error text with validation error details – you will see the same errors as if you would try to run this pipeline – such things as missing required parameter(s) or unknown parameter(s) will be reported.

Blast from the past: Configure K2 SmartForms for AAD Authentication (4.7)

DISCLAIMER: This is a post which used to be published on another web site, and as it is no longer available there just re-publishing it here again. It is an old write up from summer 2017, so a lot of things surely have changed since then.

In my previous article –  Azure AD instance set up basic walkthrough, I covered process of setting up of an Azure Active Directory (AAD) instance and also such basic things as subsequent AAD user creation, joining Windows 10 device to AAD and configuring logon to Windows 10 device using AAD account. 

In this blog post I decided to put this AAD instance to more serious/interesting use and configure K2 SmartForms for AAD authentication. When configured, such AAD integration allows AAD users to login to K2 SmartForms web sites (Runtime, Designer, Management) and enables K2 use AAD users as destination users in workflows’ client events or pull users details via Azure Active Directory Service broker and respective SmartObjects. 

When you use SharePoint Online with K2 you get your AAD authentication configured automagically (see, for example, this blogpost which run through the steps involved), and it really means that once you activated K2 App on your SharePoint online site – things gets configured automatically on K2 side. Simplicity of this integration emphasizes current product focus on clients which use SharePoint Online in their environments. 

Things are a little bit different when you do not have (or do not want to use) SharePoint Online in your environment, but still need AAD users to interact with K2. Good news – this is something you can do, bad news – current K2 documentation has not been updated to reflect AAD side UI and terminology changes and all in all it is quite lengthy manual configuration process. So vendor documentation is in place and you may follow it along but like I said current edition needs to be updated a bit, but no worries we will cover everything here, simplifying (when possible) steps involved and using modern Azure Portal UI. 

This article assumes that you have K2 4.7 installed and configured in your environment with on-premise AD or K2 SQL authentication and now just want to add AAD integration. 

Beyond installed and configured K2 blackpearl and K2 SmarForms our prerequisites are AAD subscription (this has been covered in my previous article) and SSL configuration of K2 web sites (as usual for test purposes you can get away with self-signed certificate, and this is also covered in K2 Installation and Configuration Guide). 

There are two big parts in setting up K2 and AAD integration: registering K2 SmartForms app in AAD and K2 side configuration of OAuth Resource and AAD label. 

Let’s start with registering K2 SmartForms app in AAD. In the real world you may not have access to AAD and somebody else will have to perform these steps for you. But, in any case, once app registered you will need certain values from registered K2 app which used to perform K2 side configuration. For the sake of convenience those will be written IN BOLD CAPITAL LETTERS and I suggest you create some table or text document which you can fill in as you go through AAD app set up part and then use it while performing K2 side configuration. 

Our first step is to log on to https://portalazure.com, select AAD domain and click on App registrations link under Manage

It will open App Registrations page where you need to click on “New application registration” button: 

This button starts Create app wizard where you are filling in application name, leaving application type on its default value of “Web app / API” and specifying Sign-on URL as shown below: 

Once app has been created you can see it in the list of your apps: 

If you click on it you can see your newly created app properties page: 

We now can further configure app and take a note of details we will later need to use on K2 side, for now from the Settings page copy and save your APPLICATION ID value. For additional properties and settings, use Settings pane on the right. For example, we can add customized logo for our app as shown below (that comes in handy if your list of registered apps is large, as logo helps you to find your app somewhat quicker in App Registrations list): 

While we still have app properties blade open we can adjust App ID URI replacing GUID which goes after K2 site part of URL and changing it to something less lengthy and easier to read for human. In my case I’ve changed it to “/identity/” (as you can see on the screenshot above). NOTE: This App ID URI must be unique within your AAD instance (UI won’t allow you to enter duplicate). Also on this page be sure to set Multi-tenanted option to Yes. 

Once again be sure to save APPLICATION ID into your app info table, and then click on Keys under API ACCESS section: 

There you need to type in description and select expiration time for the key, and click on Save to retrieve the key value: 

Once you click on Save, copy and save the key into table. Important warning: You will not be able to retrieve the key value once you leave this page so make sure you have it stored/copied: 

Once you copied and saved the KEY VALUE to table you can navigate away from this blade. As you can see on the screenshot above Azure UI additionally warns you that you will not be able to retrieve the key value once you leave this blade so make sure you that you saved it.  

If you navigate away from this blade and return to it you will see that key value is hidden and there in no options to retrieve it: 

Now from the app properties page click on Settings, then on Reply URLs and add two additional reply URLs (make sure to adjust {YourK2Server} value as necessary, in my case K2 server name was “k2.zlatagroup.com”, for example). So https://{YourK2Server}/runtime should be already here and you need to add the following:  

Token Endpoint Reply URL – https://{YourK2Server}/identity/token/oauth/2 

Authorization Endpoint Reply URL – https://{YourK2Server}/identity/authorize/oauth/2 

After this your Reply URLs list should look as on the screenshot below, and you can click on Save: 

Let’s also make sure that we have required app permissions. For that we need to navigate to Required permissions and then add/edit Windows Azure Active Directory permissions: 

Here we need either grant (Add button) or make sure (click on “Windows Azure Active Directory” permissions entry) and ensure that the following permissions are granted for K2 app: 

As a final touch we need to configure our AAD App to allow us to log in against all our K2 sites by modifying the app manifest file. Click on edit app manifest button: 

Within manifest file locate “identifierUris” section and append it so that it lists all your K2 sites, as shown on the screenshot below:  

Now we need to click on Endpoints button from the App registrations blade list: 

It will open Endpoints information page, from which you need to copy and save the following items: Federation Metadata Document URL, the OAuth 2.0 Token Endpoint, and the OAuth 2.0 Authorization Endpoint URL (the GUID section of all these URLs is your Directory ID). 

After saving endpoints information we need to obtain and store the Certificate Thumbprint. For that we need to navigate to Federation Metadata Document URL of our AAD using web browser and once XML is loaded search for X509Certificate value (within the Signing container) and copy it: 

You just copy entire line between <X509Certificate> and </X509Certificate> tags. Next Open a browser and browse to the online SAML fingerprint tool, paste the copied X509Certificate value into the form and make sure the sha1 algorithm is selected, then click the Calculate Fingerprint

Save FINGERPRINT VALUE. This concludes AAD app configuration part. After completing it you supposed to have this data: 

  • APPLICATION ID. Referred as Client ID in old Azure UI/K2 documentation. You can get it from app properties blade in AAD. Used as Authorization, Token and Refresh value of client_id OAuth resource property in K2. 
  • KEY. Key value which must be copied from the App Keys page at the stage of key creation. Used in all values of client_secret OAuth resource property in K2. 
  • METADATA URL. Can be copied from App Registrations > Endpoints page. 
  • OAUTH AUTHORIZATION ENDPOINT. Can be copied from App Registration > Endpoints page. Used when adding New OAuth Resource in K2. 
  • TOKEN ISSUER ENDPOINT. Can be copied from App Registration > Endpoints page. Used when adding New OAuth Resource in K2. 
  • DIRECTORY ID. GUID part which can be found in abovementioned endpoint URLs, or in Azure Active Directory Properties. Used as token value of resource_id OAuth resource property in K2.  
  • FINGERPRINT VALUE. Can be generated with SAML fingerprint tool from <X509Certificate> value of your app federation metadata document. 

Now we are done with AAD side app switch over to K2 Management site. As our first step here, we need to Register an OAuth Resource in K2. To do that we navigate to Authentication > OAuth > Resources and click on New button: 

This will open New OAuth Resource window where you need to type in resource name, select Microsoft Online as a resource type and enter Authorization and Token Endpoint values we recorded during AAD app registration. We leave Use Host Server Authorization Endpoint checkbox unchecked and click on OK button (see screenshot below). 

Our next step is to edit client_id resource parameter, which you can select from the list of Resource Parameters and then click Edit as shown below: 

Once client_id edit dialog has been opened we just need to paste APPLICATION ID we saved during AAD app registration in all three fields of this box, i.e. we use it as a value for Authorization, Token and Refresh: 

We next need to edit number of other parameters in the same fashion. We edit api_version resource parameter entering “1.0” for all values: 

Next, we edit scope resource parameter entering reader as Authorization Value as shown below: 

For all values of client_secret enter KEY we saved during AAD app registration: 

We specify https://graph.windows.net for all values of resource parameter: 

For redirect_uri we enter https://{YourK2Server}/identity/token/oauth/2 as Authorization and Token value: 

For entity_id parameter we entering DIRECTORY ID of your AAD instance as a token value: 

After all these edits you Resource Parameters for AAD resource should look like this: 

We now ready to add AAD security label in K2 (you can use something other than “AAD” as label name). For this we need to execute this SQL script or use sample script from K2 Installation and Configuration guide adjusting values for @OAuthProviderName and @SecurityLabelName variables. Be sure to back up your K2 database as you supposed to do before any direct modification of K2 database (I hope this is already your habit? If not try to form it 😊). 

Once script has been executed, we need to restart K2 service to get this new label initialized/picked up by K2. 

Our next step is to add new claim issuer from K2 Management site, as shown below: 

You need to specify the following values there: 

For Issuer value you specify https://sts.windows.net/{DIRECTORY_ID}/, for URI value you specify https://login.windows.net/{DIRECTORY_ID}/wsfed. As a Thumbprint value you need to paste your FINGERPRINT VALUE. Pay attention to trailing slash in Issuer URL – do not omit it. 

Next step is configuration of the Claim Mappings in K2. Claim mappings are used to identify the incoming claims and map them to the appropriate K2 security label. To do that we navigate to Claims, click New, select label and issuer we created earlier and fill in the form as shown on the screenshot below. We need to type in User and Group token identifiers and fill in all values under Identity Provider along with Original Issuer and Claim Type values under Identity section (see screenshot below). 

Now we need to configure the Realms and Audience URIs linking them with new issuer. For that, we navigate to K2 Management > Authentication > Claims > Realms and for each and every realm enable our AAD issuer, by means of selecting realm entry, clicking Edit and checking newly created issuer in Linked Issuers list: 

Hooray! We can now navigate to our SmartForms URL and select our AAD and logon with AAD credentials: 

Once label is selected you are redireted to Microsoft login page (https://login.microsoft.online.com) where you need to type in your AAD credentials and click Sign in: 

What you going to see next obviously depends on which K2 site you were trying to access. If it was K2 Designer then it will be opened without further questions in the default configuration: 

But in case you try to open K2 Management you are going to get insufficient permissions message: 

Which nudges us to perform our final configuration step. As we don’t have K2 Management rights we need to go and grant them, right? But as soon as you attempt to do it you will see that at this stage you are unable to do it: 

Error message tells us that OAuth token requires authorization and comes from K2 URM Service. It means that we need to obtain OAuth token for K2 service account (URM Service runs in the context of this account). To do that perform the following steps: 

1) Run SmartObject Services Tester (“C:\Program Files (x86)\K2 blackpearl\Bin\SmartObject Service Tester.exe”) in the context of your K2 service account using standard Windows “Run as different user” option. 

2) Now the tricky part. Unfortunately, now, in K2 4.7 SmartObject Services Tester only allows you to perform authorization redirect only when you are creating new instance (this possibly will be addressed in upcoming K2 Five release). Thus, to get our K2 Service account token instead of touching URMService we will simply register new Azure Active Directory service instance (and you may want to have it anyway): 

In Add Service Instance dialog we just switching Authentication Mode to OAuth, selecting our OAuth Resource (“AAD”) and typing https://graph.windows.net as OAuth Resource Audience value. We do not touch any other settings leaving them on default values as illustrated below. 

Once you click on Next, you will get this message (once again this message appears only when you are adding new instance, not editing existing one!): 

Once you hit OK browser window will be opened where you need to type in your AAD credentials. Important: you need an AAD user with Global administrator directory role for this action, otherwise you are going to see the following error: 

If you remember one of the rights we granted to our app is “Read directory data” and as it is a directory wide access only global admin AAD user can grant consent to this right: 

Once you type in your AAD global admin user credentials you just need to confirm that we are granting permissions mentioned above to the app: 

Once you click on accept you should be redirected on your K2 Identity site (you may get Windows credentials prompt at this stage – type in your K2 service account credentials) and see “Authorization Successful” message: 

This means that K2 service account token has been created and cached on your K2 server and you can see it in K2 Management > Authentication > OAuth > Tokens: 

3) You can now get back to SmartObject Services Tester and click on Next, Add to finish creation of AAD Service Instance: 

We now completed all the configuration steps and can grant rights to AAD users and use them in all K2 user pickers: 

It is good time to switch over to our Windows10 machine where we enabled AAD logon (see my previous blogpost) and look how it works there. 

First, we can conveniently see our K2 app in Access Panel application in our web browser. Just open https://account.activedirectory.windowsazure.com in your browser and without any credentials prompts (remember we are logged on to Windows 10 device with AAD credentials) you can see this: 

At this point I had realized that I have published K2 runtime site here, which is of little use as entry point for users, but you can correct this and publish either K2 Designer or K2 Management here. But in any case, from this Windows 10 machine we can navigate to K2 Management or Designer sites after just selecting AAD label and without typing in any credentials. 

I should say that the way this integration looks now in K2 4.7 is a bit inconvenient and lengthy. There are a lot of manual steps where you can easily make small mistakes which prevent your entire set up from working correctly, but with some experience you can do it. I think we have these manual configuration steps just because there was much more focus on SharePoint online AAD integration from K2 side and little attention to AAD without SharePoint online integration scenario. But in case a lot of clients will be asking vendor to simplify and better support this AAD only integration scenario, this can be improved and simplified in response to that. 

In any case I think this article gives a very good example of using AAD as an authorization provider for K2. I hope it was informative and useful and in case you have any questions you may leave them in the comments section below.