How much time can Easy Tune really save?

Too often companies make vague “saves you loads of time” sorts of claims about their products, without quantifying exactly how the product will save you time and how much time. I realized we have been guilty of this in the past, so I’m here to set the record straight! In this blog we will quantify exactly how much time (and effort) Easy Tune will save you:

“Easy Tune will save you 98.4% of the time you would spend tuning SCOM natively”

“Easy Tune takes just 7% of the clicks of the native SCOM UI to set the same tuning”

The premise & environment

I want to tune the SQL version agnostic management pack by turning off everything that isn’t a discovery globally, then tune a specific group containing production SQL servers to the MPs defaults and then tweak 4 thresholds for 5 specific problem databases.

Our management group, made up of two management servers, is lightly loaded (as it is in our test lab), is running SCOM 2019 UR2 and is hosted in Azure (East US2). The main management server is of the below spec:

The second management server is the same size but with 4gb ram. SQL is hosted on a separate server from SCOM, is not clustered/highly available and is of size “Standard B4ms” with 4 vCPUs and 16GB ram. It hosts SCOMs ops DB and data warehouse DB (and DBs for other things too) and was lightly loaded.

The specific management pack in question:

Setting up this tuning with native SCOM tooling

For this test I will use the SCOM console to set a single override, I will count the clicks from loading the Authoring pane and time how long it took to apply the override using a stopwatch. I will then figure out how many overrides would need setting to setup the desired overrides and extrapolate out the result, so here goes…

Timing setting a single override

Each point below represents a click, I used a monitor with the scope pre-set to the correct MPs to reduce some clicks. I also pre-created an MP to store my override in (again, to reduce the clicks).

1. Expand out the target header

2. Expand “entity health”

3. Expand “availability”

4. Right click the monitor to navigate to Override the Monitor

5. Select “For all objects of class: MS SQL on Windows: Database”

6. Click a parameter to override in the “Override” column

7. Click the parameter override value field and select the desired value

8. Click the Management Pack drop down

9. Select an MP to store the override in

10. Click OK

I started my stopwatch just before starting with click 1 above and stopped it when the “override properties” dialogue box had disappeared (I.E when the console became usable again) - it took 31 seconds end-to-end and I was trying to be quick about setting my override.

Calculating the time it would take to complete tuning manually

Assuming it took 31 secs to tune each override lets look at how long it would take to tune everything in this scenario.

Globally

We had planned to turn everything off that isn’t a discovery so the next thing to do is understand how many rules/monitors, etc there are in the MP. The easiest way to do this is to use Easy Tune which captures all the workflows in the selected MPs to a tuning pack stored in CSV format. In the tuning pack captured for the MPs I filtered out discoveries and all workflow parameters which aren’t involved in enabling the workflow (so parameters relating to alerting behavior, thresholds, etc).

There are 552 workflows I would need to disable to setup the desired tuning globally:

To tune all these workflows would therefore take 31 secs multiplied by 552 = 17,112 secs (or more helpfully 4 hours 45 mins and 12 secs). With 10 clicks to tune a workflow the math is nice and easy - 5520 clicks are needed… so be careful not to miss click!

That is a lot of time! and if you were to really do this in reality I guarantee it would take longer as you aren’t 100% accurate, don’t operate like a machine and you are going to need a break! So my guess is this would take you a whole working day!

Tuning the production group

As we have made no changes globally but to tune off 552 workflows for the production SQL server groups tuning in this scenario, all we would need to do is turn all 552 workflows back on. We know how long it took to turn them off globally the time needed to switch them all back on should be similar, but as we are applying tuning to a group there are a few extra clicks in the process as we need to find the group to apply overrides to.

In my testing, tuning a group added a few extra keystrokes (typing the group name), some scrolling (finding the group in the list) and 2 extra clicks (selecting the group then clicking OK). When I re-ran my stopwatch test, as before, I found setting an override for a group took 38 secs (in case it matters, I was typing “SQL” and scrolling to find the group “SQL Instances”).

With a workflow taking 38 secs to override, and 552 workflows to override, tuning my group would take 20,976 secs… or 5 hours 49 mins and 36 secs. With 12 clicks per override this comes to a whopping 6624 clicks (and some group name typing!)

Tuning an object

The good news here is because we are just tuning a single object, things should be much simpler and quicker. I want to tune specific performance monitors (see below) on some troubled databases, to decrease thresholds so we get an earlier warning when things are going wrong.

I timed the amount of time/clicks it takes to tune one of these thresholds (In-Memory OLTP Data Free Space Left) - 37 secs and 8 clicks. This means for each database we want to tune in this way will take 2 mins 28 secs / 32 clicks. So for five databases this comes to 12 mins and 20 secs / 160 clicks.

Totals for manual tuning

To pull this all together we have:

• Global tuning: 4 hours 45 mins and 12 secs / 5520 clicks

• Group tuning: 5 hours 49 mins and 36 secs / 6624 clicks

• Object tuning: 12 mins 20 secs / 160 clicks

Grand total: 10 hours 47 mins and 8 seconds / 12,304 clicks. That is a hell of a lot of time! but to complete the picture we need to compare it to how long the same task would take you with Easy Tune.

Setting this tuning with Easy Tune

When attempting to setup the tuning in Easy Tune there are a few pre-requisites, Easy Tune itself (free MP from here) + a Tuning Pack for the MPs we want to tune. In this case there is a tuning pack for the SQL version agnostic MP available from the Community store, so for the Global and Group tuning we have very little in the way of pre-req setup.

Global Tuning

The steps taken to tune globally are below with each step representing a click. I started my timed test when I had loaded the SCOM console and was in the Administration pane:

1. Click “Tuning packs” under “Easy Tune - Cookdown” in the administration pane navigation bar

2. Click the SQL version agnostic tuning pack from the community store

3. Click Tune Globally

4. Click Next on the scope screen

5. Click on the drop down to select a level

6. Click Discovery Only level

7. Click Next

8. Click Next (I accepted the default location on where to store the overrides - in a new auto created MP)

9. Click Finish on final screen of the wizard and wait.

Any guesses on how long this whole process took including the wait after going through the wizard (I.E when the overrides are actually applied)?

It took just under 5 hours to do manually and there are a large number of overrides, but we are doing it programmatically here - 1 hour? 30 mins?… wrong!

The whole process took just 39 seconds / 9 clicks!

Group tuning

The process for group tuning is very similar to that of global tuning except in step 4 for Global tuning we have to pick the scope of tuning (I.E pick the group to tune). This adds in 4 extra clicks (as you click a browse button, type a group name, click the group and click OK). There is also an extra screen in the wizard in which you can set a schedule for the tuning - so 13 clicks in total.

Once again I timed the process from the same starting point as with Global tuning above - this time it took 2 mins 25 secs / 13 clicks. The majority of this time was spent in setting the overrides rather than driving the UI, as the MP default level in any tuning contains values for all aspects of each workflow (E.G the thresholds, alerting properties, enabled/disabled, etc). Easy Tune will have been setting more overrides than in the manual case (probably x4 more) - if I had created a custom level containing only override properties of Enabled I predict things would have taken just over a minute, but compared to the nearly 6 hours it would have taken you to do this with the manual case, it only took 2 mins 25 secs, pretty reasonable!

Object tuning

For the object tuning we need to create a custom level in our tuning pack containing only the workflows to override, so we need to first copy the community store tuning pack locally and add in these overrides. We then need to go through the specific object tuning wizard for each individual object to be tuned.

The steps involved to copy the community tuning pack locally and add in our custom level are below (with each bullet representing a click). I started in the Easy Tune UI:

1. Click “Copy to Custom Store (Make Editable)”

2. Double click the copied tuning pack to open in excel

3. Right click the column header (in excel)

4. Click Insert new column (this should be between the “DefaultValues” column and the “Notes” column)

5. Give the column a name (this will be the level name in Easy Tune)

6. Find each workflow to override and type out the override value into the column you just created

7. Save and close the CSV file

8. Refresh the Easy Tune UI so the new level is picked up

This process took 3 mins 29 seconds and 90% of the time was spent in the excel file itself finding the right workflows to override. Excels filtering made the process much easier - I filtered out all workflows that where’t of type “monitor“ and filtered out all Override parameters that didn’t contain the word “Threshold” then used excels search to find the workflows by typing their name.

Next up timing setting my newly created level for a specific object - the steps and number of clicks is identical to setting Group level overrides except there is no schedule step for object level overrides (and obviously you are scoping your tuning to an object rather than a group). This brings the total number of clicks to 12.

Tuning a single object took 47 seconds.

If we multiply out 47 seconds / 12 clicks per object to be tuned (five in total) we get 3 mins 55 secs / 60 clicks, then add on the 3 mins and 29 seconds / 8 clicks we spend adding the extra level to the tuning pack and we get 7 mins and 24 secs / 68 clicks to tune all 5 databases.

If I am honest, I was expecting object tuning using Easy Tune to take a similar amount of time to using the SCOM native UI and to be writing about Easy Tune’s other benefits & savings in time later down the line … but object level tuning was still quicker than the 12 mins 20 secs / 160 clicks it would have taken without Easy Tune!

Totals tuning with Easy Tune

To pull this all together we have:

• Global tuning: 39 secs / 9 clicks

• Group tuning: 2 mins and 25 secs / 13 clicks

• Object tuning: 7 mins 24 secs / 68 clicks

Grand total: 10 mins and 28 seconds / 90 clicks. Enough said.

What did we learn today?

I am not sure I really need to write this section.. but here goes.. Easy Tune is MUCH quicker! Just look at the numbers:

• SCOM native UI: 10 hours 47 mins and 8 seconds / 12,304 clicks

• Easy Tune UI: 10 mins and 28 seconds / 90 clicks

• Easy Tune requires just 7% of the clicks that native SCOM UI requires

• Easy Tune takes just 1.6% of the time to achieve the same results as the SCOM console

While there are many things that will have an effect on the time it takes you to do this much tuning in production (environment speed, coffee breaks, other interruptions, etc), the % difference will hold true across the board.

It is also worth noting that there are other approaches to tuning in batch such as scripting against the SCOM SDK, custom MP authoring and other third party tools like Silect’s MP Studio. However, they all have significant drawbacks when compared with Easy Tune. Scripting and custom MP authoring all require additional skills and are still not likely to be quicker (even if you use fragments/existing MPs as templates/template scripts) and carry other forms of cost (custom MP authoring will likely cost you a Visual Studio license, Silect’s MP Studio while a great product is not free).

Easy Tune makes mass tuning available to all SCOM administrators and is free. It should be installed in all SCOM management groups. We consider it a fundamental right for the SCOM administrator. Like Coffee.

Virtual mic drop. Bruce out.