Quick hit: Dynamic Where-Object calls

On occasion, you might want to build up a call to Where-Object that changes based on your runtime environment.  Perhaps you have to iterate over a huge collection, or you have an expensive statement to evaluate that doesn’t need to run in all scenarios.  This post will illustrate how to build up a dynamic call to Where-Object using example code from Get-Type.

The ScriptBlock

So we want to modify what runs in Where-Object.  We know the standard call is Where-Object {<# Something #> }, so we’ll dive into the help to find out what parameter that scriptblock is.  We want to find a parameter with Position 0 or 1 that takes a ScriptBlock – we dive in and find this is the FilterScript parameter:

Get-Help Where-Object -Full

Where

Now, we want to create the scriptblock for this parameter dynamically.  If we search around, we might find that you can convert a string to a scriptblock using the create method from the System.Management.Automation.ScriptBlock class.  It sounds complicated, but the code is pretty straightforward:

$ScriptBlock = [scriptblock]::Create( $String )

Okay!  At this point, we know what parameter takes in the scriptblock, we know how to create a scriptblock from text, and hopefully, we know how to work with strings.

Putting it all together

In Get-Type, we provide a few parameters to allow filtering on the returned types.  If these are set to *, we don’t want to evaluate them to the where clause.  If they aren’t set to *, we want to add a statement to the where clause.

There are many ways to skin this cat; we’re going to build up an array of statements and join them with –and.  You can build your strings as desired.

#Build the Where array            
$WhereArray = @()            
            
#If anything but the default * was provided, evaluate these with like comparison            
if($Module -ne "*"){$WhereArray += '$_.Module -like $Module'}            
if($Assembly -ne "*"){$WhereArray += '$_.Assembly -like $Assembly'}            
if($FullName -ne "*"){$WhereArray += '$_.FullName -like $FullName'}            
if($Namespace -ne "*"){$WhereArray += '$_.Namespace -like $Namespace'}            
if($BaseType -ne "*"){$WhereArray += '$_.BaseType -like $BaseType'}            
            
#Build the where array into a string by joining each statement with -and            
$WhereString = $WhereArray -Join " -and "            
            
#Create the scriptblock with your final string            
$WhereBlock = [scriptblock]::Create( $WhereString )

At this point, we have the scriptblock created!  If we call Get-Type with –Verbose, we can see what the scriptblock looks like depending on the parameters we call at run time:

Where2

That’s about it!  We illustrated how to build up a scriptblock dynamically and used it with Where-Object – keep in mind you could use this for other scenarios where you need to build a scriptblock up in pieces.

Edit: A quick follow-up – There are situations where your scriptblock really needs to be dynamically generated.  The example itself did not need it for performance or functionality; simplicity and clarity of code would generally take priority, I was just curious.

Advertisements

Credentials and Dynamic Parameters

Everyone has their preferred way to simplify credential handling in PowerShell.  Here are some of my favorites.  Before using these, consider your security policies and posture.

Import and Export PSCredentials

Many functions and examples out there simply serialize the encrypted password to disk, leaving you to handle the username.  Years ago, Hal Rottenberg wrote two handy functions that serialize and deserialize both the username and password; Import-PSCredential and Export-PSCredential.  The links are to very slightly modified functions.

Export-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.contoso.cmonster.crd" 

$credCMonsterContoso = Import-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.contoso.cmonster.crd"

Wait, isn’t that insecure?

There are a few considerations to take into account, but this isn’t as risky as you might expect.  Serializing the password to disk uses the Windows DPAPI to encrypt your password, limiting decryption to your account, on the computer you encrypted the password from.  Here are two considerations that immediately come to mind:

  • I don’t know of any exploits that can decrypt these files.  Might these already exist?  Might we find a vulnerability and see exploits down the line?  Perhaps.  This risk should be acceptable in most organizations, given password entropy, compensating controls over where these credentials are stored, and other factors.
  • Other processes on this system running as your account could access these credentials.  Dave Wyatt discusses a workaround using secondary entropy.

I’m personally comfortable using methods Lee Holmes describes in PowerShell Security Best Practices.  If in doubt, consult your security team.

Dynamic parameters

Dynamic parameters are parameters that are generated at runtime.  They can be both handy and painful.  The basic idea is that you can dynamically generate parameters depending on the runtime environment.  A few quick resources:

Why are we talking about dynamic parameters?  How are these related to credentials?

Serializing and deserializing credentials to disk is quite handy, but we can take this a step further.  If you don’t have a password management solution with an API, working with passwords can be quite tedious.  We’re going to devise a system where you keep PSCredentials stored in variables, with simplified copy-to-clipboard access via dynamic parameters.

Wait, isn’t that insecure?

Yes.  Copying any confidential data to the clipboard is risky.  Much riskier than relying on the DPAPI.  That being said, information security is about managing risk, not completely eliminating it.  Perhaps you would consider using this on a secured system where you don’t do much day-to-day browsing or other risky activities, and with a certain class of accounts.

Simplified credential management

We’re going to cover three steps; encrypting the credentials (one time, and after any changes), getting the credentials into your session, and a copy-password function.

Prerequisite:  Download and get the dependency functions into your session before using them.

# Load dependencies.            
    . "\\Path\To\Import-PSCredential.ps1"            
    . "\\Path\To\Export-PSCredential.ps1"            
    . "\\Path\To\New-DynamicParam.ps1"

Encrypt credentials using Export-PSCredential as desired.  You only need to do this one time, and any time the credentials change.

# I name mine COMPUTER.CURRENTUSER.[domain.]USER[Qualification as needed] to help identify where I can use them and what accounts they cover.            
# Access to decrypt these is limited to the user that exported them, on the computer they were encrypted on            
# Consider storing these in a secured location.  These are on my D:\ for illustrative purposes only            
            
Export-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.contoso.cmonster.crd"            
Export-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.cmonster.crd"            
Export-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.contoso.TestUser.crd"

Now, any time you want to access these, pull them into your session.  You could put these in your profile so they are always available, or use them in a script that needs credentials.  Don’t forget to dot source Import-PSCredential function beforehand.

# Import credentials we previously exported.  I'm using names starting with 'Cred'            
    $CredCMonsterDomain = Import-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.domain.cmonster.crd"            
    $CredCMonster = Import-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.cmonster.crd"            
    $CredTestUserDomain = Import-PSCredential -Path "D:\$ENV:COMPUTERNAME.$ENV:USERNAME.domain.TestUser.crd"

Now I can use these credentials as desired:

Cred1 Cred2

This is great for scripts, but if I want quick access to a password in an interactive session, typing this out is tedious.  Let’s write a function to quickly extract passwords from these PSCredentials:

function Copy-Password             
{            
                
    [cmdletbinding()]            
    param()            
    DynamicParam            
    {            
        $Variables = Get-Variable -Name Cred* | Select -ExpandProperty Name            
        New-DynamicParam -Name Credential -ValidateSet $Variables -Mandatory -Position 0            
    }            
    Begin            
    {            
        $Credential = Get-Variable -Name $PSBoundParameters.Credential -ValueOnly            
        $Credential.GetNetworkCredential().Password | Clip            
    }            
}

Now if I have test or other credentials that I need to use very regularly, I have a simple way to get them into my session and to extract the plaintext passwords.

Creds

You could take this a step further. In the DynamicParam block, perhaps you could get all variables that are PSCredentials, using the -is comparison operator.

Get-Variable | Where-Object {$_.Value -is [PSCredential]}            

Another method would be to create the credential objects using New-Variable, with a specific description we could filter on later.

That’s about it! Keep an eye out for other resources as well.  For example, BetterCredentials from Joel Bennett offers a more functional drop-in replacement for Get-Credential. Consider writing your own functions tailored to your needs and environment.

Cheers!

Exploring PowerShell Objects: Flattening

Hidden Treasure

Knowing how to learn and explore in PowerShell is very important.  Jeffrey Snover and other experts often mention that they might not memorize exactly what to run, but they know how to use tools for discovery and exploration built into PowerShell and the .NET Framework.  Tools like Get-Command, Get-Help, Get-Member, and Select-Object go a long way.

Two of PowerShell’s greatest features can make it somewhat tedious when first working with and exploring a particular technology.

  • PowerShell can automate and control a wide variety of technologies
  • PowerShell is an object based language

Let’s look at an example.  Pretend you are working with a process object for the first time!

Get-Process -name PowerShell*

image

Not much info, is there?  What if I want the path to the process?  Details on module file names?  I would need to use Get-Member and Select-Object for this, or perhaps Lee Holmes’ excellent Show-Object.

Whichever path, none of these provide a quick outlook on all the properties and values of an object.

The Problem

Hopefully, none of us are stuck solely working with process objects. Maybe you work with the various technologies PowerShell can hit; .NET libraries, numerous RESTful or other web APIs, each of which might expose a variety of different objects. Exploring the data and schema behind these is very helpful; maybe you can use it in building a solution, or to help learn and pick up a new skillset.

Show-Object is a great way to explore these – it takes an object and breaks it down into a tree that you can explore. Nine times out of ten, this will do the trick. But what if you want to include or exclude specific nodes?  Search for a particular value of a node? Exclude the 20+ default properties that make exploring XML so painful? Here’s Show-Object with an empty XML document:

ShowObject

Wouldn’t it be nice to explore objects like this without the unnecessary properties?

Flattening

ConvertTo-FlatObject is a function that neatly fits these scenarios.  Rather than presenting a tree, it attempts to flatten the entire object, leaving you a usable object as output.  Let’s start with a concocted example.

$Object = New-Object -TypeName PSObject -Property @{            
    "A-1" = 1            
    B = $(Get-Date)            
    C = @{'HashTableKey!' = "val"}            
    D = @(1..2 | ForEach-Object {            
        New-Object -TypeName PSObject -Property @{            
            $_ = "a$_"            
        }            
    })            
}             
            
$Object | ConvertTo-FlatObject

FlatObject1

ConvertTo-FlatObject won’t handle everything, but it will recursively list out the properties of an object, including hashtables, arrays, and paths with special characters. The output is formatted so that you can copy and paste properties for future use:

FlatObject2

Who would design such a silly schema?  As you spend time with code, it becomes clear that complexity is the norm.

Practical Examples!

Commvault

Let’s play with a real example.  Commvault Simpana provides a RESTful API.  We’ll use slightly modified XML from here.  Perhaps we want to take this mess of XML and expand out any leaf that ends in “ID”, and we don’t care about common properties:

$Object = [xml]'<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
 <subClientProperties>
 <commonProperties enableBackup="true" description="compliance" encryptionFlag="ENC_NETWORK_AND_MEDIA" fsFLRReplicationOptions="540672" isKeepItemPastExpiryTime="true" numberOfBackupStreams="2" readBuffersize="256">
 <prepostProcess runPostBackup="NO"/>
 <snapCopyInfo>
 <snapToTapeProxyHost _type_="CLIENT_ENTITY" clientId="2"/>
 </snapCopyInfo>
 <storageDevice applicableReadSize="4096" networkAgents="2" softwareCompression="USE_STORAGE_POLICY_SETTINGS" throttleNetworkBandwidth="-1">
 <dataBackupStoragePolicy _type_="STORAGE_POLICY_ENTITY" storagePolicyId="7"/>
 <deDuplicationOptions enableDeduplication="true" generateSignature="ON_CLIENT"/>
 </storageDevice>
 </commonProperties>
 <content path="C:\\DOC_folder"/>
 <fsSubClientProp backupSystemState="false" backupSystemStateforFullBkpOnly="false" retentionRule="IMMEDIATELY" useGlobalFilters="OFF" useVSS="true"/>
 <impersonateUser/>
 <subClientEntity _type_="SUBCLIENT_ENTITY" applicationId="33" backupsetId="3" clientId="2" instanceId="1" subclientId="6"/>
 </subClientProperties>'            
            
$Object | ConvertTo-FlatObject -include *ID -Exclude commonProperties

FlatObject3

Rather than exploring each Commvault object I care about by hand, I can now quickly get a feel for the data and schema behind them.

StackOverflow

StackExchange has a nice public API we can hit with Invoke-RestMethod.  What data do we get back?

$Uri = "https://api.stackexchange.com/2.0/questions/unanswered?"
$Uri += "order=desc&sort=activity&tagged=powershell&pagesize=2&site=stackoverflow"            
            
Invoke-RestMethod -Uri $Uri | ConvertTo-FlatObject

FlatObject4

Active Directory

Sometimes I know the value of what I want, but not the property name.  Sure, I could manually skim through all the properties.  But this is slow, and I have bad eyes.

I know I have an alternate e-mail address… where is that stored again?

Get-ADUser REDACTED -Properties * |            
    ConvertTo-FlatObject -Value "*cookie.monster*" -MaxDepth 1

FlatObject5

On a side note, that MaxDepth 1 saves a bit of time if you know you only want to see the first layer of properties.

Under the Hood

So what are we using to flatten these objects, ignoring a non-developer’s sad attempt at recursion?

Reflection

PowerShell uses the .NET Framework.  This means we have access to System.Reflection.Assembly.  One handy method from this class is GetType.

Refl

You can run this on any object in PowerShell.  So, how does this come up in ConvertTo-FlatObject?  Here’s an illustration with XML, one of the main reasons I use this method:

Refl2

In ConvertTo-FlatObject, we use GetType to get the type we are working with, and then GetProperties to find the default properties that we may want to ignore.

PSObject

You might have read various posts on inserting a type name into a custom object.  These use the hidden property PSObject, which has a handy Properties property, allowing for ordered extraction of properties and their values.

Here’s a quick example getting properties of a DirectoryInfo object:

image

In ConvertTo-FlatObject, we use this to extract property names and values.

Back to Basics

Don’t forget the basics!  Tools like Show-Object and ConvertTo-FlatObject are convenient, but you should still spend time getting to know how to learn and explore in PowerShell.

I can’t imagine a day going by where I don’t depend on Get-Command, Get-Help, Get-Member and Select-Object.  Get very familiar with these Cmdlets!