Script variables and custom fields in scripts

A script in the Scripts library can refer to data of the device it runs on and to custom field values. The values are filled in per device when the script is handed out, so one script serves hundreds of tenants without a copy per customer. A typical case is an antivirus installer that needs the customer's OU id: the id lives in a custom field on the tenant, the script reads it through a variable.

This guide covers reading values. Writing values back into custom fields from a script is covered in Write custom field values from a script.

Note: Script variables need web console and server version 3.2.0.4 or newer. Reading variables needs no agent update. Custom field values on tenant, location, group and global level, and the Secret field type, are described in Chapter 8.4.

Before you start

• Required permissions: collections_scripts_enabled plus collections_scripts_add or collections_scripts_edit for the script; collections_custom_fields_enabled to see custom fields in the editor's variable menu and in the preview.
• A custom field definition with at least one Manual field. A field that should be set once per tenant, location or group must be marked Inheritable in the builder.

Syntax

A variable is written as a token with a namespace and a key:

{{namespace.key}}

Spaces inside the braces are allowed. The key may contain letters, digits, underscores and hyphens. The six namespaces:

Fallback. A token may carry a default after a pipe: {{field.av_ou_id|ou-default}}. The fallback is used when the variable cannot be resolved and also when the resolved value is empty. When the same variable appears several times with different fallbacks, the first non-empty fallback is used. Without a fallback an unresolved or empty variable is an empty string.

Native names. Every variable also has an environment variable name of the form NL_<NAMESPACE>_<KEY>, for example NL_FIELD_AV_OU_ID or NL_DEVICE_NAME. A script may use that name directly in the shell's own syntax ($env:NL_FIELD_AV_OU_ID, ${NL_FIELD_AV_OU_ID}, os.environ['NL_FIELD_AV_OU_ID']) instead of a token. Both forms are resolved the same way and can be mixed. The name is derived from the key: the key is written in upper case and every character that is not a letter or a digit becomes an underscore, so the keys av_ou_id and AV-OU ID both map to NL_FIELD_AV_OU_ID.

What is not a variable. Only the six namespaces are recognised. Other double braces such as {{0}} in a PowerShell format string or a Python format escape are left untouched.

Device built-ins

The device namespace needs no setup. The built-ins and their native names:

The device's access key is not available as a variable.

{{device.hwid}} changes once on a Windows device when its agent is updated to version 3.3.0.1 and the server migrates the device to the new hardware ID format; see How the hardware ID is computed and stored.

Custom fields and levels

{{field.key}} names a custom field by its key. The key is the Key of the field in the custom field definition; the definition name is not part of the token.

Inheritance. The value is looked up in this order, and the first level that holds a value wins:

1. the device's own value,
2. the device's group,
3. the device's location,
4. the device's tenant,
5. the global value.

Levels 2 to 5 are only consulted for fields marked Inheritable in the definition. A field without the flag has device values only, as before. A level the device does not have (a device without a group) is skipped.

Explicit levels. {{tenant.key}}, {{location.key}}, {{group.key}} and {{global.key}} read exactly that level and nothing else, whether the field is inheritable or not. Use them when a script needs the tenant's value even though the device carries its own.

Which fields work. Fields with the data source Manual of type Text, Multiline or Secret. Only inheritable manual fields can hold level values; Job Result and SQL Select fields have no level values and are read from the device only.

Key rules and legacy keys. A key is a lowercase identifier: it starts with a letter and continues with lowercase letters, digits and underscores, at most 64 characters. Keys created before this rule keep working through their native name (AV-OU ID is reachable as NL_FIELD_AV_OU_ID); a key with a space cannot be written as a token. The Insert variable menu handles this for you and inserts the native reference for such a key. When two definitions share a key, scripts receive the value of the definition that was created first; the menu marks such keys.

Unknown keys. A token whose key no definition knows resolves to an empty string on every device. Saving a script with such a token shows a warning with the token names; the script is saved anyway. The server also writes a warning to its log each time the script is rendered.

What happens on the device

Values are never pasted into the script text. The server defines every referenced variable as an environment variable in one prelude line and replaces each token by the shell's reference to that variable. A value therefore stays data whatever it contains; a script has to execute it on purpose (Invoke-Expression, eval) before it can act as code.

Rules that follow from this:

• Single quotes do not expand. In PowerShell, Bash and Zsh a token works bare ($ou = {{field.av_ou_id}}) and inside double quotes ("OU: {{field.av_ou_id}}"). Inside single quotes the shell keeps the reference literally, so '{{field.av_ou_id}}' does not give you the value.
• In Python a token is an expression. Write ou = {{field.av_ou_id}} or print("Device: " + {{device.name}}). Inside a string literal the token is not evaluated.
• The value is always a string. {{device.id}} in PowerShell is the string "42", not a number. Cast it when you need a number.
• Values with line breaks and special characters are escaped so that the prelude stays a single line and the value arrives unchanged, including quotes, $, backticks and typographic quotation marks.

Where the prelude line goes. The line is not blindly line 1:

• PowerShell: after leading comments, #Requires lines, <# ... #> blocks, using statements, attribute lines such as [CmdletBinding()] and the param(...) block, so param stays the first statement.
• Bash and Zsh: after a shebang on line 1.
• Python3: after a shebang, leading comments including an encoding comment, a module docstring and from __future__ imports.

Because the prelude is exactly one line, line numbers in error messages are shifted by one. The script's line ending style (CRLF or LF) is kept.

Scripts without variables are delivered byte for byte as before. A script that names no token and no native name is not touched. Existing scripts do not change their behaviour.

Warning: A PowerShell script that uses param(...) followed by named blocks (begin, process, end or dynamicparam) cannot take a statement between param and the first block. The prelude line would be placed there and the script fails to parse. Do not combine variables with that script layout.

Where variables are resolved

Variables are resolved at the moment a script is handed to a device, always for exactly that device:

Jobs and sensors are rendered when the server builds the device's policy; the script stored in the library, in the job and in the sensor keeps its tokens. Remote shell and API calls render at send time.

Not rendered: System scripts (MySQL), the CMD shell of the real-time terminal, Software Deployment and App Hub install scripts, patch management, and text fields that are not scripts (a sensor's expected result, job names, MOTD messages, webhooks).

When resolution fails. A database error while resolving is handled per path: a policy keeps the tokens in the script and the server logs an error with the job or sensor name; the remote shell and the live test refuse to send and show an error; the public API answers 500 (a bulk entry is marked dispatch_failed). An empty value because of an unknown key is not a failure; see above.

Secrets

A field of type Secret is used in a script like any other field: {{field.av_token}}. The script does not see where the value comes from and receives the plain text. Keep these points in mind:

• Whoever can run a script on a device can read every value that device receives. That includes secrets. The audit entry of a remote shell, bulk shell or live test dispatch lists the names of the referenced variables and, separately, the names of the referenced secret variables, never the values; the stored command text is the unrendered script. The public API likewise keeps the unrendered text in the action record.
• Do not print a secret. Script output lands in events (including the Remote Shell event of a classic remote shell command) and in API results.
• The preview masks secrets for everyone, also for accounts that hold the reveal permission.
• The real-time terminal keeps the rendered script in a file while the session runs; see Write custom field values from a script. With a Remote Agent older than 3.2.0.3 a template is typed into the terminal line by line and echoed on screen, so the console refuses to send a template that refers to a secret field to such an agent and says why. Update the agent to 3.2.0.3 or newer for scripts that use secrets in the terminal.
• A field type changed from Secret to Text keeps its stored values encrypted; scripts then receive the ciphertext. Enter the values again after such a change.
• Sensor events no longer carry the script body. Server 3.2.0.4 removes it from the events of older agents as well, because a rendered sensor script contains the resolved values.

Changed values and the policy re-sync

A device fetches its policy again only when it is marked for a re-sync. Every write path marks the affected devices:

• a device value marks that device,
• a group, location or tenant value marks the devices of that level,
• a global value marks every device.

The next check-in of the device brings the policy with the new values; jobs and sensors use them from then on. A value written back by a script marks the device only when the value actually changed. The remote shell and the public API render at send time and always use the current values.

The editor: Insert variable and Preview for device

The Add Script and Edit Script dialogs (Collections → Scripts) carry two helpers for variables.

Insert variable opens a searchable menu above the editor and inserts the chosen token at the cursor. The menu has four groups:

• Device built-ins: the table above, with the native name of each entry.
• Custom fields: every manual field of every definition, with its definition name, native name and a secret or inheritable chip.
• Explicit levels: for every inheritable field the four tokens tenant., location., group. and global..
• Write back to custom fields: the helper call per shell; see Write custom field values from a script.

Each entry has a copy button for its native name. A key that does not follow the naming rule is marked and inserts the native reference in the syntax of the selected shell instead of a token. A key that also exists in an earlier definition is marked, with the name of the definition whose value scripts receive. For the System platform the menu is disabled, because MySQL scripts are not rendered.

Preview for device in the dialog's button row renders the current editor text for a device of the script's platform, online or not, and shows:

• a table of the referenced variables with token, native name, value and source (Built-in, Device, Group, Location, Tenant, Global, or None when no level holds a value), with an Unknown key chip where no definition knows the key;
• the prelude line;
• the rendered script, with a copy button for both.

Secret values are masked. An account without collections_custom_fields_enabled sees the built-ins resolved and every field reference with an empty value and the source Hidden. For a script without any reference the preview says so and shows the text unchanged; for a MySQL script it says that scripts of this shell are delivered unchanged.

Saving. When the script names a key no definition knows, saving shows a warning listing the tokens. The script is saved.

Example: install an antivirus product with the tenant's OU id

The customer's OU id differs per tenant, the installer URL is the same for everyone.

1. In Collections → Custom Fields, add a definition with a section holding two fields, both of type Text with data source Manual and Inheritable enabled: key av_ou_id (label AV OU id) and key av_installer_url (label AV installer URL).

2. Open Settings → Custom Fields, tab Global values, and set av_installer_url to the download URL.

3. For each customer open the tenant's Tenant Settings page, section Custom Fields, and set av_ou_id to the customer's OU id.

4. Create a Windows / PowerShell script:

   # Installs ExampleAV with the tenant's OU id when it is not installed yet.
   $ou = {{field.av_ou_id}}
   if ([string]::IsNullOrWhiteSpace($ou)) {
       Write-Output "No OU id configured for tenant {{device.tenant}}; nothing to do."
       exit 0
   }
   
   $installed = Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*',
       'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*' -ErrorAction SilentlyContinue |
       Where-Object { $_.DisplayName -like 'ExampleAV*' }
   
   if ($installed) {
       Write-Output "ExampleAV is already installed on {{device.name}}."
       exit 0
   }
   
   $installer = Join-Path $env:TEMP 'exampleav.msi'
   Invoke-WebRequest -Uri {{global.av_installer_url}} -OutFile $installer
   Start-Process msiexec.exe -ArgumentList "/i `"$installer`" /qn OU_ID=`"$ou`"" -Wait
   Write-Output "ExampleAV installed with OU $ou."

5. Click Preview for device, pick a device of the customer and check that av_ou_id shows the tenant's value with the source Tenant.

6. Save the script, wrap it in a recurring job and attach the job to the customers' policies as described in Write, test, and schedule a PowerShell script.

Delivered to a device of the tenant Contoso, the script starts like this:

# Installs ExampleAV with the tenant's OU id when it is not installed yet.
$env:NL_FIELD_AV_OU_ID = 'ou-4711'; $env:NL_DEVICE_TENANT = 'Contoso'; $env:NL_DEVICE_NAME = 'PC-01'; $env:NL_GLOBAL_AV_INSTALLER_URL = 'https://downloads.example.com/exampleav.msi'
$ou = ${env:NL_FIELD_AV_OU_ID}
if ([string]::IsNullOrWhiteSpace($ou)) {
    Write-Output "No OU id configured for tenant ${env:NL_DEVICE_TENANT}; nothing to do."
    exit 0
}

A device with its own av_ou_id value uses that value instead of the tenant's; a device in a tenant without a value skips the installation and says so in its job result.

Troubleshooting

• The token appears literally in the output. The token is inside single quotes, or the script runs through a path that is not rendered (a CMD terminal, a System script, a deployment script).
• The value is empty on every device. The key is unknown or misspelt. Open the script and save it: the warning names the tokens no definition knows. For a legacy key with spaces or upper-case letters, use the native name from the Insert variable menu.
• The value is empty on some devices. Those devices, their groups, locations and tenants hold no value and there is no global value, or the field is not marked Inheritable. Check the effective value on the device page or with Preview for device.
• A job still uses the old value. The device has not checked in since the value changed. Wait for the next check-in or force a sync from the device page.
• A PowerShell script fails to parse after adding a variable. See the warning about param with begin/process/end blocks above.

Related

• Write custom field values from a script — the write-back helpers, limits and examples.
• Chapter 8.4 — Custom Fields — field types, keys, levels and secrets.
• Chapter 8.1 — Scripts — the Scripts library and the editor.
• X.9 — Public API — custom field values per level and run-script over the API.