# DTS Expression Language (DEL)

DTS Expression Language (DEL) is used to write DTS expressions, which populate (or *resolve*) values in declarative DTS functions.  This enables runtime deployment of the programs as data (rather than traditional build-release pipelines) and takes advantage of the DTS engine's parallel evaluation and shared cache capabilities.

Vendors and Acronis entities alike may contribute flexible business logic used to construct objects composed of data from various API requests and transformations.  DTS expressions are a safe and efficient alternative to imperative Go code, which may only be contributed directly by Acronis and requires careful review and testing to ensure correctness.

DTS functions may be implemented using Go where necessary, but we encourage the use of DTS Expression Language where possible due to its advanced features, security, and ease of use.  Furthermore, declarative DTS functions may be deployed at runtime as data from Application Manager (i.e. via normal CyberApp deployment process) without requiring the compilation and release of a binary via traditional deployment channels.

DEL is heavily influenced by functional programming styles.  In fact, it works similarly to many classic functional programming languages such as Lisp or Haskell, where the program is represented as a single (possibly nested) function invocation with no persistent variable assignments.  However, DEL also works with the idea that these individual expressions being composed to produce an object as an output, possibly constructing and referring to a context and parameters to provide variable values to resolvers.

# Motivation and Concept for DEL

DEL is not intended to be a general-purpose programming language, so experienced developers approaching it for the first time may not fully appreciate the role and purpose it is intended to serve.

DEL is the result of multiple generations of evolution on the concept of templating objects.  In many systems, (such as Ansible), structured data (such as YAML) is constructed using text templating.  This is a powerful concept for declarative "configuration as code", but the representation mismatch of working with structured data as text is generally not a computing best practice.

Declarative DTS functions are intended specifically to construct objects for various purposes.  In particular, these objects may be presentation objects which require localization and user-specialization (including business logic based on user characteristics), and/or they may composite data from multiple backend APIs (including business logic that "mashes up" the different APIs).

Declarative DTS functions, and by extension DEL, are not intended to implement complex algorithms or directly interact with the underlying system environment.  They are specifically limited and sandboxed to allow contribution by vendors and to ensure resilient performance in critical scenarios.

# Basic DEL Expressions

A DEL expression is a a string that starts with a `$` character, and contains a resolver expression.  A single resolver espression may use arguments that are either literal or a nested DEL expression.

## Simple Examples

`$.context.message` is a DEL expression that directly references the value of the `message` property of the `context` object.

`$func[cti.a.p.dts.func.v1.0~a.example_project.greeting.v1.0](first_name='Geoffry', last_name='Fitzsimmons')` is a DEL expression that invokes a function identified by CTI ID, providing `first_name` and `last_name` arguments.

`$cel(1 > 2)` is a DEL expression that evaluates to false, and represents the application of logical expressions.

# DTS Expression Language Syntax

This guide explains how to use DTS Expression Language, including syntax rules, variable resolution, and common operations.

## Syntax Rules

A DEL expression may be expressed using the following notation:

$<identifier>(<args|kwargs>)

Where:

- <identifier> - an alphanumeric name of the value resolver.
- <args> - an ordered list of positional arguments.
- <kwargs> - an unordered list of named arguments.

For example:

* $cel(true) - invokes a `cel` resolver and accepts `true` as a first positional argument.
* $trans(key='dictionary-key') - invokes a resolver by name `trans` and accepts a named argument `key` with a string literal 'dictionary-key'.

Some resolvers may support indexing by CTI using the `[]` notation. For example:

* $func[cti.a.p.dts.func.v1.0~x.y.fn.v1.0](arg1="val", arg2=1) - invokes a resolver by name `func` that finds the CTI object with ID `cti.a.p.dts.func.v1.0~x.y.fn.v1.0` and accepts a list of named arguments (as defined by the CTI object).

For more details on value resolvers, see [definitions of value resolvers](#).

# 2. DEL Resolvers

DEL resolvers can be categorized into various functional groupings:

- Data field accessors like $.params, $.context
- Data field transformers like $trans [possibly adding e.g. $localtime, $localnum]
- Data field generators like $uuid
- Function Invocation and Flow Control via $func, $funcs and $loop
- Expressions, such as $cel [possibly adding $jq]
- Data requests like $http [possibly adding explicit $grpc]
- Chainable error handling (coming soon)
- Chainable complex functional operators - none currently but 
possibly adding e.g. $reduce, $merge
- Deprecated resolvers - currently only $graphql_v1

## 2.1 Data Field Accessors

Data Field Accessors provide access to context, parameters, and other useful execution-scoped data. Here's a quick reference:

| Variable    | Description                                                                                     |
|-------------|-------------------------------------------------------------------------------------------------|
| `context`   | The DTS function context variables                                                              |
| `params`    | The parameters passed to the DTS function                                                       |
| `return`    | The return value from the function, can be used in filters                                      |
| `loop`      | The loop variable when using `$loop` resolver. Contains `elem` (the current loop element), and `n` (the index of the current loop element, starting with 0) |
| `tags`      | The DTS function tags, if they exist                                                            |
| `cti`       | The CTI of the DTS function                                                                     |
| `tenant_id` | The tenant UUID provided in the DTS resolution request, extracted from the JWT token if not provided |

Use the `$.<path>` syntax to select a value from DTS resolution variables.

**Examples:**
```plaintext
$.context.my_value
$.params.input.some_input
$.return
$.loop.elem
$.tags
$.cti
$.tenant_id
```
---

## 2.2 Data Transformers

Data Transformers tend to be used by presentation functions to tailor data into an appropriate human representation.  A Data Transformer is a pure function that reformats or otherwise makes a simple, deterministic operation on a particular data type.

We may consider adding additional resolvers such as for time and number localization.

### 2.2.1 Translation
| Syntax                                                                                                   | Description                                                                                     |
|----------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `$trans(<dictionary_key>)`                                                                              | Translate a dictionary key.                                                                    |
| `$trans(key=<dictionary_key> [, dictionaries=<dictionary_ctis>] [, language=<language>] [, context=<translation_context>])` | Translate a dictionary key with additional parameters.                                         |
| `$trans(template=<raw_template> [, dictionaries=<dictionary_ctis>] [, language=<language>] [, context=<translation_context>])` | Evaluate a raw template with optional parameters.                                              |

---

## 2.3 Data Generators

### 2.3.1 UUID Creation

| Syntax                                                                                     | Description                                                                                     |
|--------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `$uuid([data=<data_for_uuidv5>] [, space=<uuid_namespace>])`                               | Generate a UUID v4. If `data` is provided, generate a deterministic UUID v5 using the provided data. |

---

## 2.4 Function Invocation

DEL allows for direct function invocation with the `$func` and `$funcs` resolvers.  This allows for recursion, and makes DEL more flexible than, for example CEL or JQ.

### 2.4.1 Simple Function Invocation

| Syntax                                                                                     | Description                                                                                     |
|--------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `$func[<function_cti>](<param_name>=<parameter>, ...)`                                                      | Invoke another DTS function by CTI. Passes the provided named parameters to the DTS function.         |

### 2.4.2 Multiple Function Invocations

| Syntax                                                                                     | Description                                                                                     |
|--------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `$funcs[cti=<function_cti_or_wildcard> [, tags=<function_tags_or_wildcars>] [, order=<path>] [, exclude_errors=<true_or_false>]](<parameters>)` | Invoke a set of DTS functions and collect results as an array. Functions are filtered by wildcard expressions for `cti` and `tags`. |

#### 2.4.3 Loop Function Invocations

| Syntax                                                                                     | Description                                                                                     |
|--------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `$loop(collection=<collection_of_items>, do=<dts_expression> [, order=<path>])`           | Execute a DTS expression (or DTS function) for each element in the input collection. Collects the results as an array.  The collection elements can be accessed as `.loop.elem`, with the element index as `.loop.n`. |

---

## 2.5 Expressions

In order to apply logic, some form of simple expression language is necessary.  DEL primarily supports Common Expression Language (CEL), but support for JQ is also considered for the future.

### 2.5.1 CEL

**Syntax:** `$cel(<cel_expression>)`  
Evaluate a CEL expression, using DTS context variables. Everything inside of `()` is considered to be the CEL expression.  **Important Please Note** You must refer to the DTS context in CEL syntax, which **DOES NOT** include the `$`, for instance: `.context.some_variable` is correct, while `$.context.some_variable` is not.

CEL expressions have access to all DTS resolution variables, in addition to the `now` variable which has the current time as a CEL timestamp.

> **Note:**  
> - `$` is not used in CEL expressions when selecting a value, and DTS expressions cannot be used within CEL expressions.  
> - All CEL variables acquired by DTS expressions should be part of the DTS function context.  
> - In addition to base CEL functionality, DTS includes the `strings`, `lists`, `sets`, `math`, and `encodings` extensions.  
> - We use the go-cel implementation

**Guide:** CEL Quick Reference Guide

---

## 2.6 Data Requests

Data Requests are used to fetch information from various APIs for business logic or to populate the result object.  Currently only HTTP and GraphQL_V1 (deprecated) are supported as Data Requests.

### 2.6.1 HTTP Request

**[Complete $http resolver documentation](/dts/docs/HTTP.md)**

| Syntax                                                                                     | Description                                                                                     |
|--------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `$http[<route_cti>](<path>)`                                                               | Make an HTTP request through OAGW or to an internal service.                                   |
| `$http[<route_cti>](path=<path> [, method=<http_method>] [, body=<request_body>] [, headers=<http_headers>] [, tenant_id=<oagw_tenant_id>])` | Make an HTTP request with additional parameters.                                               |

### 2.6.2 GraphQL v1 Request (Deprecated)

| Syntax                                                                                     | Description                                                                                     |
|--------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `$graphql_v1[<graphql_query_or_mutation_name>](data=<graphql_variables>)`                  | Execute a GraphQL v1 query or mutation. The GraphQL variables are passed as an object under the `data` parameter. To see a complete listing of available `graphql_query_or_mutation_name` [check GraphQL Gateway schema](https://git.acronis.work/projects/ASK/repos/platform-graphql-gateway/browse/graph/schema.graphqls)|

---

## 2.7 Escape

Because DEL expressions begin with `$` to help distinguish DEL expressions from literal strings, when it is intended to start a literal string inside a DEL Expression Object with the `$` character, you must escape it with an escape character `$$` which will be parsed by the DTS Engine as a string `"$"`.

| Syntax | Description                                                                                     |
|--------|-------------------------------------------------------------------------------------------------|
| `$$`   | Escape a string literal that begins with `$`.                                                  |

# DEL Expressions - Scalars and Objects

In some cases, due to the schema of the function as defined, a DEL expression must resolve to a scalar value.  In those cases, you may provide a DEL expression string or the appropriate scalar literal.  In other cases, the DEL expression must resolve to an object.  In those cases, you can provide either a single string which must be a DEL Expression (i.e. a literal string will not be an object type and is therefore invalid), or an object which then recursively may be either strings or objects with the same stipulations.  Finally, in some cases, either a scalar or an object is a valid response.  In such cases, just remember that the DEL expression must ultimately resolve to a compatible data type given whatever complex arrangement is possible.

## Example

Below is an example that illustrates how DEL expressions can be composed to construct objects:

### DTS Function Excerpt

Below is an excerpt of a DTS function, showing the context and return parts of the `cti-traits`.

```yaml
context:
  message: Hello, World!
return:
  non_del_obj:
    str: val # String literal that will be passed as is
    nested_del: $.context.message # This is a string that will be computed by DTS engine as a DEL.
    escaped_del: $$.context.message # And this will not be computed by the DTS engine.
  del_obj: "$cel({str: 'val', nested_cel: '$.context.message'})" # This is an expression too and will be
```

### DTS Engine Output

Below is the output of the DTS engine after processing the source above.

```yaml
context:
  message: Hello, World!
return:
  non_del_obj:
    str: val
    nested_del: Hello, World! # The DEL expression will be resolved into string value in runtime.
    escaped_del: $$.context.message # The value was kept as is since the expression was escaped.
  del_obj: # And this value - into an object value
    str: val
    nested_del: $.context.message # IMPORTANT: This was not resolved because it was a part of CEL expression. Nested CEL expressions produced by CEL value resolver are not recursively processed by the runtime as defined further in this document.
```