How Expressions Work

Canvas expressions execute via a system of functions and types. So how does this work in practice?

Let’s deconstruct at an example expression. See the Glossary at the bottom if you get lost on any of the terms.

esdocs fields=bytes,@timestamp
| staticColumn column=total value={math 'sum(bytes)'}
| head '10'
| mapColumns column=@timestamp, fn=${getColumn @timestamp | rounddate 'YYYY-MM-DD'}

Now, whether it looks like it or not, this function takes advantage of all of the features of the Canvas interpreter. So how does execution look?

In the beginning there is null

esdocs fields=bytes,@timestamp

You might think that executing esdocs is the first thing that happens, but thats not quite true.

The first thing that happens is that the interpreter looks up what esdocs requires as context (see Glossary). But of course, here, esdocs is the first function in the expression, so there’s no context. Which means it will receive null right? Nope. If we look at the esdocs function we see that it requires a context of type query.

At this point, the interpreter goes into casting mode. It looks at the null value and says “Ah yes, the query type knows how to create itself from null” So it does that, creating an object that looks something like {type: “query”}.

Then it looks at all of the arguments for esdocs. If any of them are an expression, it runs those expressions, but we'll get to that in a bit. No expressions here, just a plain string argument.

At this point, esdocs executes, receiving both context and the resolved values all of its arguments. So it gets a query as context, and 1 argument telling it to retrieve @timestamp and bytes

A delicious meatball sub-expression

| staticColumn column=total value={math 'sum(bytes)'}

Oh good esdocs has output a datatable object with 2 columns: @timestamp and bytes. Of course we already knew it would output a datatable, because esdocs declares that it outputs a datatable. Now we have to deal with staticColumn, which creates a new column with a value that is the same for all rows. Conveniently for us, staticColumn accepts and outputs a datatable, nothing new to learn there: phew.

But there clearly is something new to learn. Check out those {}'s. Curly braces mean we're dealing with a sub-expression. As we noted in the above section, we need to execute this expression before we can execute staticColumn. In order to do this we pass each sub-expression the same context we will eventually give to staticColumn: The datatable output by esdocs. math accepts a datatable and, in this case, uses it to sum up all the values of a column.

Once all of the expressions have run, the interpreter checks the outputs of each and casts them as necessary for the types that each argument has declared it requires. In this case the value argument accepts string, number, null or boolean, so we're all good. We now execute staticColumn which outputs a new datatable with a new column called total containing the sum of all values in the bytes column for every row.

Cast out thy casting

| head '10'

Because staticColumn outputs a datatable must head declare it requires a datatable as context? No, but it does. If it didn't, then head would need to accept anything we throw at it. Likewise, it doesn't technically have to declare that it returns a datatable either, but again, it does. We strongly recommend that, whenever possible, you declare the types a function accepts and returns.

But then we see head been passed 10. Where, or more specifically, what, is the argument name?

Canvas’s grammar allows for unnamed arguments and automatically assigns them the name of _. Like any argument, _ can be typed, and in the case of head, it is set to [‘number’]. Now '10' looks like a number, but it isn’t. It is surrounded by single quotes, so its a string. No matter, number knows how to cast from string, so it does.

head now executes, receiving the datatable output from esdocs as well as the number 10 as the _ argument. head chops off everything but the first 10 rows

We put the “fun” in “functions”. I think the “ions” come from electricity. Not sure on the “ct”

| mapColumns column=@timestamp, fn=${getColumn @timestamp | rounddate 'YYYY-MM-DD'}

Now then, mapColumn is the most complex function we’ve seen yet. It accepts a datatable as context, check. It expects a column argument as a string, also check. It then expects a function argument. Nope.

We have an fn. Is that good enough? Yes, actually it is, because the function argument to mapColumn declares that fn is a valid alias for function, so it gets renamed before being passed into mapColumn.

But what about that expression that starts with getColumn, don’t we have to execute that and pass its value in? No, oddly enough, we don’t do that. This is because it starts with a $ that $ means that it is intended to be passed into mapColumn as an executable function, for mapColumn todo as it pleases. The Canvas interpreter basically wraps its self around getColumn @timestamp | rounddate 'YYYY-MM-DD' and passes itself into mapColumn (OOoooo, meta).

It is now up to mapColumn to run getColumn @timestamp | rounddate 'YYYY-MM-DD' with whatever context it pleases. Of course, all the casting stuff still happens if needed, even though mapColumn is the one doing the executing. mapColumn will of course get the output of rounddate during its execution, and presumably do something with it. In this case it will use the output to do some date rounding.

All done…or are we?

Yeah, mostly, we’re done.

If this was being run from the UI there would be one more thing. The UI would receive the datatable and say “Uh, I only know how to render stuff of type render”. It would then pass the datatable back into the interpreter as context, and with a function of render. This will take advantage of casting to wrap the datatable with a render object and tell the UI to show it as an HTML table.

Now we’re done. Good stuff.

Glossary

Type

Function outputs, arguments, and contexts, are all optionally typed. Types are declared as an array, anything is the array is acceptable. If a value is presented that does not match the declared type the interpreter will attempt to cast the value to an acceptable type, in the order in which the types are declared in the types array.

Typing is encouraged as it aids in UI hinting, autocomplete, and may be used for performance improvements and cache optimization in the future. For example, when we know an output type is incompatible with an expected input, we can choose not to execute the function, knowing that it will not succeed. If you don’t declare types, we have to execute the expression to see the failure, which is a waste of resources.

Argument

Functions declare the arguments they accept along with the type that argument is expected to be.

Context

Every function is supplied with a context. Context is usually the output of the previous function. If the function is the first one in the expression, the context is null, unless cast to something else.

Function

Functions take in a context, and, optionally some arguments.

Casting

If the type of the presented value does not match the expected input type, then the interpreter will attempt to convert it. The casting system is supplied with the presented value, and an array of acceptable types. It first checks the definition of the presented type, to see if it knows how to convert itself to any of the required types. If that fails it checks each of the acceptable types to see if they can create themselves from the presented type. As long as one of these conversions successes, the expression continues, otherwise it fails.

Alias

Alternative names for arguments or functions, usually for the purpose of shortening. Argument aliases will be resolved to their original declared name before being passed into the function.