Difference between display and fields directives in CloudWatch Logs Insights query syntax

What is the difference between display and fields directives in CloudWatch Logs Insights query syntax?

These are descriptions from the AWS documentation that look very similar to me :

display: Specifies which fields to display in the query results.

fields: Retrieves the specified fields from log events for display.

As an example, I have logs stored in Cloudwatch in this structure (with these fields):

• @timestamp
• @message
• stream (stdout|stderr)
• kubernetes.namespace_name
• ...

Here are examples of valid queries that confuse me:

1. I can display any non-retrieved field:

limit 8
| display @message, stream

2. I can display a field even if I haven't specified it in fields.

fields @message, stream
| limit 8
| display @message, stream, kubernetes.namespace_name

3. It doesn't matter if I specify a field in fields when parsing:

fields @message
| parse @message "[*] *" as loggingType, loggingMessage
| display loggingMessage

parse @message "[*] *" as loggingType, loggingMessage
| display loggingMessage

What is the meaning of the fields directive? Wouldn't it be enough to just use display?

The difference in fields and display commands is that fields behavior is cumulative and display is not (replace-like behavior).

From the CloudWatch Logs Insights query syntax guideline:

If your query contains multiple fields commands and doesn't include a display command, you'll display all of the fields that are specified in the fields commands.

So a display command would replace the output defined by any preceding display or fields commands (or any other command that defines ephemeral fields), and the fields would add to the currently defined output.

Examples:

1. Returns @timestamp, @message, @logStream, @log fields

fields @timestamp, @message, @logStream, @log
| sort @timestamp desc
| limit 20

2. Returns @message

fields @timestamp, @message, @logStream, @log
| sort @timestamp desc
| limit 20
| display @message

3. Returns @message, requestId, text, hasRequestId

fields @timestamp, @message, @logStream, @log
| sort @timestamp desc
| limit 20
| display @message
| parse @message "(*) *" as requestId, text
| fields !isblank("requestId") as hasRequestId

The last example defines ephemeral fields using parse command (these do not have @ in the beginning) and expression field defined by last fields command

By my reading of the documentation, fields would be used during the query, and display only for presentation at the end. display is also only effective once (the last invocation is used).

What I mean is that in the example:

fields concat(Operation, '-', StatusCode) as opStatus

I don't think you could use display to do the same thing.

You might want to gather multiple fields to calculate a value (using fields), or filter some fields for speed, but only display the final result at the end.