Join Log in
Score:0
melink14 avatar Server

Creating a stacked area distribution percentile graph in Google Cloud Monitoring

cn flag
melink14

I've created a log based metric using logs stored in Google Cloud Logs based on HTTP response duration. I'd like to a graph which shows stacked area for 50th, 95th, and 99th percentile.

I've read the Google Cloud Monitoring docs on various charts and while I can get any single metric to display as a percentile; I can't seem to have all three percentiles render stacked at the same time.

What is possible is adding the 3 percentiles to a chart as 3 separate metrics with different alignment which seems to work well for regular line charts but if I change the chart type to stacked area then each percentile shades its area independentally of the other percentile lines.

I can also get all three lines to show up if I select a "Heatmap" chart with sum or delta alignment and click 'hide heatmap' button on the chart in Metrics explorer.

This is a very common way to visualize distribution percentiles so I think it should be possible (may by editing the query direction to include multiple percentiles) so any help is appreciated.

Metric Details:

Type: Distribution

Filter: logName = "projects/lv-server-1/logs/tail-logs" AND jsonPayload.httpRequest:*

Units: s

Field name: jsonPayload.duration

Labels:

  • "status, INTEGER, jsonPayload.status"
  • 'content_type, STRING,jsonPayload.resp_headers."Content-Type"'

Histogram Buckets:

  • Type: Exponential
  • Number of Buckets: 64
  • Exponetnial growth Factor: 2.0
  • Linear scale: .00005
87
1 + 4
Pit avatar
dz flag
Pit
In order to make the reproduction of the scenario, please could you share exactly same log based metrics, and which is the limit for a http response duration time. We have to take into account that the aligned data point is valid only for gauge and delta metrics when they have a distribution data type, as seen in [Align data's documentation](https://cloud.google.com/monitoring/charts/selecting-aggregating-metrics#alignment).
0
Reply
melink14 avatar
cn flag
melink14
Hi @Pedro, I updated the question with more details. I included the histogram buckets since that exactly gives the limit of recorded HTTP response duration times.
0
Reply
Pit avatar
dz flag
Pit
Please could you share as well the build filters? This [similar post](https://stackoverflow.com/questions/67742984/gcp-alert-filters-criteria-for-multiple-resources) might be helpful.
0
Reply
melink14 avatar
cn flag
melink14
I added the metric filters as well. The similar post seems to be related to metrics and alerting but I think my problem is more on how to create charts for my metrics. Though perhaps there's some way to use `||` in the query editor?
0
Reply
Score:1
melink14 avatar Server
cn flag
melink14

I found what seems to be the answer on stackoverflow after reading the Monitoring Query Language docs.

union_group_by allows you to chart multiple time series at once which would allow stacked area graph to work correctly. I searched for examples of usage and came across this stack overflow post which had been posted recently.

Essentially, you create a table of percentiles from your latency data and then pass it to union_group_by:

fetch global::logging.googleapis.com/user/my_metrics.response_time |
{
    percentile_from 50 | add [p: "50th percentile"]
    ;
    percentile_from 90 | add [p: "90th percentile"]
    ;
    percentile_from 95 | add [p: "95th percentile"]
} |
union_group_by [p]

The resulting graph is stacked correctly but uses a different interpolation than when using the basic editor's percentile aligner. The values seem roughly correct though.

0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.