Too Many Labels

Errors

• 时间：2025-03-28T07:29:53.959Z
• 等级：WARN
• 调用位置：VictoriaMetrics/lib/timeserieslimits/timeseries_limits.go:72
• 消息：忽略具有 41 个标签的时序数据。
• 原因：超出了 -maxLabelsPerTimeseries=40 的限制。
• 建议：减少该指标的标签数量，或提高命令行参数 -maxLabelsPerTimeseries 的值。

Source

{"ts":"2025-03-28T07:29:53.959Z","level":"warn","caller":"VictoriaMetrics/lib/timeserieslimits/timeseries_limits.go:72","msg":"ignoring series with 41 labels for {__name__=\"istio_requests_total\",app=\"istio-ingressgateway\",chart=\"gateways\",connection_security_policy=\"unknown\",destination_app=\"unknown\",destination_canonical_revision=\"11.4.0\",destination_canonical_service=\"grafana\",destination_cluster=\"Kubernetes\",destination_principal=\"unknown\",destination_service=\"grafana.monitor.svc.cluster.local\",destination_service_name=\"grafana\",destination_service_namespace=\"monitor\",destination_version=\"unknown\",destination_workload=\"grafana\",destination_workload_namespace=\"monitor\",heritage=\"Tiller\",install_operator_istio_io_owning_resource=\"unknown\",instance=\"10.244.125.70:15090\",istio=\"ingressgateway\",istio_io_rev=\"1-19-8\",job=\"istio-ingressgateway\",kubernetes_namespace=\"istio-system\",kubernetes_pod_name=\"istio-ingressgateway-75b8d7b6cf-9ld69\",kubernetes_service_name=\"istio-ingressgateway-prometheus\",operator_istio_io_component=\"IngressGateways\",pod_template_hash=\"75b8d7b6cf\",release=\"istio\",reporter=\"source\",request_protocol=\"http\",response_code=\"101\",response_flags=\"DC\",service_istio_io_canonical_name=\"istio-ingressgateway\",service_istio_io_canonical_revision=\"1-19-8\",sidecar_istio_io_inject=\"false\",source_app=\"istio-ingressgateway\",source_canonical_revision=\"1-19-8\",source_canonical_service=\"istio-ingressgateway\",source_cluster=\"Kubernetes\",source_principal=\"unknown\",source_workload=\"istio-ingressgateway\",source_workload_namespace=\"istio-system\"}; either reduce the number of labels for this metric or increase -maxLabelsPerTimeseries=40 cmd-line flag value"}

Metric Labels

{
  "__name__": "istio_requests_total",
  "app": "istio-ingressgateway",
  "chart": "gateways",
  "connection_security_policy": "unknown",
  "destination_app": "unknown",
  "destination_canonical_revision": "11.4.0",
  "destination_canonical_service": "grafana",
  "destination_cluster": "Kubernetes",
  "destination_principal": "unknown",
  "destination_service": "grafana.monitor.svc.cluster.local",
  "destination_service_name": "grafana",
  "destination_service_namespace": "monitor",
  "destination_version": "unknown",
  "destination_workload": "grafana",
  "destination_workload_namespace": "monitor",
  "heritage": "Tiller",
  "install_operator_istio_io_owning_resource": "unknown",
  "instance": "10.244.125.70:15090",
  "istio": "ingressgateway",
  "istio_io_rev": "1-19-8",
  "job": "istio-ingressgateway",
  "kubernetes_namespace": "istio-system",
  "kubernetes_pod_name": "istio-ingressgateway-75b8d7b6cf-9ld69",
  "kubernetes_service_name": "istio-ingressgateway-prometheus",
  "operator_istio_io_component": "IngressGateways",
  "pod_template_hash": "75b8d7b6cf",
  "release": "istio",
  "reporter": "source",
  "request_protocol": "http",
  "response_code": "101",
  "response_flags": "DC",
  "service_istio_io_canonical_name": "istio-ingressgateway",
  "service_istio_io_canonical_revision": "1-19-8",
  "sidecar_istio_io_inject": "false",
  "source_app": "istio-ingressgateway",
  "source_canonical_revision": "1-19-8",
  "source_canonical_service": "istio-ingressgateway",
  "source_cluster": "Kubernetes",
  "source_principal": "unknown",
  "source_workload": "istio-ingressgateway",
  "source_workload_namespace": "istio-system"
}