Clearly Specify Router Metrics file format returned #10549

mburke5678 · 2018-06-29T21:21:43Z

https://bugzilla.redhat.com/show_bug.cgi?id=1548500

mburke5678 · 2018-06-29T21:39:18Z

@knobunc Is this something your team can help with? (I pulled a QuickLab instance to test, but it won't allow users to get metrics.) Per BZ:

Describe the issue: 
1. Documentation states, if ROUTER_METRICS_TYPE is set (by default) to haproxy then 
http://admin:<Password>@localhost:1936/metrics should return Prometheus format statistics.
My testing revealed we get a text file with csv Prometheus format statistics with only /metrics exposed.

2.  Documentation states, if ROUTER_METRICS_TYPE is deleted we get HAproxy csv metrics on http://admin:<Password>@localhost:1936/metrics.

My testing revealed we get an html file with statistics on ANY url. Meaning /metrics /ANYTHING will return that html file that has an option to CSV export.

Suggestions for improvement: 
Please clearly specify both the url to get the metrics and the actual file time returned, txt or html

mburke5678 · 2018-07-02T19:55:45Z

@knobunc PTAL

mburke5678 · 2018-07-03T20:37:04Z

@pecameron Can you take a look at this? I believe you wrote the original text that a customer is seeking more information/clarification on.

knobunc · 2018-07-09T13:49:43Z

install_config/router/default_haproxy_router.adoc

 ====
 ----
 $ oc adm router --service-account=router
 ----
 ====
-metrics are automatically available in Prometheus format on the stats-port, default 1936. To suppress metrics collection,
+
+* To get a CSV text file that contains Prometheus format statistics with *metrics* exposed:


It is not a CSV format. I'd say:

To get the Prometheus statistics for the exposed metrics:

knobunc · 2018-07-09T13:51:10Z

install_config/router/default_haproxy_router.adoc

+
+* To get a CSV text file that contains Prometheus format statistics with *metrics* exposed:
+
+Launch the stats window use the following URL in a browser:


This isn't going to work since localhost from a browser is likely the machine they are on. Really they need to tell prometheus to gather from the host:1936 with the appropriate credentials.

@knobunc @pecameron
The customer is looking for:
Please clearly specify both the url to get the metrics and the actual file time returned, txt or html

If I read our docs right, the URL is the STATS_PORT address set in the router env variables and the user can curl <user>:<password>@<router_IP>:<STATS_PORT>/metrics.

oc get po router-1-6tuk1 --template={{.status.hostIP}} for router IP

oc describe po router-1-6tuk1 to get the admin name/password?

pecameron

I think the thrust of the doc should push Prometheus metrics. This plugs into the standard way metrics are collected in Openshift. These metrics include the haproxy native metrics an some controller metrics. The previous way was to use the native metrics from haproxy 1.5 and this should be for backward compatibility only.

pecameron · 2018-07-09T17:27:29Z

install_config/router/default_haproxy_router.adoc

+Launch the stats window use the following URL in a browser:
+
+----
+http://admin:<Password>@localhost:1936/metrics


I think "localhost" should be the router's ip address or FQN. @knobunc is this correct?

pecameron · 2018-07-09T17:30:54Z

install_config/router/default_haproxy_router.adoc

+Launch the stats window use the following URL in a browser:
+
+----
+http://admin:<Password>@localhost:1936/metrics


"/metrics" is not needed here since any path (or no path) will yield the same results. Again, localhost is the IP or FQN of the router.

pecameron · 2018-07-09T17:56:09Z

install_config/router/default_haproxy_router.adoc

@@ -1531,10 +1550,10 @@ metadata:
    prometheus.openshift.io/username: admin
 ----
 ====
-
+


@knobunc didn't we change authorization to not pass the username and password as annotations?

@lihongan Could you review this PR?

mburke5678 · 2018-07-10T19:57:12Z

@anpingli PTAL

lihongan · 2018-07-11T09:41:32Z

install_config/router/default_haproxy_router.adoc

 ====
 ----
 $ oc adm router --service-account=router --stats-port=0
 ----
 ====

-To switch to the HAProxy CSV format metrics, edit the xref:../../architecture/networking/routes.adoc#env-variables[environment variables]
+* To switch to the HAProxy CSV format metrics, edit the xref:../../architecture/networking/routes.adoc#env-variables[environment variables]


Above should be switch to the HTML format metrics instead of CSV format, see https://bugzilla.redhat.com/show_bug.cgi?id=1548500

mburke5678 · 2018-07-16T19:16:39Z

@lihongan PTAL

lihongan · 2018-07-17T02:15:37Z

LGTM, thank you @mburke5678

mburke5678 · 2018-07-17T14:16:03Z

@openshift/team-documentation PTAL

mburke5678 · 2018-07-17T14:19:46Z

@knobunc @pecameron How far back should I roll these changes back? All the way to 3.1?

knobunc · 2018-07-17T14:25:25Z

The router stats were protected by delegated auth in this PR openshift/origin#16975 (this went into 3.7)

And the new stats were added by openshift/origin#13337 (this went into 3.6)

kalexand-rh

I have a couple of picks and suggestions for you to consider. :)

kalexand-rh · 2018-07-17T14:21:34Z

install_config/router/default_haproxy_router.adoc

 link:https://cbonte.github.io/haproxy-dconv/1.5/configuration.html#9[HAProxy router] in its own CSV format.
+These metrics include the HAProxy native metrics and some controller metrics.
+
+When you create a router using the following command, {product-title} makes metrics available in Prometheus format on the stats-port, default 1936. 


maybe s/stats-port/stats port
maybe s/default/by default

kalexand-rh · 2018-07-17T14:22:50Z

install_config/router/default_haproxy_router.adoc

-            value: 0.0.0.0:1936
-          - name: ROUTER_METRICS_TYPE
-            value: haproxy
+oc describe pod <router_pod>


kalexand-rh · 2018-07-17T14:23:06Z

install_config/router/default_haproxy_router.adoc


-Information needed to access the metrics is found the router service annotations:
+* To suppress metrics collection,


kalexand-rh · 2018-07-17T14:24:03Z

install_config/router/default_haproxy_router.adoc

-====
+* To extract the raw statistics in Prometheus format use the following.
+
+You can get the information you need to access the metrics from the router service annotations:


I'd make this line an annotation for line 1501 (annotations:)

kalexand-rh · 2018-07-17T14:25:13Z

install_config/router/default_haproxy_router.adoc


-====
+* To extract the raw statistics in Prometheus format use the following.


s/format/format,
maybe s/use the following./run the following command and modify the output:

kalexand-rh · 2018-07-17T14:25:50Z

install_config/router/default_haproxy_router.adoc

-Use the above username and password to access the metrics. The path is "/metrics".
-
+
+The metrics port is the STATS_PORT, default 1936. You might need to configure your firewall to permit access.


`STATS_PORT` ?
s/default/by default

kalexand-rh · 2018-07-17T14:26:47Z

install_config/router/default_haproxy_router.adoc

-
+
+The metrics port is the STATS_PORT, default 1936. You might need to configure your firewall to permit access.
+Use the above user name and password to access the metrics. The path is "/metrics".


s/above/previous (or established or <modifier_of_choice>)
s/"/metrics"/`/metrics`

kalexand-rh · 2018-07-17T14:27:34Z

install_config/router/default_haproxy_router.adoc

+  value: haproxy
+----
+
+Launch the stats window using the following URL in a browser, where 1936 is the `STATS_PORT` value:


I think the organization of this section could be improved. I think this section is actually three substeps and that you have them presented in 1-3-2 order.
s/Launch the stats window using the following URL in a browser, where 1936 is the STATS_PORT value/Open the stats window at the following URL:
The note about 1936 might be better as an annotation on line 1557.

kalexand-rh · 2018-07-17T14:32:09Z

install_config/router/default_haproxy_router.adoc

+oc describe pod <router_pod>
+----
+
+You get a statistics file in HTML format.


Do the statistics display or do they automatically download? I might say one of these instead:
"The statistics display."
"An HTML file that contains the statistics downloads."

@knobunc Can you help with this? (I tried to set up an QuickLab instance, but I cannot access the stats page). What output does the user get when here?

pecameron · 2018-07-23T14:22:44Z

On 07/23/2018 10:00 AM, Michael Burke wrote: ***@***.**** commented on this pull request. ------------------------------------------------------------------------ In install_config/router/default_haproxy_router.adoc <#10549 (comment)>: > + value: haproxy +---- ++ +Launch the stats window using the following URL in a browser, where 1936 is the `STATS_PORT` value: ++ +---- +http://admin:<Password>@<router_IP>:1936 +---- ++ +To get the router IP, admin name, and password: ++ +---- +oc describe pod <router_pod> +---- ++ +You get a statistics file in HTML format. @knobunc <https://github.com/knobunc> Can you help with this? (I tried to set up an QuickLab instance, but I cannot access the stats page). What output does the user get when here?

I don't have a working cluster at present. When I get it back up I will give it a try. Also on PTO 7/25-31. Sorry for the delay.

…

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10549 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANUgegMlpb9SwaexFjz2_8tf-4KaweQqks5uJdbrgaJpZM4U9pHM>.

openshift-docs-preview-bot · 2018-07-23T14:50:36Z

The preview will be availble shortly at:

install_config/router/default_haproxy_router.adoc: https://BZ-1548500--ocpdocs.netlify.com/openshift-enterprise/(head%20detached%20at%20fetch_head)/install_config/router/default_haproxy_router.html

mburke5678 · 2018-07-24T00:03:41Z

@lihongan
Can you take another look? I got a question from my peer review that caused me to change things a little.

lihongan · 2018-08-01T02:29:43Z

@mburke5678 Looks good. Thank you.
Maybe the notification from GitHub was missed, sorry for late feedback.

vikram-redhat · 2018-08-01T19:36:15Z

@mburke5678 I have added the 3.11 branch label and the next milestone.

openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 29, 2018

openshift deleted a comment from jcantrill Jun 29, 2018

mburke5678 requested review from pecameron and knobunc July 6, 2018 23:40

knobunc requested changes Jul 9, 2018

View reviewed changes

pecameron suggested changes Jul 9, 2018

View reviewed changes

lihongan reviewed Jul 11, 2018

View reviewed changes

mburke5678 force-pushed the BZ-1548500 branch from 8ecfabe to 7372250 Compare July 17, 2018 14:12

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 17, 2018

mburke5678 added peer-review-needed Signifies that the peer review team needs to review this PR branch/enterprise-3.3 branch/enterprise-3.4 branch/enterprise-3.5 branch/enterprise-3.6 branch/enterprise-3.7 branch/enterprise-3.9 branch/enterprise-3.10 labels Jul 17, 2018

kalexand-rh reviewed Jul 17, 2018

View reviewed changes

kalexand-rh added peer-review-done Signifies that the peer review team has reviewed this PR and removed peer-review-needed Signifies that the peer review team needs to review this PR labels Jul 17, 2018

mburke5678 removed branch/enterprise-3.3 branch/enterprise-3.4 branch/enterprise-3.5 labels Jul 17, 2018

mburke5678 force-pushed the BZ-1548500 branch from 7e6a05a to e078574 Compare July 23, 2018 15:19

Clearly Specify Router Metrics file format returned

82cff2a

mburke5678 force-pushed the BZ-1548500 branch from e078574 to 82cff2a Compare August 1, 2018 16:51

mburke5678 merged commit a224255 into openshift:master Aug 1, 2018

mburke5678 deleted the BZ-1548500 branch August 1, 2018 17:01

openshift deleted a comment from openshift-cherrypick-robot Aug 1, 2018

vikram-redhat added this to the Next Release milestone Aug 1, 2018

vikram-redhat added the branch/enterprise-3.11 label Aug 1, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clearly Specify Router Metrics file format returned #10549

Clearly Specify Router Metrics file format returned #10549

mburke5678 commented Jun 29, 2018

mburke5678 commented Jun 29, 2018

mburke5678 commented Jul 2, 2018

mburke5678 commented Jul 3, 2018

knobunc Jul 9, 2018

knobunc Jul 9, 2018

mburke5678 Jul 9, 2018 •

edited

Loading

pecameron left a comment

pecameron Jul 9, 2018

pecameron Jul 9, 2018

pecameron Jul 9, 2018

anpingli Jul 11, 2018

mburke5678 commented Jul 10, 2018

lihongan Jul 11, 2018

mburke5678 commented Jul 16, 2018

lihongan commented Jul 17, 2018

mburke5678 commented Jul 17, 2018

mburke5678 commented Jul 17, 2018

knobunc commented Jul 17, 2018

kalexand-rh left a comment

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

kalexand-rh Jul 17, 2018

mburke5678 Jul 23, 2018

pecameron commented Jul 23, 2018 via email

openshift-docs-preview-bot commented Jul 23, 2018

mburke5678 commented Jul 24, 2018 •

edited

Loading

lihongan commented Aug 1, 2018

vikram-redhat commented Aug 1, 2018


		Information needed to access the metrics is found the router service annotations:
		* To suppress metrics collection,


		====
		* To extract the raw statistics in Prometheus format use the following.

Clearly Specify Router Metrics file format returned #10549

Clearly Specify Router Metrics file format returned #10549

Conversation

mburke5678 commented Jun 29, 2018

mburke5678 commented Jun 29, 2018

mburke5678 commented Jul 2, 2018

mburke5678 commented Jul 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mburke5678 Jul 9, 2018 • edited Loading

Choose a reason for hiding this comment

pecameron left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mburke5678 commented Jul 10, 2018

Choose a reason for hiding this comment

mburke5678 commented Jul 16, 2018

lihongan commented Jul 17, 2018

mburke5678 commented Jul 17, 2018

mburke5678 commented Jul 17, 2018

knobunc commented Jul 17, 2018

kalexand-rh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pecameron commented Jul 23, 2018 via email

openshift-docs-preview-bot commented Jul 23, 2018

mburke5678 commented Jul 24, 2018 • edited Loading

lihongan commented Aug 1, 2018

vikram-redhat commented Aug 1, 2018

mburke5678 Jul 9, 2018 •

edited

Loading

mburke5678 commented Jul 24, 2018 •

edited

Loading