-
Notifications
You must be signed in to change notification settings - Fork 272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Store stack trace in a separate filed during logging #2444
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 4 of 4 files at r1, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @jianglai)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 4 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jianglai)
util/src/main/java/google/registry/util/GcpJsonFormatter.java
line 111 at r1 (raw file):
stacktrace = sw.toString(); } String message = '\n' + record.getMessage() + stacktrace;
Could you try appending \r\n
to the stack trace and see if it helps?
The fluent rules for collecting multi-line logs is at https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions/blob/8e5608209d9c2a8927b4531b2077f7057d65616c/lib/fluent/plugin/exception_detector.rb#L54,
and it seems that '\r\n` would help.
Code quote:
stacktrace
For reasons unclear to me, if the stack trace is appended directly to the message, the log entry will be lumped together with following logs on GKE. Also updated the GKE service account for Nomulus in the manifest so we can use workload identity just for Nomulus, not other pods on the same cluster.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @weiminyu)
util/src/main/java/google/registry/util/GcpJsonFormatter.java
line 111 at r1 (raw file):
Previously, weiminyu (Weimin Yu) wrote…
Could you try appending
\r\n
to the stack trace and see if it helps?The fluent rules for collecting multi-line logs is at https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions/blob/8e5608209d9c2a8927b4531b2077f7057d65616c/lib/fluent/plugin/exception_detector.rb#L54,
and it seems that '\r\n` would help.
Tried it, didn't help :(
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: 3 of 4 files reviewed, all discussions resolved (waiting on @jianglai)
For reasons unclear to me, if the stack trace is appended directly to
the message, the log entry will be lumped together with following logs
on GKE.
Also updated the GKE service account for Nomulus in the manifest so we
can use workload identity just for Nomulus, not other pods on the same
cluster.
This change is