Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store stack trace in a separate filed during logging #2444

Merged
merged 1 commit into from
May 20, 2024

Conversation

jianglai
Copy link
Collaborator

@jianglai jianglai commented May 15, 2024

For reasons unclear to me, if the stack trace is appended directly to
the message, the log entry will be lumped together with following logs
on GKE.

Also updated the GKE service account for Nomulus in the manifest so we
can use workload identity just for Nomulus, not other pods on the same
cluster.


This change is Reviewable

Copy link
Collaborator Author

@jianglai jianglai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 4 of 4 files at r1, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @jianglai)

@jianglai jianglai requested a review from weiminyu May 15, 2024 15:29
@jianglai jianglai enabled auto-merge May 15, 2024 15:29
Copy link
Collaborator

@weiminyu weiminyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 4 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @jianglai)


util/src/main/java/google/registry/util/GcpJsonFormatter.java line 111 at r1 (raw file):

      stacktrace = sw.toString();
    }
    String message = '\n' + record.getMessage() + stacktrace;

Could you try appending \r\n to the stack trace and see if it helps?

The fluent rules for collecting multi-line logs is at https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions/blob/8e5608209d9c2a8927b4531b2077f7057d65616c/lib/fluent/plugin/exception_detector.rb#L54,

and it seems that '\r\n` would help.

Code quote:

 stacktrace

For reasons unclear to me, if the stack trace is appended directly to
the message, the log entry will be lumped together with following logs
on GKE.

Also updated the GKE service account for Nomulus in the manifest so we
can use workload identity just for Nomulus, not other pods on the same
cluster.
@jianglai jianglai requested a review from weiminyu May 20, 2024 15:25
Copy link
Collaborator Author

@jianglai jianglai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @weiminyu)


util/src/main/java/google/registry/util/GcpJsonFormatter.java line 111 at r1 (raw file):

Previously, weiminyu (Weimin Yu) wrote…

Could you try appending \r\n to the stack trace and see if it helps?

The fluent rules for collecting multi-line logs is at https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions/blob/8e5608209d9c2a8927b4531b2077f7057d65616c/lib/fluent/plugin/exception_detector.rb#L54,

and it seems that '\r\n` would help.

Tried it, didn't help :(

Copy link
Collaborator

@weiminyu weiminyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 4 files reviewed, all discussions resolved (waiting on @jianglai)

@jianglai jianglai added this pull request to the merge queue May 20, 2024
Merged via the queue into google:master with commit d96a554 May 20, 2024
7 of 9 checks passed
@jianglai jianglai deleted the gke-identity branch May 20, 2024 17:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants