Stream your hapi.js logs to Google Cloud
4 min read
I have been using Google Cloud Platform for a while now, deployed several Node.js applications using App Engine custom runtime. Node and GAE is a relatively new combination and of course there are advantages and disadvantages of going down this path instead of the traditional GCE. Writing about this experience might be a good idea for my next post but this time I want to share with you how to fully use GCP Logging with your node application. It always bugs me that the out-of-the-box logging solution that Google provides to GAE custom runtime is a simple fluentd agent that reads the app logs as plain text without the ability to define the log pattern. This way developers can’t analyze their logs and figure out what’s going on with their application. Recently, I have started working on a new application and as always I chose my hapi.js stack on top of GAE. The only problem is that I needed a good logging infrastructure to monitor all the data that is being streamed to my application. I tore down the web looking for an elegant solution of how to stream my logs to GCP Logging, without any lead. This is when I decided to take a step forward and build a module which streams Good (the logging module for hapi) logs to GCP Logging. Thanks to the recent refactor made by Adam Bretz in version 7, Good reporter is just a stream object which is very easy to implement. Using the gcloud module (GCP api module), I extended the Stream.Writable class to create a stream which maps the incoming Good data into a GCP Log Entry and send it to the cloud. Now all the logs are fully structured in GCP Logging and I can create custom metrics and get insights out of my logs.It is very easy to use this module if you are already familiar with hapi.js (if not, I am encouraging you to give it a try).
- Install the module npm install good-gcloud
- Add good-gcloud to your Good reporter as the end stream
- Log as usual using server.log or request.log
- Analyze your logs using GCP Logging
For more detailed instructions about good-gcloud configuration, you are welcome to visit the github page.
Waiting for your feedback and activity in the repository!P.S I created two other useful streams for Good
- good-separator — adds a separator after each object in the stream. Used to add line break after each object when streaming to stdout.
- good-requests-filter — filters requests logging using regex. Used to filter swagger related requests.