Why we care about Golang
For us at Sipfront, it is crucial to use the latest and most advanced tools and technologies to stay ahead of the competition. Go’s unique features, such as its fast compile times, built-in concurrency support, and strong type system, enable us to create robust and scalable software systems that can meet the demanding needs of modern applications, microservices and distributed systems.
The following page is intended as a small documentation with suggestions/tips on how to set-up a logger for Go, that logs directly to OpenSearch via the logrus package and the official opensearch-client. Most parts of the code are straightforward and you can find the repository at https://github.com/sipfront/go-opensearch-logger-playground . There are one to two pitfalls one might encounter which will be explained a little further down.
Why logrus instead of the built-in logging library?
Because it is more flexible and offers more functionality. Also, to quote the developers/maintainers:
Logrus is a structured logger for Go (golang), completely API compatible with the standard library logger.
This means, that if you already have implemented a logger using the log package you should not have any issues. Furthermore, logrus is in maintenance-mode. No new features are introduced that could break your code.
Why using the opensearch-client , instead of net/http
Just for convenience reasons. The client library is basically nothing more than a wrapper around net/http functions/methods.
The straightforward Parts - Part 1
In general, logrus writes directly to Stdout / Stderr. If you want to write to somewhere else e.g. to OpenSearch or a database, you need to write a custom type using the Struct construct that implements the Write interface. For more information see, Writer. This custom type could be implemented for example as follows:
1// LogMessage describes a simple log message, which is then encoded into a json
2type LogMessage struct {
3 Timestamp time.Time `json:"@timestamp"`
4 Message string `json:"message"`
5 Function string `json:"function_name"`
6 Level string `json:"level"`
7}
8
9// Custom type that will later implement the Write method/interface
10// for logging directly to Opensearch, without the help of logstash.
11type OpenSearchWriter struct {
12 Client *opensearch.Client
13}
14
15// Write 'method' to write directly to opensearch
16func (ow *OpenSearchWriter) Write(p []byte) (n int, err error) {
17 trimmedString := strings.Trim(string(p), "{}")
18 splittedString := strings.SplitAfterN(trimmedString, ",", 3)
19
20 function := strings.SplitAfter(splittedString[0], ":")[1]
21 logLevel := strings.SplitAfter(splittedString[1], ":")[1]
22
23 r := strings.NewReplacer("\\n", "", "\\r", "", "\\t", "", "\"", "", "\\", "")
24 message := strings.SplitAfterN(splittedString[2], ":", 2)[1]
25 messageCleaned := r.Replace(message)
26
27 logMessage := LogMessage{
28 Timestamp: time.Now().UTC(),
29 Message: messageCleaned[1:strings.LastIndex(messageCleaned, ",")],
30 Function: function[2 : len(function)-2],
31 Level: logLevel[2 : len(logLevel)-2],
32 }
33
34 logJson, err := json.Marshal(logMessage)
35 if err != nil {
36 return 0, err
37 }
38
39 req := opensearchapi.IndexRequest{
40 Index: "sipfront-gotest-" + time.Now().UTC().Format("2006.01.02"),
41 Body: strings.NewReader(string(logJson)),
42 }
43 insertResponse, err := req.Do(context.Background(), ow.Client)
44 if err != nil {
45 return 0, err
46 }
47 defer insertResponse.Body.Close()
48 fmt.Println(insertResponse)
49
50 return len(p), nil
51}
The snippet above should just give you a first idea of what a custom Struct implementing Write might look like.
The straightforward Parts - Part 2
To actually use the logger the following steps have to be taken, which will be illustrated with the following example
- In func main() we first initialize the OpenSearch Client. As you can see, it is really easy.
1func main() {
2 var clientConfiguration opensearch.Config = opensearch.Config{
3 Transport: &http.Transport{
4 TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
5 },
6 Addresses: []string{
7 "adress-url-to-your-opensearch-server"},
8 }
9 client, err := opensearch.NewClient(clientConfiguration)
10 if err != nil {
11 fmt.Println("cannot initialize", err)
12 os.Exit(1)
13 }
14
15...*
- After that, we set up the logger. To quote from the documentation
WithField creates an entry from the standard logger and adds a field to it. […] Note that it doesn’t log until you call Debug, Print, Info, Warn, Fatal or Panic on the Entry it returns
1*...
2 var l *logrus.Logger = logrus.New()
3 e := l.WithField("function_name", "main")
4
5 l.SetOutput(&OpenSearchWriter{Client: client})
6 l.SetLevel(logrus.InfoLevel)
7 l.SetFormatter(&OpensearchFormatter{PrettyPrint: true})
8...*
But what is an Entry? To reference the documentation again:
An entry is the final or intermediate Logrus logging entry. It contains all the fields passed with `WithField{,s}. It’s finally logged when Trace, Debug, Info, Warn, Error, Fatal or Panic is called on it. These objects can be reused and passed around as much as you wish to avoid field duplication.
Since we need the function_name for logging, it makes sense to declare an Entry at the top level of the module. Here it is declared in main just for convenience reasons. Now every log output will contains the field function_name. We could have also written l.WithField("function_name", "main").Info("here-comes-your-log-message"), but then you would have to include WithField(...) with every log. Therefore I recommend using the approach with Entry.
- Last we can test the logger with:
1*...
2 e.Info("this-is-a-test")
3}
And that’s it. The logger is now fully functional. The attentive reader will have noticed l.SetFormatter(&OpensearchFormatter{...}). OpensearchFormatter basically formats the log output. It has been copied and adapted from here.
The Pitfalls
Now for the pitfalls. If we go back to the section The straightforward Parts - Part 1, lines 17-25 have nothing to do with logging per se but with parsing the message. Now the question arises ”Logrus is a structured logger, you can just pass the message provided by e.Info("here-is-your-message") and the formatter will do the rest".
The problem with this approach is, that if you pass the message to the Body parameter Opensearch will create an index, if it does not exist yet. That’s actually great, but we have not provided any mapping, so we cannot search for the fields of the index documents created. To be more precise, each index document needs the field @timestamp, which must be of type date. Using this approach, @timestamp would be of type text/string. These types cannot be aggregated and therefore we cannot search for the index documents.
We could provide a mapping by creating an index beforehand, but every time we want to log into Opensearch, we would have to check if the index already exists, otherwise we would get an HTTP 400 error. This also results in approximately 2x the amount of http requests. If you have time constraints or a module like sipfront-persistor, that is called a lot, this is not desirable!
The solution to overcome this problem was to implement two custom structs, LogMessage and OpenSearchWriter. LogMessage describes a simple log message, which is then encoded into a json. The struct field Timestamp is of type time, so the resulting field in Opensearch is aggregatable.