Logstash: How to Parse JSON Logs

In this tutorial, I will show you how to parse JSON logs in Logstash. There are two ways to parse JSON logs in Logstash:

Parse JSON logs from a file
Parse JSON logs from a string field in an event

Let's discover each way in detail.

Parse JSON from a File
Parse JSON from a String Field
Conclusion

Parse JSON from a File

To parse JSON logs from a file in Logstash config, you should use the multiline codec to input the file and use the json filter to parse the JSON logs.

logstash.conf


input {
  file {
    path => "/path/to/log.json"
    start_position => "beginning"
    codec => multiline {
      pattern => "^\{"
      negate => true
      what => "previous"
    }
  }
}
 
filter {
  json {
    source => "message"
  }
}
 
output {
  stdout {
    codec => rubydebug
  }
}

Here is the explanation:

I used the multiline codec to read the file line by line, each line will be an event and will be stored in the message field.
I used the json filter to parse the message field into a JSON object.

For example, I have a JSON log file like this:

log.json


{"name": "Minh Vu", "age": 22}
{"name": "Desmond", "age": 99}

For JSON data like below:
log.json
{
  "name": "Minh Vu",
  "age": 22
}
Please visit the Logstash Input from JSON File tutorial.

After running Logstash with the above config, I will get the output like this:


{
    "name" => "Minh Vu",
    "age" => 22,
    "path" => "/path/to/log.json",
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}
{
    "name" => "Desmond",
    "age" => 99,
    "path" => "/path/to/log.json",
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}

Parse JSON from a String Field

If you have an event or document that has a string field which contains a JSON object, you can use the json filter to parse that field into a JSON object.

For example, I have a document like this:


{
  "message": "{\"name\": \"Minh Vu\", \"age\": 22}"
}

To parse the message field into a JSON object, I will use the json filter with specfied source field:

logstash.conf


# ...
 
filter {
  json {
    source => "message"
  }
}
 
# ...

After running Logstash with the above config, I will get the output like this:


{
    "message" => "{\"name\": \"Minh Vu\", \"age\": 22}",
    "name" => "Minh Vu",
    "age" => 22,
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}

The json filter also allows you to change the target field name by using the target option:

logstash.conf


# ...
 
filter {
  json {
    source => "message"
    target => "json_message"
  }
}
 
# ...

After running Logstash with the above config, I will get the output like this:


{
    "message" => "{\"name\": \"Minh Vu\", \"age\": 22}",
    "json_message" => {
        "name" => "Minh Vu",
        "age" => 22
    },
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}

Conclusion

I have shown you 2 ways to parse JSON logs in Logstash. Hope it helps you!

Logstash: How to Parse JSON Logs

Contents

Parse JSON from a File

Parse JSON from a String Field

Conclusion

Contents

Parse JSON from a File

Parse JSON from a String Field

Conclusion

Most Read