Logstash: How to Parse JSON Logs

In this tutorial, I will show you how to parse JSON logs in Logstash. There are two ways to parse JSON logs in Logstash:

1. Parse JSON logs from a file
2. Parse JSON logs from a string field in an event

Let's discover each way in detail.

Contents

• Parse JSON from a File
• Parse JSON from a String Field
• Conclusion

Parse JSON from a File

To parse JSON logs from a file in Logstash config, you should use the multiline codec to input the file and use the json filter to parse the JSON logs.

input {
  file {
    path => "/path/to/log.json"
    start_position => "beginning"
    codec => multiline {
      pattern => "^\{"
      negate => true
      what => "previous"
    }
  }
}
 
filter {
  json {
    source => "message"
  }
}
 
output {
  stdout {
    codec => rubydebug
  }
}

Here is the explanation:

1. I used the multiline codec to read the file line by line, each line will be an event and will be stored in the message field.
2. I used the json filter to parse the message field into a JSON object.

For example, I have a JSON log file like this:

{"name": "Minh Vu", "age": 22}
{"name": "Desmond", "age": 99}
 

For JSON data like below:

{
  "name": "Minh Vu",
  "age": 22
}

Please visit the Logstash Input from JSON File tutorial.

After running Logstash with the above config, I will get the output like this:

{
    "name" => "Minh Vu",
    "age" => 22,
    "path" => "/path/to/log.json",
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}
{
    "name" => "Desmond",
    "age" => 99,
    "path" => "/path/to/log.json",
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}

Parse JSON from a String Field

If you have an event or document that has a string field which contains a JSON object, you can use the json filter to parse that field into a JSON object.

For example, I have a document like this:

{
  "message": "{\"name\": \"Minh Vu\", \"age\": 22}"
}

To parse the message field into a JSON object, I will use the json filter with specfied source field:

# ...
 
filter {
  json {
    source => "message"
  }
}
 
# ...

After running Logstash with the above config, I will get the output like this:

{
    "message" => "{\"name\": \"Minh Vu\", \"age\": 22}",
    "name" => "Minh Vu",
    "age" => 22,
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}

The json filter also allows you to change the target field name by using the target option:

# ...
 
filter {
  json {
    source => "message"
    target => "json_message"
  }
}
 
# ...

After running Logstash with the above config, I will get the output like this:

{
    "message" => "{\"name\": \"Minh Vu\", \"age\": 22}",
    "json_message" => {
        "name" => "Minh Vu",
        "age" => 22
    },
    "host" => "minhvu.local",
    "@timestamp" => "2024-01-17T20:29:00.000Z",
    "@version" => "1"
}

Conclusion

I have shown you 2 ways to parse JSON logs in Logstash. Hope it helps you!