Logstash Convert Date to ISO8601 Format

In this tutorial, I will show you how to convert a date field to ISO8601 format in Logstash.

In general, we can use the ruby filter with the DateTime.parse() method to parse a date string to a DateTime object. Then, we can use the iso8601() method to get the ISO8601 format.

Continue reading to see it in action.

Contents

• How to Convert Date to ISO8601 Format in Logstash using the ruby Filter
• How to Parse Custom Date Format and Convert to ISO8601 Format in Logstash
• How to Convert Date to ISO8601 Format in Logstash with date Filter
• How to Convert Date with Current Timezone to UTC ISO8601 Format in Logstash

How to Convert Date to ISO8601 Format in Logstash using the ruby Filter

In some cases, the date field would be yyyy-MM-dd HH:mm:ss, or some other format. But we want to convert it to ISO8601 format to normalize the date format.

To convert a date string to ISO8601 format, we can use the following Logstash configuration:

filter {
  ruby {
    code => '
      date = event.get("date")
      event.set("iso8601_date", DateTime.parse(date).iso8601())
    '
  }
}

For example, if I have the following sample log:

{
  "date": "2024-01-03 22:00:00"
}

The result after using the above Logstash configuration would be:

{
  "date": "2024-01-03 22:00:00",
  "iso8601_date": "2024-01-03T12:03:00+00:00"
}

You can specify iso8601(3) to get the ISO8601 format with milliseconds of 3 digits.

{
  "date": "2024-01-03 22:00:00",
  "iso8601_date": "2024-01-03T12:03:00.000+00:00"
}

If you need to convert date to ISO8601 with the current timezone, please go to the fourth example.

How to Parse Custom Date Format and Convert to ISO8601 Format in Logstash

Usually, the date will be in a custom format. For example, dd.MM.yyyy,HH:mm.

We can still use the same method to parse the date string to a DateTime object by specifying the format of the date string.

filter {
  ruby {
    code => '
      date = event.get("date")
      event.set("iso8601_date", DateTime.strptime(date, "%d.%m.%Y,%H:%M").iso8601())
    '
  }
}

The result would be:

{
  "date": "03.01.2024,12:03",
  "iso8601_date": "2024-01-03T12:03:00+00:00"
}

If you need to convert date to ISO8601 with the current timezone, please go to the fourth example.

How to Convert Date to ISO8601 Format in Logstash with date Filter

We can also use the date filter to convert the date to ISO8601 format.

If the date field is Jan 03 12:03:00, we can use the following Logstash configuration:

filter {
  date {
    match => ["date", "MMM dd HH:mm:ss"]
    target => "iso8601_date"
  }
}

The result would be:

{
  "date": "Jan 03 12:03:00",
  "iso8601_date": "2024-01-03T05:03:00.000Z"
}

How to Convert Date with Current Timezone to UTC ISO8601 Format in Logstash

As you can see in the first example and the second example, the timezone is +00:00. However, my current timezone is +07:00 as I'm in Vietnam. So, this is a wrong conversion.

In the third example, Logstash automatically detects my timezone and converts it to UTC time, which is correct and the one we want.

To add the timezone in the conversion, we can simply add our timezone to the end of the date string and replace iso8601() with to_time.

Let's try again with the following config:

filter {
  ruby {
    code => '
      date = event.get("date")
      event.set("iso8601_date", DateTime.parse("#{date} +07:00").to_time)
    '
  }
}

Applying the above config to this sample log:

{
  "date": "2024-01-03 22:00:00"
}

The result would be:

{
  "date": "2024-01-03 22:00:00",
  "iso8601_date": "2024-01-03T12:03:00+00:00",
  "iso8601_date": "2024-01-03T05:03:00.000Z"
}

This is exactly what we want, the correct time with timezone.