Configuring Logstash

Any Questions/Issues regarding WinForwarder should go here
mreggio
Site Admin
Posts: 12
Joined: Thu Feb 19, 2009 3:10 pm

Configuring Logstash

Postby mreggio » Sat Mar 08, 2014 2:54 am

Depending on the port and protocol selected in the configuration you need to adjust your logstash settings accordingly. Due to changes in the processing of json data in Logstash the configuration for 1.4.x is different to 1.3.x. I have given examples below of what needs to be configured for logstash to be able to process the json correctly.

LOGSTASH 1.3.x
Below demonstrates a port configuration of 33444 and a TCP protocol selection:

Code: Select all

  tcp {
     port => 33444
     codec => json
  }


Below demonstrates a port configuration of 33444 and a UDP protocol selection:

Code: Select all

  udp {
     port => 33444
     codec => json
  }


LOGSTASH 1.4.x
Below demonstrates a port configuration of 33444 and a TCP protocol selection:

Code: Select all

  tcp {
     port => 33444
     codec => json_lines
  }


Below demonstrates a port configuration of 33444 and a UDP protocol selection:

Code: Select all

  udp {
     port => 33444
     codec => json_lines
  }


The primary requirement is to ensure the "codec" is set at "json" for Logstash 1.3.x and "json_lines" for Logstash 1.4.x.

I'm working on a template for logstash but for now to get the data into elasticsearch properly with the correct type you will want to add a filter as follows:

Code: Select all

filter {
  if [@tags] == "[disk,space]" {
    mutate {
      convert => ["[disktotal].[C]","float"]
      convert => ["[diskfree].[C]","float"]
      convert => ["[disktotal].[D]","float"]
      convert => ["[diskfree].[D]","float"]
      convert => ["[disktotal].[E]","float"]
      convert => ["[diskfree].[E]","float"]
      convert => ["[disktotal].[F]","float"]
      convert => ["[diskfree].[F]","float"]
      convert => ["[disktotal].[G]","float"]
      convert => ["[diskfree].[G]","float"]
    }
  }

  if [@tags] == "[memory]" {
    mutate {
      convert => ["[details].[availablephysical]","float"]
      convert => ["[details].[totalmemory]","float"]
      convert => ["[details].[percentfree]","float"]
    }
  }

  if [@tags] == "[process]" {
    mutate {
      convert => ["[details].[processid]","float"]
      convert => ["[details].[processmemoryworkingset]","float"]
      convert => ["[details].[processmemoryprivatebytes]","float"]
      convert => ["[details].[processcpu]","float"]
    }
  }

  if [@tags] == "[machinecpu]" {
    mutate {
      convert => ["[details].[machinecpu0]","float"]
      convert => ["[details].[machinecpuuser0]","float"]
      convert => ["[details].[machinecpu1]","float"]
      convert => ["[details].[machinecpuuser1]","float"]
      convert => ["[details].[machinecpu_Total]","float"]
      convert => ["[details].[machineuser_Total]","float"]
    }
  }
}

Return to “WinForwarder”

Who is online

Users browsing this forum: No registered users and 0 guests