Re: Need help setting up filters for logstash for Mongo 3.0

From: Wan Bachtiar <wan.bachtiar@xxxxxxxxxxx>
To: mongodb-user <mongodb-user@xxxxxxxxxxxxxxxx>
Date: Tue, 19 Apr 2016 23:34:53 -0700 (PDT)
Why ads?


Did anyone set up the filters for logstash to parse logs for mongo 3? 

Hi, 

It’s been a while since you posted the question, have you found a solution 
for this ?

I ran a quick test for this on latest Logstash currently v2.3.1 
<https://www.elastic.co/guide/en/logstash/current/package-repositories.html
and latest MongoDB currently v3.2.5 
<https://www.mongodb.org/downloads#production>, and found that logstash 
already has support for MongoDB log v3+ format. 

An example conf file:

input {
    file {
        path => "/path/to/mongodb.log"
    }
}
filter {
    grok {
        match => [ "message", "%{MONGO3_LOG}"]
    }
}

Should be able to capture common log fields such as timestamp, severity, 
components, etc. For example:

{
       "message" => [
        [0] "2016-04-20T16:02:34.328+1000 I COMMAND  [conn3] command test.$cmd command: isMaster { isMaster: 1.0 } keyUpdates:0 writeConflicts:0 numYields:0 reslen:163 locks:{} protocol:op_command 0ms",
        [1] "command test.$cmd command: isMaster { isMaster: 1.0 } keyUpdates:0 writeConflicts:0 numYields:0 reslen:163 locks:{} protocol:op_command 0ms"
    ],
      "@version" => "1",
    "@timestamp" => "2016-04-20T06:02:35.213Z",
          "path" => "/path/to/mongodb.log",
          "host" => "hostname01",
     "timestamp" => "2016-04-20T16:02:34.328+1000",
      "severity" => "I",
     "component" => "COMMAND",
       "context" => "conn3"
}

For more patterns spec see logstash-patterns-core: mongodb_spec.rb 
<https://github.com/logstash-plugins/logstash-patterns-core/blob/master/spec/patterns/mongodb_spec.rb>

If you have further questions on logstash, you may get faster responses by 
posting a question on logstash forum <https://discuss.elastic.co/c/logstash>. 
See also Logstash Help 
<https://github.com/logstash-plugins/logstash-patterns-core#need-help>.

Best regards, 

Wan. 


-- 
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.

For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
--- 
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@xxxxxxxxxxxxxxxx.
To post to this group, send email to mongodb-user@xxxxxxxxxxxxxxxx.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/5181e40c-0201-4ef6-9aeb-b54d4ce4ce5e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Why ads?