Regex for Heka to read Monolog logfile

Question

I am trying to get Heka to read my Monolog files (so I can view them using Kibana), but it keeps telling me there is no match.

Here is my match_regex from my Heka config:

match_regex = "^\\[(?P<Timestamp>.*)\\] (?P<Channel>.+?)\\.(?P<Severity>[A-Z]+): (?P<Message>.*)"

And here is a sample line from the log

[2014-03-24 19:07:08] event.DEBUG: Notified event "kernel.terminate" to listener "Symfony\Component\HttpKernel\EventListener\ProfilerListener::onKernelTerminate"

according to regex101 (see here) the regex is perfectly fine. Yet hela tells me no match.

Could someone please point me in the right direction, I spent all day yesterday trying to get this to work.

EDIT: the only documentation I can seem to find regarding Heka and Monolog are these slides here

EDIT: I just found this : https://regoio.herokuapp.com/

This tells me that it should work using the following:

match_regex = '^\[(?P<Timestamp>.*)\] (?P<Channel>.+?)\.(?P<Severity>[A-Z]+): (?P<Message>.*)'

But it's not.

Answer 1

clearly i was not properly awake yesterday, after making sure the regex was correct 15 times, i realised i had forgotten

[LogOutput]
 message_matcher = "TRUE"

[ElasticSearchOutput]
 message_matcher = "TRUE"

in my hekad.toml file

My complete file now looks like this :

[pos_log]
type = "LogfileInput"
logfile = "/home/sam/git/PosBranch/app/logs/dev.log"
decoder = "monolog_decoder"

[monolog_decoder]
type = "PayloadRegexDecoder"
match_regex = '^\[(?P<Timestamp>.*)\] (?P<Channel>.+?)\.(?P<Severity>[A-Z]+): (?P<Message>.*)'
timestamp_layout = "2006-01-02 15:04:05"
timestamp_location = "UTC" # optional, default value

[monolog_decoder.severity_map]
DEBUG = 7
INFO = 6
NOTICE = 5
WARNING = 4
ERROR = 3
CRITICAL = 2
ALERT = 1
EMERGENCY = 0

[monolog_decoder.message_fields]
Type = "monolog_log"
Logger = "sam"
Hostname = "local"
Channel = "%Channel%"
Message = "%Message%"
Payload = ""

[LogOutput]
 message_matcher = "TRUE"

[ElasticSearchOutput]
 message_matcher = "TRUE"

and messages are now rolling into kibana :)