SexiLog listens on several ports for log sourcing:
UDP/514for syslog protocol: A Lot of grok magic designed for VMware ESXi™ here but you can also send anything that looks like syslog. We designed a very tolerant filter 🙂
UDP/162for SNMP traps internaly forwarded as syslog: A bit of grok magic here too, designed for VMware ESXi™ and Veeam B&R™ but you can send any SNMP traps as well
UDP/1514for vCenter logs in json forwared by nxlog agent: The grok filters here are dedicated to vpxd and vpxd-profiler exclusively
UDP/1515for Windows eventlog forwarded by nxlog agent from your Windows vCenter: You can forward from any Windows as well
SexiLog is loaded with a lot of logstash grok filters to enhance VMware ESXi™ logs (mostly).
Here are some examples of few messages where colorized parts are parsed and “fielded” in elasticsearch as you can see in screenshots below:
<14>2014-12-10T18:01:03.496Z esx.vmware.com vobd: [scsiCorrelator] 14807183227307us: [esx.problem.scsi.device.io.latency.high] Device naa.60a9800041764b6c463f437868556b7a performance has deteriorated. I/O latency increased from average value of 1343 microseconds to 28022 microseconds.
<181>2015-01-28T11:30:29.653Z esx.vmware.com vmkernel: cpu4:19834926)NMP: nmp_PathDetermineFailure:2084: SCSI cmd RESERVE failed on path vmhba0:C0:T1:L15, reservation state on device naa.6006016084b02800b4a07969cd74e011 is unknown.
<166>2015-01-25T20:16:44.502Z esx.vmware.com Hostd: [34F99B90 verbose ‘vm:/vmfs/volumes/548076ca-1a5e9feb-b886-fc15b415a120/vm_name/vmx_name.vmx’] Handling message _vmx3: There is no more space for virtual disk PARG1TZCTXWEB02.vmdk. You might be able to continue this session by freeing disk space on the relevant volume, and clicking Retry. Click Cancel to terminate this session.
Sexilog comes with pre-configured kibana dashboards (aka SexiBoards) where you’ll find critical information about your VMware vSphere™ infrastructure.
The one you want to look at first and as often as possible (think wallscreen) is SexiBoard:Kommandantur
During on-call duty, lunch, beer or if you are AFK, you need a sharp and clear alerting system, that’s where Riemann comes in. Logstash forwards filtered warnings and alerts to Riemann where they’re rolled up by type and send by e-mail.
Medium alerts are aggregate and sent every hour. Critical ones are sent every minutes and have an
* (asterisk) sign in the subject.
We configure Riemann e-mails with a strict format that will let you know with a single look what’s happening and where. Here are format details: