We are collecting syslog with a syslog collector, and dumping it to text files. Splunk ingests those txt files from the drive using the Splunk Universal Forwarder and everything works perfectly for all syslog events except the switch data from sourcetype cisco:ios. Every night there is a gap in the data from 12a-4a. Meanwhile, all *other* syslog data is indexed and reporting properly with nothing missing. Every sourcetype is using the same method and source syslog server. Its only this cisco:ios sourcetype during these hours. At 4:00am everything resumes like nothing ever happened. The text files contain data straight through the night, so its not with the syslog server or the data collecting.
I am completely stumped.
Backups dont run at those times.
Has anyone ever seen anything like this? I feel like my sanity is being tested :)
↧