I have a script that creates a file, thoguh the command that is run, has a very long output, and it takes about 20 seconds to build the input. This script is ran every 60 seconds, and I use crcSALT = <SOURCE> to capture the input, due to this just being an output of a command with no timestamp.
I am getting discrepancies in data between a simply 'grep | wc -l' and splunk, and I'm pretty sure it's due to timing. I think the forwarder is ingesting the file before it is fully populated, so it only gets a portion of the data, but it still misses some things.
My question is, how do I get the forwarder to either wait the 10 seconds it takes to populate that file every minute, or to know that the who file hasn't loaded yet, and wait until the file itself has fully populated before ingestion?
so far in inputs.conf I have attempted:
followTail = 0
time_before_close = 0
neither of them had an effect.
Thank You!
↧