My current system is (vastly underpowered, 3.5gig a day tops) a single indexer/search head combo, and 2 heavy forwarders.
I have recently been given a requirement to bump this up to ~120GB a day indexed.
I am looking at this document to determine hardware requirements: http://docs.splunk.com/Documentation/Splunk/6.5.1/Capacity/Referencehardware but nowhere in here does it comment on a heavy forwarder.
My reading tells me that the HF does parsing before it ever sends data to the indexer. So, does that mean if I have a small lightweight VM acting as a heavy forwarder sending 100GB a day to the indexer with 12 cores+64gig ram, my indexer performance is mostly pointless, because my heavy forwarder is my bottleneck?
Should I plan my heavy forwarder to be the same spec as the indexer, or make my indexer underpowered and beef up the HF? (No logs go directly to the indexer.)
Or, do I keep my underpowered heavy forwarder VM and just convert it to use the universal forwarder? I would then make sure that all transforms/props/etc get placed on the indexers, not the forwarder.
The only thing on the forwarder I do that isn't just passthrough is adding a metadata tag "forwarder=locationX", which I guess I would have to find a substitute for. It is useful for me to track where a log originated, though.
↧