Hi, i'm sorry for my poor English, I have a Windows Universal Splunk Forwarder 6.5.0 and a CentOS Splunk Entreprise 6.5.0. I added a new index. I edited and added `index = myindex" to "etc\apps\SplunkUniversalForwarder\local\inputs.conf`, restated windows service, and tested again with "etc\system\local\inputs.conf", but no data is forwarded in my index or in the main index with default conf. Thank you for your help
↧
How can I forward data in Windows Universal Splunk Forwarder 6.5.0 and a CentOS Splunk Entreprise 6.5.0?
↧
Splunk Universal forwarder 6.6.4 installation on Tru64 UNIX V5.1B
Hi All,
I am trying to install the universal forwarder on a UNIX Tru64 server. I am using the zip version of splunkforwarder-6.6.4-00895e76d346-FreeBSD9-amd64.tgz the universal forwarder. After extracted the zip in my server i tried to start the splunk forwarder using **./splunk start**, but I'm receiving **cannot execute**. Any suggestion will help me a lot.
Thanks in advance.
↧
↧
kubernetes 1.9.4 breaking changes: Universal Forwarder
I've setup splunk universal forwarder as a daemonset on our kubernetes cluster. 2 nodes are running kuberntes 1.9.3 and one is running 1.9.4. On the 1.9.4 node the splunk forwarder pod is unable to start:
`
chown: changing ownership of ‘/opt/splunk/etc/system/local/inputs.conf’: Read-only file system
chown: changing ownership of ‘/opt/splunk/etc/system/local/..2018_03_15_23_51_19.952137038/inputs.conf’: Read-only file system
chown: changing ownership of ‘/opt/splunk/etc/system/local/..2018_03_15_23_51_19.952137038/SPLUNK_FORWARD_SERVER’: Read-only file system
chown: changing ownership of ‘/opt/splunk/etc/system/local/..2018_03_15_23_51_19.952137038’: Read-only file system
chown: changing ownership of ‘/opt/splunk/etc/system/local/SPLUNK_FORWARD_SERVER’: Read-only file system
chown: changing ownership of ‘/opt/splunk/etc/system/local/..data’: Read-only file system
chown: changing ownership of ‘/opt/splunk/etc/system/local’: Read-only file system
`
I believe this is related to changes recently made in 1.9.4:
https://github.com/kubernetes/kubernetes/pull/58720
Wondering if anyone has come across this or has a workaround?
thanks
Garry
↧
How to index .EVTX file stored in a different location on a universal forwarder?
HI All,
I would like to index .evtx file stored in a different location in my universal forwarder.
E:\Logs\Events\Fixed.Evtx
What are the approaches we have, to index these files?
I read some documentation but with few concerns, like it should not be written while read by splunk? if so, how can we achieve this?
Regards,
BK
↧
How to repack the installation package MSI for Splunk universal forwarder?
Hey guys
Is it possible to rebuild msi installation package for Splunk Forwarder?
I want to assemble my installation package (msi) with pre-installed data, such as IP servers, the selected log, the account with which UF will be started, and possibly other parameters.
Unpacking and packing are very simple but in which file has stored the parameters that can be set initially, I can not find.
Where is this file and what is it called?
↧
↧
Universal Forwarder 6.4.0 to HEC
Unfortunately, I have a few hundred hosts running 6.4 universal forwarder and I cannot upgrade them. I have a subset of hosts that need to send an application log to HEC on customer's splunk deployment but still send this and all the other logs to our splunk. We already have a few apps for the forwarder to handle different logs in a different way so I decided to create an app for those forwarders to send to the customer so that I can make sure that only the application log gets sent to them.
Hosts are Ubuntu 14.04
Splunk servers are on 16.04 and we run Splunk Enterprise 6.5.2
We don't use HEC in any of our hosts hence why I'm asking for help here.
I'm confident about all the config files except output.conf, I'm not clear on what defaultGroups does or if it's even necessary and the tcpout:app_upstart_logs stanza is also nebulous to me.
output.conf
[tcpout]
defaultGroup = ???
forwardedindex.filter.disable = true
indexAndForward = false
maxQueueSize = 250MB
maxConnectionsPerIndexer = 5
[tcpout:app_upstart_logs]
useACK = false
token = XXXXXXXXXXXXXXXXXXXXXXXXXX
server = customer.splunk-server.net:8080
This is my inputs.conf
[monitor:///var/log/upstart/]
disabled = false
sourcetype = app_upstart_logs
blacklist = (\.gz$|\.0$|.1$|\.2$|.3$|\.4$|.5$|\.6$|.7$|\.8$|\.9$|\.10$|\.report$|lost\+found)
PS: We are running some really old stuff. Feel free to roll your eyes like a teenage valley girl (is that still a thing?) I stand ashamed.
↧
What are the house keeping activities we can do in Splunk apart from clearing dispatch directory?
Been trying to create a manual for doing a daily house keeping activities on Splunk and Universal forwarder to make the product work better. Please kindly suggest the same
↧
How can I disable Splunk Universals Forwarder input after installing Splunk TA_windows via deployment?
I currently have a Splunk Universal Forwarder installed on all my servers. It was recommended by Splunk to install the TA_windows plug-in on top of the Universal Forwarder. I built out a deployment from the Search Head to deploy TA_windows add-on to my servers but I noticed the regular Universal forwarder input.conf is still active/enabled along with the TA_windows add-on.
How can I disable the regular Universal Forwarder app automatically when using the TA_windows add-on.
↧
ERROR TcpInputProc - Indexer not receiving data from forwarder
Hi all, I am getting these errors in my log files. First is from the spunkd.log from the indexer and second is is from the splunkd.log on the forwarder. I have done multiple searches on Splunk answers, but I haven't found one that pertain to both. It obvious in the error log on the forwarder that the connection is refused however I can telnet to the port 9997. What am I missing? This was all working until upgrading to 7.02. Thankfully this is just a test machine and not in production. Please let me know what I can provide you all to assist me in troubleshooting such as .conf/log files etc. I will continue to search & troubleshoot, but at this point I am loss.
Splunk IDX Error:
ERROR TcpInputProc - Message rejected. Received unexpected message of size=369295616 bytes from src=xxx.xx.xxx.xx:64529 in streaming mode. Maximum message size allowed=67108864. (::) Possible invalid source sending data to splunktcp port or valid source sending unsupported payload.
ERROR TcpInputProc - Message rejected. Received unexpected message of size=369295616 bytes from src=xxx.xx.xxx.xx:61330 in streaming mode. Maximum message size allowed=67108864. (::) Possible invalid source sending data to splunktcp port or valid source sending unsupported payload.
Splunk UF Error:
WARN TcpOutputProc - Applying quarantine to ip=xxx.xx.xxx.xx port=9997 _numberOfFailures=2
WARN TcpOutputProc - Tcpout Processor: The TCP output processor has paused the data flow. Forwarding to output group primary_indexers has been blocked for 3601 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.
INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_xxx.xx.xxx.xx_8089__XA5D5CF2-F5DB-4F1F-BAE9-909B3A7FEA00
INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_xxx.xx.xxx.xx_8089__XA5D5CF2-F5DB-4F1F-BAE9-909B3A7FEA00
WARN TcpOutputProc - Tcpout Processor: The TCP output processor has paused the data flow. Forwarding to output group primary_indexers has been blocked for 3701 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.
INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_xxx.xx.xxx.xx_8089_ _XA5D5CF2-F5DB-4F1F-BAE9-909B3A7FEA00
WARN TcpOutputProc - Tcpout Processor: The TCP output processor has paused the data flow. Forwarding to output group primary_indexers has been blocked for 3801 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.
INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_xxx.xx.xxx.xx_8089__XA5D5CF2-F5DB-4F1F-BAE9-909B3A7FEA00
INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_xxx.xx.xxx.xx_8089__XA5D5CF2-F5DB-4F1F-BAE9-909B3A7FEA00
INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_xxx.xx.xxx.xx_8089__XA5D5CF2-F5DB-4F1F-BAE9-909B3A7FEA00
WARN TcpOutputProc - Tcpout Processor: The TCP output processor has paused the data flow. Forwarding to output group primary_indexers has been blocked for 3901 seconds. This will probably stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.
INFO TcpOutputProc - Removing quarantine from idx=xxx.xx.xxx.xx:9997
ERROR TcpOutputFd - Connection to host=xxx.xx.xxx:9997 failed
ERROR TcpOutputFd - Connection to host=xxx.xx.xxx:9997 failed
Thank You
↧
↧
Understanding/Control over the hourly fishbucket snapshots
Hi,
A couple days ago I posted a question regarding hourly CPU spikes on UF. It was found that the hourly fishbucket snapshots causes a brief CPU and I/O spike.
My new question: In order to reduce the CPU and I/O impact, do we have any control over this process? Anything we could do that would result in reduced CPU usage?
Thank you.
[1]: https://answers.splunk.com/answers/626127/why-are-the-universal-forwarders-cpu-spiking-every.html
↧
What are the steps to install forwarder on Tru64 UNIX V5.1B server?
Hi All,
I am trying to install the universal forwarder on a Tru64 UNIX V5.1B alpha system. Please help me with the steps to achieve the same. Any ideas or suggestions will be much appreciated.
Thanks
↧
How to have control over the hourly fishbucket snapshots in order to reduce the CPU and I/O impact?
Hi,
A couple days ago I posted a question regarding hourly CPU spikes on Universal Forwarder. It was found that the hourly fishbucket snapshots cause a brief CPU and I/O spike.
My new question: In order to reduce the CPU and I/O impact, do we have any control over this process? Anything we could do that would result in reduced CPU usage?
Thank you.
[1]: https://answers.splunk.com/answers/626127/why-are-the-universal-forwarders-cpu-spiking-every.html
↧
Can SNMP Modular Input be installed in the Universal Forwarder?
Hi, our requirement is to install SNMP Modular Input but we are not sure yet how and where are we required to configure it in Splunk deployment? Please help.
Thanks!
↧
↧
What are the steps to install a universal forwarder on Tru64 UNIX V5.1B server?
Hi All,
I am trying to install the universal forwarder on a Tru64 UNIX V5.1B alpha system. Please help me with the steps to achieve the same. Any ideas or suggestions will be much appreciated.
Thanks
↧
splunk universal forwarder batch input forwarding but not deleting
Hi, we have an indexer cluster, to which we index many many small files.
we have about a few hundreds thousand files.
we run a universal forwarder on a strong machine(130GB 24CPU) and have a batch input on local directory.
our problem is as follows:
the data is indexed very slowly, and also the batch input is freaking a little....
it used to write logs about every indexed file("Batch input finished reading file..."), but now it writes a few, than stops, than continue to forward data but doesn't delete the files.
the only log we can see is when we turn on DEBUG level logging.
I have checked the logs and I dont have any blocked queues.
We would really appreciate if anyone would either have a reasonable explanation for the problem i'm having, or if someone will be able to suggest another way of indexing this immense amount of files.
↧
Why are Windows event logs with MSSQLSERVER$AUDIT as source getting truncated and the message is empty?
Hi,
We have an auditing setup which logs in Windows event logs (Forwarded Events) as "MSSQLSERVER$AUDIT" source.
they are well displayed in event viewer console, but the log is truncated and message is empty :
![alt text][1]
here the input file on windows server :
![alt text][2]
on the other side "Microsoft Windows Security auditing" events that are in "Forwarded Events" too are correctly sent and parsed in Splunk Indexer.
how come ?
where elsewhere do I have to check ?
*Splunk UF is installed on Windows server with the Windows Events Collector*
Thanks
[1]: /storage/temp/234579-splunk.png
[2]: /storage/temp/234580-splunk2.png
↧
How to configure Splunk Stream on Windows?
Hi!
Having some trouble configuring windows to collect data from a Windows forwarder(UF). I have a heavy forwarder configured with token where I also have Splunk_TA_stream installed.
On the search head I have both TA_stream and the actual stream app.
On the Windows forwarder, I have pushed out the TA_stream app with inputs.conf pointing towards the search head. I have also made sure the FW openings have been made so that not an issue.
However can't seem to get a connection to the Windows server. I have it configured on a Linux host which works fine.
Read something about WinPcap. I found the docs a bit confusing here though. Something I need to manually install?
Does anyone have other tips or "good to know" knowledge when it comes to stream and windows forwarder?
Perhaps to get some help I need to specify more info. Let me know in that case!
Thanks!
↧
↧
New Universal Forwarder read timeout
We are trying to setup the universal forwarder on a Windows AD server. After configuring the index to receive on port 9997 and installing the UF on the server. The Forwarder does not appear under the Data Inputs/Windows Event Log of Forwarded inputs.
I have verified the firewall is allowing packets on port 9997.
I have verified using tcpdump that packets are being received on port 9997.
I have checked the splunkd.log and found the error indicating TcpInputProc connection from Read Timeout Timed out after 600 seconds.
Documentation indicated sslVersion possible issue - verified the sslVersion on both the inputs.conf of the indexer and the web.conf and outputs.conf of the UF.
Documentation indicated the internal queue on the indexer may be blocked, which causes a timeout after 600 seconds.
How do I find the inrernal queue and troubleshoot if it is blocked?
Thanks in advance for any suggestions.
J
↧
Why can't I download universal forwarder credentials on macOS High Sierra 10.13.3?
When I tried to download the Universal Forwarder Credentials from my trial Splunk Cloud on to my MacBook Pro, I got a prompt stating
"This type of file can harm your computer. Do you want to keep splunkclouduf.spi anyway?". There is a "Discard" button next to the prompt and clicking on the select box, the choices are "Open When Done", "Pause", "Show in Finder", and "Cancel". Selecting "Open When Done", however, does not download the file.
↧
Remove UF from RHEL7
I need to remove UFs from some REHLs.
I stopped splunk and disabled boot-start.
I installed .rpms but rpm -e is not working.
Should I cd /opt and rm -rf splunkforwarder ?
Thank you
↧