Hi,
I'm getting errors with parsing of json files in the universal forwarder.
I'm generating json outputs - a new file is generated every time a run a routine. Output has the below:
[
{
"datetime":"2017-10-25 14:33:16+01:00",
"user":"",
"category":"ST",
"type":"ABC",
"frontend":"3.0",
"backend":"",
"r_version":"",
"b_version":"",
"status":"R",
"next_planned_r_version":"",
"next_planned_b_version":"",
"comment":""
}
]
Splunk forwarder gives me the following log entries in *splunkd.log*:
10-25-2017 14:33:16.273 +0100 ERROR JsonLineBreaker - JSON StreamId:16742053991537090041 had parsing error:Unexpected character: ':' - data_source="/root/status-update/environment_health_status_50.json", data_host="hostxyz", data_sourcetype="_json"
The line above repeats about the same number of lines with ":" in the output. Then lines below:
10-25-2017 14:33:16.273 +0100 ERROR JsonLineBreaker - JSON StreamId:16742053991537090041 had parsing error:Unexpected character: '}' - data_source="/root/status-update/environment_health_status_50.json", data_host="hostxyz", data_sourcetype="_json"
10-25-2017 14:33:16.273 +0100 ERROR JsonLineBreaker - JSON StreamId:16742053991537090041 had parsing error:Unexpected character: ']' - data_source="/root/status-update/environment_health_status_50.json", data_host="hostxyz", data_sourcetype="_json"
I've tried universal forwarders versions 7.0 and 6.5.3.
I've been trying to isolated the root cause but had no luck with that - even without changing anything. Sometimes it goes fine, but mostly it doesn't. If I stop splunk, erase fishbucket and start it again, it will ingest all files just fine. However, when I run my test afterwards that is creating new files, it will fail. (or not, as I explained).
monitor in inputs.conf:
[monitor:///root/status-update/environment_health_status_*.json]
index=dev_test
sourcetype=_json
_json stanza on the forwarder by using btool:
*PS:* I haven't made any config in props.conf, only inputs.
[_json]
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE =
BREAK_ONLY_BEFORE_DATE = True
CHARSET = UTF-8
DATETIME_CONFIG = /etc/datetime.xml
HEADER_MODE =
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER =
MUST_NOT_BREAK_AFTER =
MUST_NOT_BREAK_BEFORE =
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS =
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = false
maxDist = 100
priority =
pulldown_type = true
sourcetype =
↧