API Studio¶
Collect from a REST API that has no Splunk add-on, such as an in-house application, a niche SaaS product or a regional service. Chuka describes the API from the vendor's documentation, Vyrux tests that description on the real API, and only then does the collector run. Open it under Data Engineering, API Studio. Only the vyrux_admin role sets up collectors; everyone with Vyrux sees their health.
How it works¶
Every collector runs the same collector engine that ships with Vyrux. The engine is fixed, tested code. What changes from one source to the next is the spec: a short JSON description of the API (its address, how it signs in, which request returns events, where the events and their times are in the response, and how it pages). No code is written or run per source.
| Step | What happens |
|---|---|
| 1. The source | Paste the endpoint's documentation or OpenAPI file, and optionally a real response. Say which events you want |
| 2. The spec | Chuka writes the spec from the documentation only. Anything the documentation does not say is listed as a question for you, not guessed |
| 3. Credentials | Enter the key, token, client secret or client certificate. They go straight into Splunk's password store |
| 4. Test | Vyrux reads three pages from the real API, then stops and restarts from the checkpoint. Nothing is written to any index |
| 5. Run it | Choose where it runs, how often, and the index. It turns on only after a passing test |
Vyrux never sends a request to a host named in pasted documentation before a person has seen it. When Chuka writes a spec for a host nobody has tested yet, the spec and its host are shown first, and the first request goes out when you select Test. After that, writing and repairing test on that host by themselves.
When a test fails, Chuka is given the failed step, the HTTP status and the start of the source's real answer, and repairs the spec. Writing, testing and repairing repeat up to three times. A refused sign-in, an address that cannot be reached or an untrusted certificate stops the loop, because a new spec cannot fix those.
What the engine handles¶
| Need | Supported |
|---|---|
| Sign-in | No auth, API key in a header or query, basic, bearer token, OAuth client credentials, OAuth with a person signing in once (refresh token), client certificates (mTLS) |
| Paging | Cursor, token, next-page link in the body or the Link header, offset, page number, and feeds followed with a next link |
| Time | Only new events each run, with an overlap so late events are not missed, a delay for events still being written, and a maximum window so a long outage is caught up in steps |
| Limits | The documented request rate, retries with backoff on network faults, 429 and 5xx, and Retry-After honoured |
| Duplicates | The overlap reads some events twice on purpose. Each event's id is remembered so it is written once |
| Restarts | The checkpoint is saved after every page. A restart carries on from the last saved page. Removing a collector deletes its checkpoint and credentials, and one made again under the same name starts fresh |
Trust and safety¶
- Tested before it runs. A collector turns on in Splunk only after that exact spec passed a test, checked on the Splunk side, not only in the browser. Any edit to how the API is read needs a new test. Changing only the index does not.
- Credentials. Kept in Splunk's password store under the collector. They are never shown again, never sent to Chuka and never in the forwarder add-on.
- Where credentials go. Only to the source's own host over HTTPS with its certificate checked. A redirect or next-page link to another host, or to plain HTTP, is refused. A host that belongs to the same service can be named in the spec's
hostson purpose. The page lists every host that receives credentials. - GET and POST only. A collector sends GET and POST requests, never PUT, PATCH or DELETE. Use POST only for an API that takes its search in the request body, and check the spec before it runs.
- Your company's proxy and CA. The spec can name a proxy and a CA file for TLS inspection.
- What Chuka sees. The documentation, and when a test fails, the start of the source's answer with personal data masked. With sample events turned off under Admin, AI, Chuka gets the documentation only.
Where it runs¶
| Option | Use it when |
|---|---|
| This Splunk server | The source can be reached from the search head. On a search head cluster one member runs each collector at a time. The checkpoint is in the KV store, so if that member stops, another takes over within two intervals without gaps or repeats |
| A heavy forwarder | The source allows only certain addresses, the volume is high, or you keep collection off the search head |
For a forwarder, Download the forwarder add-on. It holds the engine and the chosen collectors, with no credentials. Its README gives the one command per collector that sets the credentials on the forwarder, and the command that tests a collector from the forwarder without writing anything:
$SPLUNK_HOME/bin/splunk cmd splunkd print-modinput-config vyrux_api vyrux_api://<collector> | \
$SPLUNK_HOME/bin/splunk cmd python3 $SPLUNK_HOME/etc/apps/TA-vyrux-collectors/bin/vyrux_api.py --test
Saving a collector to run on a forwarder turns off any copy running here, so no event arrives twice.
Events in Splunk¶
Each record from the API is one event, with its own time from the record. The first time a sourcetype is used, Vyrux sets it to read JSON fields, never cut a long event and never join records. A sourcetype that already has settings, from an add-on or from you, is left as it is.
After it is on, Check events in Splunk counts the collector's events in its index. Make sure the index exists on your indexers or in Splunk Cloud, or Splunk drops the events. Then use Parse and map these events next to start an onboarding request from the tested sample, so the fields are mapped to CIM in the Parsing Studio like any other log.
Enterprise Security and the ad hoc search head¶
A collector writes its events to your indexers, so every search head that searches those indexers sees them: the Enterprise Security search head and the ad hoc search head alike. Collect in one place only, on the search head Vyrux runs on or on a heavy forwarder. A second copy of the same collector would bring every event in twice. If that happens, API Studio shows the collector as Collecting twice and names both servers, and the background scan alerts on it.
What a search head does not share is how it reads the events: field extractions, CIM mapping and tags. Add-on for other search heads, on the API Studio and Parsing Studio pages, downloads TA-vyrux-searchtime with every parser setting Vyrux has deployed and the settings of each collector sourcetype. Install it on the ad hoc search head, and on any other search head, so searches, dashboards and data models give the same fields everywhere. Settings that apply when data is indexed, such as line breaking and time stamps, belong where the data is parsed, so it is safe to install the same add-on on the indexers or the heavy forwarder too. Download it again after deploying new parser settings.
Health¶
Every run writes one line to var/log/splunk/vyrux_collector.log, which reaches _internal from any server, forwarders included. API Studio reads 24 hours of these lines.
| Health | Meaning |
|---|---|
| Healthy | Good runs, with events written in 24 hours and how far behind the newest event is |
| Retrying | The last run failed, but there was a good run within three intervals. The next run catches up from the checkpoint |
| Sign-in refused, Cannot reach the source, ... | No good run for three intervals, or 15 minutes. The source's own answer is shown |
| Collecting twice | Two servers ran the same collector within one and a half intervals, such as one copy on the ES search head and one on the ad hoc search head. Every event reaches Splunk twice. Keep it on one server only |
| Stopped | No run at all for three intervals. Splunk stopped running it, or its server is down |
| No runs reported | A forwarder collector with no run lines. The add-on is not installed or _internal is not forwarded |
| No new events | A full day with nothing new. The source may have stopped sending, or the time filter is wrong |
| Fields changed | The fields in the source's records changed. Check the parsing still maps them |
| Rate limited | The source slowed it down. It waited and caught up |
The background scan checks collectors every time it runs. A collector that is not collecting shows on the Data Engineering overview and is sent to alert channels that have An API collector fails turned on, once, and again only after it recovered and failed again. Channels that already sent silent logs get this alert too after the upgrade.
Sources that only send events¶
Some services can only push events by webhook. Source that sends events creates an HTTP Event Collector token for that source alone, locked to one index and sourcetype, so it can be revoked on its own. Point the source at your HEC endpoint: on Splunk Enterprise, your HEC load balancer or forwarders on port 8088; on Splunk Cloud, https://http-inputs-<stack>.splunkcloud.com. Then start an onboarding request for its sourcetype.
Load on the search head¶
A collector set to run here uses one Splunk modular input. Each run is limited to 80% of its interval, so runs never pile up, and a large backlog is caught up over several runs. Health is read from _internal only when API Studio is open and when the background scan runs, and only when collectors exist.