Step 1

Set Up the Lab

Get Splunk Enterprise running locally in under five minutes using Docker Compose. No Splunk licence is required — the free trial tier covers everything in this lab.

Prerequisites

🐳

Docker Desktop

Install and make sure it's running before you start.

🤖

Claude Code or Claude Desktop

Claude Code v2.1+ or Claude Desktop v0.10.5+ for MCP integration. Optional — Ask Splunk works in the browser without it.

Installation

Clone the repository

bash
git clone https://github.com/andrewkriley/splunk-lab.git cd splunk-lab
Configure your password

Copy the example environment file and set a password for the Splunk admin account.

bash
cp .env.example .env

Open .env and set your password:

.env
SPLUNK_PASSWORD=YourPassword123

ℹ️ Password must be at least 8 characters and contain both letters and numbers.
Start the stack

bash
docker compose up -d

Splunk takes 60–90 seconds to initialise on first boot. Watch the logs:

bash
docker compose logs -f splunk

When you see Ansible playbook complete, Splunk is ready.
Open Splunk

Navigate to http://localhost:8000 and log in with:

Usernameadmin Passwordthe value you set in .env

ℹ️ Before you run Buttercup searches in Search & Reporting, set the time range to All time (sample data is dated January 2025; Last 24 hours usually shows no events). Step 2 copies SPL that includes earliest=0 latest=now so results work regardless of the picker.

Step 2

Explore Buttercup Data

The lab ships with Buttercup Games sample data — a fictional online game store. Three datasets are automatically indexed at startup with no manual import needed.

What's in the data

Sourcetype	Description	Index
`buttercup_web`	Web storefront access logs (Apache Combined format)	`buttercup`
`buttercup_sales`	Vendor sales transactions — units, revenue, product	`buttercup`
`buttercup_products`	Product catalogue with prices and categories	`buttercup`

Try these searches

Open Search & Reporting in Splunk and run the queries below. The sample data is from January 2025; Splunk’s default time range is often “last 24 hours,” which would show no events. Each query includes earliest=0 latest=now so it matches all indexed data — or set the picker to All time and you can omit those terms.

Traffic by HTTP status code

index=buttercup earliest=0 latest=now sourcetype=buttercup_web | stats count by status

Sales revenue by vendor

index=buttercup earliest=0 latest=now sourcetype=buttercup_sales | stats sum(revenue) as total_revenue by vendor | sort -total_revenue

Connect to MCP

🔒 The MCP server has no authentication and listens on 127.0.0.1:8050 only — fine for this local lab, but do not expose that port to untrusted networks.

The Splunk MCP server runs alongside Splunk and exposes tools for search, dashboards, and alerts over the Model Context Protocol. Use Ask Splunk in the sidebar for a browser UI, or connect Claude Code / Claude Desktop using the tabs below.

Claude Code

Claude Desktop

No setup required

Claude Code v2.1+ uses Streamable HTTP transport and connects directly to the MCP server — no Node.js, npx, or proxy process needed. The config is committed in this repo as .mcp.json and is picked up automatically when you open Claude Code in the splunk-lab directory with the stack running.

.mcp.json

{
  "mcpServers": {
    "splunk-lab-guide": {
      "type": "http",
      "url": "http://localhost:8050/mcp"
    }
  }
}

If Splunk tools do not appear, confirm the stack is running (docker compose ps) and check that Claude Code is open inside the splunk-lab directory. Run claude mcp list to verify the server is registered.

Try these prompts

Search Splunk for the top 5 vendors by revenue
Show me HTTP error rates from the Buttercup web logs
Create a dashboard showing web traffic by status code
What products have the highest units sold this month?

Add the MCP server

Claude Desktop v0.10.5+ supports Streamable HTTP natively — no Node.js or proxy required. Open the config file, add the mcpServers block below, then restart Claude Desktop.

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json

json

{
  "mcpServers": {
    "splunk-lab-guide": {
      "type": "http",
      "url": "http://localhost:8050/mcp"
    }
  }
}

Restart Claude Desktop — the Splunk tools will appear in the tool list whenever the lab stack is running.

If tools do not appear after restarting, confirm Claude Desktop is v0.10.5 or later. Older versions do not support Streamable HTTP and require a different configuration — upgrade Claude Desktop to use the config above.

Try these prompts

Search Splunk for the top 5 vendors by revenue
Show me HTTP error rates from the Buttercup web logs
Create a dashboard showing web traffic by status code
What products have the highest units sold this month?

Troubleshooting

Symptom	Fix
Tools don't appear in Claude Code	Confirm the stack is running (`docker compose ps`), then run `claude mcp list` to verify `splunk-lab-guide` is registered. Make sure Claude Code is opened from inside the `splunk-lab` directory — `.mcp.json` is project-scoped.
Tools don't appear in Claude Desktop	Restart Claude Desktop after editing the config — it only reads the file at startup. Confirm the version is v0.10.5+ and that the config uses `"type": "http"`, not the old `command`/`args` form.
MCP shows unhealthy in Status tab	Check container state: `docker compose ps`. View MCP logs: `docker compose logs splunk-mcp`. The MCP server waits for Splunk — if Splunk is still starting, wait for `Ansible playbook complete` in `docker compose logs -f splunk`.
Connection refused on port 8050	The lab stack is not running. Start it: `docker compose up -d`

Step 4

Your first dashboard

With Claude Code open in this repo and the stack running, use the project skill splunk-lab-dashboard-gen to build a Dashboard Studio view from real Buttercup data, generate an AI background image, and deploy it to Splunk in one flow.

ℹ️ This step needs Claude Code on your machine. If you only use the browser, use Lab Tools → Ask Splunk to run searches instead — dashboard generation is not available there yet.

Before you start

🐳

Stack up

docker compose up -d — Splunk and MCP must be healthy (see Status in the sidebar).

🔌

MCP servers

splunk-lab-guide is in .mcp.json. Connect the HuggingFace MCP in Claude Code for background images.

🔑

.claude/env.sh

Gitignored file with SPLUNK_PASS (match .env), a minted SPLUNK_API_TOKEN when install succeeds, and HF_TOKEN for Hugging Face. Run ./install.sh and accept the skill prompt, or copy from env.sh.example.

Invoke the skill

In Claude Code, use the slash command or describe what you want. Claude will ask for index, SPL, and dashboard title if they are missing. Buttercup data is dated January 2025 — the skill already uses a wide earliest_time for MCP searches.

Slash command (sets dashboard title)

/splunk-lab-dashboard-gen Buttercup Web Traffic

Natural language — Buttercup web logs by HTTP status

Generate a dashboard: index buttercup, SPL is sourcetype=buttercup_web | stats count by status, title Buttercup Web by Status

When the run finishes, open the live link Claude returns (under app/search/). Artefacts are also written under ~/dev/claude-created-dashboards/<slug>/ on your machine.

Learn more

See Lab Tools → Skills for prerequisites, troubleshooting, and how the skill uses MCP plus Splunk’s REST API.

Containers

—

Services

—

OpenTelemetry Stack

—

Documentation

Architecture, configuration reference, and guides for the Splunk Lab.

Architecture

Four public-facing Docker Compose services on the host (Splunk, MCP, lab guide, status sidecar). The Ask Splunk backend runs internally and is reached through the lab guide. All host ports are bound to 127.0.0.1 — accessible from your machine only.

Service	Image / Build	Port	Role
splunk	`splunk/splunk:10.2.1`	`8000` `8088` `8089`	Splunk Enterprise — Web UI, HEC, REST API. Buttercup sample data auto-indexed on boot.
splunk-mcp	`mcp/Dockerfile`	`8050`	MCP server (Streamable HTTP). Bridges Claude / Ask Splunk to Splunk search, indexes, and saved searches.
lab-guide	`nginx:alpine`	`${LAB_GUIDE_PORT}`	This guide — static HTML plus nginx proxies: `/api/status` → status-api, `/ask/api/*` → Ask Splunk backend (FastAPI on `chat:3000`, internal only).
chat	`chat/Dockerfile`	(internal)	Ask Splunk FastAPI — MCP Tool Explorer + optional Chat. Not exposed on the host; use Lab Tools → Ask Splunk here or open `/ask/` under the lab guide URL.
status-api	`status-api/Dockerfile`	8081 (internal)	Health-check sidecar. Probes container states and service endpoints via Docker API.

Data flow

Browser ──▶ Lab guide (/ask/) ──▶ Ask Splunk API (chat) ──▶ MCP (:8050) ──▶ Splunk (:8089)
              │                                    │
              │                                    └──▶ Anthropic API (Chat tab only, when API key set)
                                         │
                    ┌────────────────────┼────────────────────┐
                    ▼                    ▼                    ▼
             buttercup_web      buttercup_sales      buttercup_products

Networking

Containers communicate on the splunk-lab_default bridge network using service names as hostnames (e.g. splunk-mcp connects to splunk:8089). Host port bindings are 127.0.0.1-only — not accessible from other machines. No authentication on the MCP endpoint by design.

Volumes

Volume / Mount	Purpose
`splunk-var`	Named volume — persists Splunk indexes and data across restarts. Deleted on `docker compose down -v`.
`./buttercup_app`	Bind mount — Buttercup sample data app auto-loaded by Splunk at boot.
`./lab-guide`	Bind mount (read-only) — this lab guide, served by nginx.
`/var/run/docker.sock`	Bind mount (read-only) — gives status-api read access to container states.

Configuration Reference

All configuration lives in .env (gitignored). Copy .env.example to .env before starting. Variables are read by Docker Compose at startup.

Required

Variable	Default	Description
`SPLUNK_PASSWORD`	`Chang3d!`	Splunk `admin` password. Must be ≥8 chars with letters + numbers.

Optional

Variable	Default	Description
`SPLUNK_HEC_TOKEN`	`a8b4c2d6-…`	Pre-set HEC token. Any UUID works — set before boot so you can send events immediately.
`LAB_GUIDE_PORT`	`3131`	Host port for this lab guide (includes Ask Splunk at `/ask/`).
`ANTHROPIC_API_KEY`	unset	Enables the Chat tab in Ask Splunk. The Explore Tools tab works without it.

MCP server environment

Set automatically by Docker Compose. Only change if you know what you're doing.

Variable	Value	Notes
`SPLUNK_HOST`	`splunk`	Hostname only — the MCP client prepends `https://`.
`SPLUNK_PORT`	`8089`	Splunk REST API port (internal).
`VERIFY_SSL`	`false`	Splunk uses a self-signed cert locally.
`TRANSPORT`	`http`	MCP transport mode. Streamable HTTP (MCP spec 2025-03-26) — no persistent SSE connection needed.

Common commands

Start the lab

docker compose up -d

Watch Splunk startup (ready when "Ansible playbook complete" appears)

docker compose logs -f splunk

Stop containers (data preserved)

docker compose down

Full reset — wipe all indexed data

docker compose down -v

Run integration tests

pip install -r requirements-test.txt && pytest tests/ -v

Coming Soon

🔭

OpenTelemetry Setup

Connecting an OpenTelemetry collector and visualising traces and metrics in Splunk.

Coming soon

🔒

Security Use Cases

Threat detection, anomaly hunting, and SIEM-style searches using the Buttercup data.

Coming soon

Skills

Claude Code skills extend what Claude can do from this project directory. The skills below are project-scoped — they activate automatically when Claude Code is opened inside splunk-lab with the stack running.

splunk-lab-dashboard-gen

Generates a Splunk Dashboard Studio dashboard with an AI-generated background image and deploys it live — all from a single Claude prompt.

🔌

splunk-lab-guide MCP

Already configured in .mcp.json — no action needed.

🤗

HuggingFace MCP

Connect via Claude Code MCP settings. Required for AI background image generation.

🔑

env.sh credentials

One-time setup — see instructions below. The skill sources .claude/env.sh (gitignored): prefers SPLUNK_API_TOKEN (Bearer) when set, else SPLUNK_USER/SPLUNK_PASS, plus HF_TOKEN for Hugging Face.

🐳

Lab stack running

docker compose up -d — Splunk must be up for query and deployment.

One-time setup

The skill calls the Splunk REST API directly and reads credentials from a shell file separate from Docker's .env:

Create .claude/env.sh

cp env.sh.example .claude/env.sh && chmod 600 .claude/env.sh

Then open .claude/env.sh and set SPLUNK_PASS to match SPLUNK_PASSWORD in .env, and HF_TOKEN to your Hugging Face token (hf_...). SPLUNK_HOST and SPLUNK_USER default to localhost / admin. Or run ./install.sh and accept the skill prompt: install **mints SPLUNK_API_TOKEN** against Splunk’s REST API when the stack is up, then writes all values into .claude/env.sh.

How to use

Invoke the skill in Claude Code by describing a dashboard or using the slash command. Claude will ask for any missing details.

Slash command (provides title)

/splunk-lab-dashboard-gen Buttercup Web Traffic

Natural language example

Generate a dashboard from index=buttercup, sourcetype=buttercup_sales, stats sum(revenue) as total by vendor | sort -total, title: Sales by Vendor

What Claude will do

Run your SPL via the splunk-lab-guide MCP
Synthesise a thematic background image prompt from the data
Generate the image via HuggingFace (gr1_z_image_turbo_generate)
Build a Dashboard Studio JSON with the image embedded as base64
Deploy to Splunk via REST API
Return a direct link: http://localhost:8000/en-US/app/search/<slug>

Output files are saved to ~/dev/claude-created-dashboards/<slug>/ — background PNG, dashboard.json, and the wrapped XML used for deployment.