Building OSINT Adapters with Elixir: A Practical Guide

Intelligence gathering at scale requires adapters that are reliable, rate-limited, and recoverable. In this guide, we walk through building a production-grade OSINT adapter using Elixir and the Prismatic Plugin Kit.

The Adapter Contract

Every OSINT adapter in Prismatic implements the PrismaticOsintCore.Adapter behaviour:

@callback search(query :: String.t(), opts :: keyword()) ::
{:ok, list(map())} | {:error, term()}


@callback metadata() :: %{
name: String.t(),
category: atom(),
rate_limit_rpm: pos_integer(),
confidence_tier: atom()
}

This contract guarantees that every adapter -- whether it queries the Czech ARES registry or Shodan's global index -- provides a consistent interface for the pipeline.

Scaffolding with Plugin Kit

The [Plugin Kit](/developers/plugins/) provides a scaffold command:

mix prismatic.gen.adapter --name my_custom_source --category global

This generates:

lib/adapters/my_custom_source.ex -- Adapter implementation

test/adapters/my_custom_source_test.exs -- Contract tests

config/my_custom_source.exs -- Configuration template

Implementing the Adapter

Here is a minimal adapter that queries a hypothetical threat intelligence API:

defmodule PrismaticOsintSources.Adapters.Global.ThreatFeed do
@behaviour PrismaticOsintCore.Adapter


@impl true
def metadata do
%{
name: "ThreatFeed Intelligence",
category: :global,
rate_limit_rpm: 30,
confidence_tier: :commercial_database
}
end


@impl true
def search(query, opts \\ []) do
with {:ok, response} <- fetch_data(query, opts),
{:ok, parsed} <- parse_response(response) do
{:ok, parsed}
end
end


defp fetch_data(query, _opts) do
# HTTP client call with timeout
{:ok, %{status: 200, body: %{"results" => []}}}
end


defp parse_response(%{status: 200, body: body}) do
results = Map.get(body, "results", [])
{:ok, Enum.map(results, &normalize/1)}
end


defp parse_response(%{status: status}) do
{:error, {:http_error, status}}
end


defp normalize(record) do
%{
name: Map.get(record, "name", "unknown"),
type: :domain,
identifiers: %{domain: Map.get(record, "domain")},
raw_data: record
}
end
end

Rate Limiting and Backpressure

Prismatic Hydra's pipeline automatically enforces rate limits declared in metadata/0. But for adapters making HTTP calls, you should also implement client-side throttling:

# The pipeline's circuit breaker will open after 5 consecutive failures
# and automatically retry after 30 seconds in half-open state

The circuit breaker states:

Closed (normal) -- requests flow freely

Open (stopped) -- 5 consecutive failures trigger a 30-second pause

Half-open (testing) -- one test request; success resumes, failure re-opens

Confidence Scoring

Every adapter result flows through the evidence chain, where confidence is assigned based on the source category:

CategoryConfidenceExample Sources

|----------|-----------|-----------------|

Official Registry0.95-1.00ARES, Justice Registry Commercial Database0.80-0.94Shodan, VirusTotal Web Scraping0.60-0.79Social media, web Derived0.40-0.59Computed data Unverified0.00-0.39Raw feeds

Testing Your Adapter

The Plugin Kit includes contract tests that verify your adapter conforms to the expected interface:

defmodule PrismaticOsintSources.Adapters.Global.ThreatFeedTest do
use PrismaticOsintCore.AdapterContractTest,
adapter_module: PrismaticOsintSources.Adapters.Global.ThreatFeed
end

This automatically runs 15+ contract assertions covering metadata validation, return type compliance, error handling, and idempotency.

Next Steps

[SDK Documentation](/developers/sdk/) -- Use the SDK to call your adapter from TypeScript or Python

[Interactive Labs](/lab/) -- Test your adapter in a sandboxed environment

[Academy](/academy/) -- Deeper courses on OSINT adapter patterns and OTP supervision

[OSINT Sources](/osint/) -- Browse all 121+ existing adapters for reference

---

The Prismatic Plugin Kit is MIT licensed. Build adapters, contribute back, and join the intelligence community.