Settings reference¶

Bases: StrEnum

What a bearer token may do; write implies read.

READ = 'read'

WRITE = 'write'

Bases: BaseModel

One named bearer token with its scopes.

Configure several as JSON, e.g.::

REFINDERY_AUTH_TOKENS='[{"name":"agent","token":"...","scopes":["read"]}]'

name = Field(min_length=1)

token

scopes = (Scope.READ, Scope.WRITE)

Bases: StrEnum

Which vector store adapter to use.

QDRANT = 'qdrant'

LANCEDB = 'lancedb'

Bases: StrEnum

Which reranker adapter to use.

NONE = 'none'

API = 'api'

LOCAL = 'local'

Bases: StrEnum

Where OpenTelemetry spans are exported.

OFF = 'off'

CONSOLE = 'console'

OTLP = 'otlp'

Bases: BaseModel

Qdrant connection settings.

url = 'http://127.0.0.1:6333'

api_key = None

collection = 'refindery_chunks'

Bases: BaseModel

LanceDB storage settings.

path = Path('data/lancedb')

Bases: BaseModel

SQLite metadata store settings.

path = Path('data/refindery.db')

Bases: BaseModel

DuckDB observability sink settings.

path = Path('data/observability.duckdb')

Bases: BaseModel

Huey queue storage settings.

path = Path('data/huey.db')

Bases: BaseModel

Active embedding model.

dim/max_input_tokens are authoritative here because not every provider SDK exposes them.

provider = 'voyage'

model = 'voyage-3.5'

dim = Field(default=1024, ge=1)

max_input_tokens = Field(default=32000, ge=1)

Bases: BaseModel

Reranker selection.

kind = RerankerKind.API

provider = 'cohere'

model = 'rerank-v3.5'

Bases: BaseModel

Canonical chunking parameters (model-independent).

target_tokens = Field(default=448, ge=1)

overlap_tokens = Field(default=64, ge=0)

hard_max_tokens = Field(default=512, ge=1)

Bases: BaseModel

PDF text-extraction tuning (pypdf extractor).

Running headers/footers and page-number lines otherwise repeat into every chunk; strip_repeated_lines detects lines recurring in the top/bottom repeated_line_scan lines of at least repeated_line_ratio of the pages (only when a document has min_pages_for_stripping pages).

strip_repeated_lines = True

repeated_line_ratio = Field(default=0.6, gt=0.0, le=1.0)

repeated_line_scan = Field(default=2, ge=1)

min_pages_for_stripping = Field(default=3, ge=1)

Bases: BaseModel

URL canonicalization overrides.

tracking_params = DEFAULT_TRACKING_PARAMS

Bases: BaseModel

Indexing pipeline knobs.

page_vector_pooling = PoolingStrategy.MEAN

Bases: BaseModel

Outbound fetch limits for the fetch_and_index path.

The youtube_* knobs control transcript ingestion for YouTube video URLs (requires the youtube extra): preferred caption languages, whether auto-generated captions are acceptable, and whether captionless videos fall back to local Whisper transcription (transcribe / transcribe-mlx extra + ffmpeg).

The audio_* knobs control transcription of audio URLs — podcast enclosures discovered by watches and audio files posted directly. Audio downloads stream to a temp file bounded by audio_max_bytes and audio_timeout_s instead of the in-memory max_bytes cap. The Whisper model is shared with youtube_whisper_model; transcription requires the transcribe or transcribe-mlx extra + ffmpeg.

podcast_transcripts prefers Podcasting 2.0 published transcripts and chapter metadata when a podcast watch discovers them. It requires the podcast extra and falls back to the audio/Whisper path above.

timeout_s = Field(default=10.0, gt=0)

max_bytes = Field(default=10000000, ge=1)

youtube_captions = True

youtube_caption_langs = ('en', 'en-US', 'en-GB')

youtube_allow_auto_captions = True

youtube_transcribe_fallback = True

youtube_whisper_model = 'small'

youtube_timeout_s = Field(default=60.0, gt=0)

audio_transcripts = True

audio_max_bytes = Field(default=250000000, ge=1)

audio_timeout_s = Field(default=300.0, gt=0)

podcast_transcripts = True

Bases: BaseModel

Watch-mode polling configuration.

default_interval_hours applies when a watch is created without an explicit interval. poll_tick_enabled gates the minute-level scheduler periodic (tests drive WatchService.tick directly instead).

default_interval_hours = Field(
    default=24, ge=1, le=MAX_WATCH_INTERVAL_HOURS
)

poll_tick_enabled = True

max_due_per_tick = Field(default=20, ge=1)

max_items_per_poll = Field(default=200, ge=1)

youtube_max_entries = Field(default=100, ge=1)

Bases: BaseModel

Durable job execution parameters.

handler_timeout_s bounds one handler execution (cooperative cancellation); None derives it from lease_minutes. It exists mainly so tests can use sub-second timeouts.

max_attempts = Field(default=5, ge=1)

lease_minutes = Field(default=15, ge=1)

backoff_base_s = Field(default=2.0, gt=0)

handler_timeout_s = Field(default=None, gt=0)

Bases: BaseModel

Circuit breaker, in-call retry, and provider timeout knobs.

breaker_failure_threshold = Field(default=5, ge=1)

breaker_cooldown_s = Field(default=30.0, gt=0)

retry_attempts = Field(default=3, ge=1)

retry_base_delay_s = Field(default=0.25, gt=0)

retry_max_delay_s = Field(default=2.0, gt=0)

embed_timeout_s = Field(default=60.0, gt=0)

rerank_timeout_s = Field(default=15.0, gt=0)

Bases: BaseModel

MCP server surface configuration.

enable_mutating_tools = False

Bases: BaseModel

Server-sent events (GET /v1/events) configuration.

heartbeat_s = Field(default=15.0, gt=0)

queue_size = Field(default=256, ge=1)

max_subscribers = Field(default=16, ge=1)

Bases: BaseModel

Entity extraction and canonicalization configuration.

extractor_chain = ('gliner', 'spacy', 'gazetteer')

gazetteer_patterns_path = None

surface_embedder = 'static'

cosine_threshold = Field(default=0.85, ge=0.0, le=1.0)

edit_distance_threshold = Field(
    default=0.15, ge=0.0, le=1.0
)

Bases: BaseModel

Search ranking defaults.

recency_half_life_days = Field(default=None, gt=0)

Bases: BaseModel

Clustering configuration.

algorithm = 'hdbscan'

reducer = 'umap'

leiden_resolution = Field(default=1.0, gt=0)

cron = None

min_pages = Field(default=50, ge=1)

min_new_pages = Field(default=20, ge=1)

idle_default_minutes = Field(default=15, ge=1)

Bases: BaseModel

Optional OpenAI-compatible endpoint (labels, LLM entity extraction).

base_url = None

api_key = None

model = 'llama3.2'

timeout_s = Field(default=30.0, gt=0)

Bases: BaseModel

Tracing/logging/metrics configuration.

traces = TraceExporter.OFF

otlp_endpoint = None

json_logs = True

metrics_snapshot_interval_s = Field(default=15.0, gt=0)

Bases: BaseSettings

Root settings object; the composition root builds everything from this.

model_config = SettingsConfigDict(
    env_prefix="REFINDERY_",
    env_nested_delimiter="__",
    env_file=".env",
    env_file_encoding="utf-8",
    extra="ignore",
)

auth_token = None

auth_tokens = ()

bind_host = '127.0.0.1'

bind_port = 8000

vector_store = VectorStoreKind.QDRANT

qdrant = QdrantSettings()

lancedb = LanceDbSettings()

sqlite = SqliteSettings()

duckdb = DuckDbSettings()

huey = HueySettings()

embedder = EmbedderSettings()

reranker = RerankerSettings()

chunking = ChunkingSettings()

pdf = PdfSettings()

canonicalize = CanonicalizeSettings()

indexing = IndexingSettings()

fetch = FetchSettings()

watch = WatchSettings()

jobs = JobsSettings()

resilience = ResilienceSettings()

mcp = McpSettings()

events = EventsSettings()

entity = EntitySettings()

search = SearchSettings()

cluster = ClusterSettings()

llm = LlmSettings()

observability = ObservabilitySettings()

resolved_tokens()