User Guide

This guide provides detailed information on configuring and using apiout.

Configuration Files

API Configuration

The API configuration file defines which APIs to call and their parameters.

Basic Structure

[[apis]]
name = "api_name"              # Unique identifier for this API
module = "module_name"         # Python module to import
client_class = "Client"        # Class name (default: "Client")
method = "method_name"         # Method to call on the client
url = "https://api.url"        # API endpoint URL
serializer = "serializer_ref"  # Reference to serializer (optional)

[apis.params]                  # Parameters to pass to the method
key = "value"

Required Fields

name: Unique identifier for the API
module: Python module containing the client class
method: Method name to call on the client instance
url: API endpoint URL

Optional Fields

client_class: Name of the client class (default: “Client”)
serializer: Reference to a serializer configuration (string) or inline serializer (dict)
params: Dictionary of parameters to pass to the API method
method_params: Default parameters for the method, can be overridden at runtime
init_params: Parameters to pass to the client class constructor
client: Reference to a client configuration from the [clients] section

Multiple APIs

You can define multiple APIs in one file:

[[apis]]
name = "api1"
module = "module1"
method = "method1"
url = "https://api1.example.com"

[apis.params]
key = "value"

[[apis]]
name = "api2"
module = "module2"
method = "method2"
url = "https://api2.example.com"

[apis.params]
key = "value"

Serializer Configuration

Serializers define how to transform API response objects into structured data.

Basic Structure

[serializers.name]
[serializers.name.fields]
output_field = "InputAttribute"

Field Mapping Types

Simple Attribute Access

[serializers.example.fields]
latitude = "Latitude"      # result["latitude"] = obj.Latitude
longitude = "Longitude"    # result["longitude"] = obj.Longitude

Method Calls

[serializers.example.fields.current]
method = "Current"         # Call obj.Current() method
[serializers.example.fields.current.fields]
time = "Time"             # result["current"]["time"] = obj.Current().Time

Nested Objects

[serializers.example.fields.data]
method = "GetData"
[serializers.example.fields.data.fields]
value = "Value"
status = "Status"

Iteration

Iterate over collections with indexed access:

[serializers.example.fields.variables]
iterate = {
  count = "VariablesLength",    # Method returning item count
  item = "Variables",            # Method taking index parameter
  fields = { value = "Value" }  # Fields to extract from each item
}

Iteration with Method

[serializers.example.fields.data]
method = "GetContainer"
[serializers.example.fields.data.fields.variables]
iterate = {
  count = "Length",
  item = "GetItem",
  fields = { name = "Name", value = "Value" }
}

Serializer Referencing

Inline Serializers

Define serializers in the same file as APIs:

[serializers.myserializer]
[serializers.myserializer.fields]
field1 = "Attribute1"

[[apis]]
name = "myapi"
serializer = "myserializer"
# ... rest of config

Separate Serializers File

Keep serializers in a separate file for better organization:

serializers.toml:

[serializers.myserializer]
[serializers.myserializer.fields]
field1 = "Attribute1"

apis.toml:

[[apis]]
name = "myapi"
serializer = "myserializer"
# ... rest of config

Run with both files:

apiout run --config apis.toml --serializers serializers.toml

Priority Order

When using both inline and separate serializer files:

Serializers from -s file are loaded first
Inline serializers from config file are merged in
Inline serializers override external ones with the same name

No Serializer

If no serializer is specified, apiout uses default serialization:

Primitive types (str, int, float, bool, None) are returned as-is
Lists and tuples are recursively serialized
Dictionaries are recursively serialized
Objects are converted to dictionaries (public attributes only)
NumPy arrays are converted to lists

Advanced Features

Reusable Client Configurations

When multiple APIs use the same client with identical initialization parameters, you can define the client once in a [clients] section and reference it from multiple APIs. This eliminates repetition and makes configurations easier to maintain.

Client references automatically create shared instances - all APIs referencing the same client will share one instance.

Configuration

[clients.mempool]
module = "pymempool"
client_class = "MempoolAPI"
init_params = {api_base_url = "https://mempool.space/api/"}

[[apis]]
name = "block_tip_hash"
client = "mempool"
method = "get_block_tip_hash"

[[apis]]
name = "block_tip_height"
client = "mempool"
method = "get_block_tip_height"

[[apis]]
name = "recommended_fees"
client = "mempool"
method = "get_recommended_fees"

With Init Method

For clients that require initialization before use, specify init_method in the client definition:

[clients.btc_price]
module = "btcpriceticker"
client_class = "Price"
init_params = {fiat = "EUR", days_ago = 1, service = "coinpaprika"}
init_method = "update_service"

[[apis]]
name = "btc_price_usd"
client = "btc_price"
method = "get_usd_price"

[[apis]]
name = "btc_price_eur"
client = "btc_price"
method = "get_fiat_price"

The init_method is called once when the client is first created. All subsequent APIs reuse the same instance without re-initialization.

How It Works

Define a client in the [clients.<name>] section with:
- module: Python module containing the client class
- client_class: Name of the client class
- init_params: Parameters to pass to the constructor (optional)
- init_method: Method to call once after instantiation (optional)
Reference the client from APIs using client = "<name>"
The client is instantiated once when first referenced
If init_method is specified, it’s called after instantiation
All APIs referencing the same client share this instance
Only the method and params need to be specified for each API

Benefits

Eliminate Repetition: Define client configuration once, reference it multiple times
Easier Maintenance: Update client settings in one place
Cleaner Configs: Focus on what each API does, not how to initialize the client
Automatic Sharing: All APIs using the same client reference share one instance
Performance: Avoid redundant initialization and data fetching

Compatibility

The client reference can be used alongside traditional configuration:

If client is specified, module, client_class, and init_params are taken from the client definition
Inline init_params can override or extend client-level init_params
If no client is specified, traditional inline configuration is used

Multiple Configuration Files

Client definitions are merged from multiple configuration files:

apiout run --config base.toml -c apis.toml

If the same client name appears in multiple files, later files override earlier ones.

Config Directory Support

For easier configuration management, you can store configs in ~/.config/apiout/ and reference them by name:

# Load config from ~/.config/apiout/mempool.toml
apiout run --config mempool --json

# Mix config names and file paths
apiout run --config mempool --config ./local.toml --json

The tool follows XDG Base Directory specification: - Uses $XDG_CONFIG_HOME/apiout/ if set - Falls back to ~/.config/apiout/ otherwise

Multiple Configuration Files

You can use multiple configuration and serializer files with the -c and -s options:

apiout run --config base.toml --config apis.toml --config more_apis.toml --serializers serializers1.toml --serializers serializers2.toml

Merging Behavior

APIs: Appended in order (base → apis → more_apis)
Post-processors: Appended in order
Serializers: Merged (later files override earlier ones)

This allows you to:

Share common configurations across projects
Override serializers for different environments
Organize large configurations into multiple files

Post-Processors

Post-processors allow you to combine and transform data from multiple API calls using any Python class.

Configuration Format

[[post_processors]]
name = "processor_name"          # Required: unique identifier
module = "module_name"           # Required: Python module
class = "ClassName"              # Required: class to instantiate
method = "method_name"           # Optional: method to call
inputs = ["api1", "api2"]        # Required: list of API names
serializer = "serializer_name"   # Optional: serializer reference

Execution Order

All [[apis]] are fetched first and stored in a results dictionary
Post-processors execute in the order they appear in the configuration
Each post-processor receives the specified API results as arguments
The class is instantiated with the inputs (or a method is called if specified)
The result is optionally serialized
The result is added to the output under the post-processor’s name
Later post-processors can reference outputs from earlier ones

Example

[[apis]]
name = "recommended_fees"
module = "pymempool"
client_class = "MempoolAPI"
method = "get_recommended_fees"
url = "https://mempool.space/api/"

[[apis]]
name = "mempool_blocks_fee"
module = "pymempool"
client_class = "MempoolAPI"
method = "get_mempool_blocks_fee"
url = "https://mempool.space/api/"

[[post_processors]]
name = "fee_analysis"
module = "pymempool"
class = "RecommendedFees"
inputs = ["recommended_fees", "mempool_blocks_fee"]
serializer = "fee_analysis_serializer"

Benefits

Declarative Configuration: Define data transformation in TOML instead of code
Reusability: Post-processors can be reused across different configurations
Modularity: Separate data fetching from data processing
Composability: Chain multiple post-processors together
Integration: Use any existing Python class from installed packages

NumPy Array Handling

NumPy arrays are automatically converted to Python lists:

[serializers.example.fields.data]
values = "ValuesAsNumpy"  # Returns numpy array, auto-converted to list

Generator Tool

The generator tool introspects API responses and generates serializer configurations:

apiout generate \
  --module openmeteo_requests \
  --method weather_api \
  --url "https://api.open-meteo.com/v1/forecast" \
  --params '{"latitude": 52.52, "longitude": 13.41, "current": ["temperature_2m"]}' \
  --name openmeteo > serializers.toml

This outputs a TOML serializer configuration that you can refine manually.

JSON Input

apiout supports two ways to use JSON with stdin:

1. JSON Parameters via stdin

Pass method parameters as JSON via stdin (works with -c):

echo '{"time_period": "24h"}' | apiout run --config config.toml

This is equivalent to:

apiout run --config config.toml -p time_period=24h

When both stdin and -p are provided, stdin parameters override -p parameters.

How it works:

Method parameters from stdin (or -p flags) are merged with method_params defaults
Runtime parameters have highest priority, followed by method_params defaults, then environment variables
Parameters are also available for variable substitution in URLs, params, and headers using ${param_name} syntax
When init_params are overridden, a new client instance is created with the updated parameters

Example: Override method_params values

Configuration file (api.toml):

[[apis]]
name = "docs_api"
module = "requests"
client_class = "Session"
method = "get"
url = "https://api.example.com/docs"
method_params = {topic = "default_topic", tokens = 5000}

Override with stdin:

echo '{"topic": "routing", "tokens": 100}' | apiout run --config api.toml

This will send topic=routing and tokens=100 instead of the defaults.

Example: Override init_params

Configuration file (btcpriceticker.toml):

[clients.btc_price]
module = "btcpriceticker"
client_class = "Price"
init_params = {fiat = "EUR", days_ago = 1, service = "coinpaprika"}

[[apis]]
name = "btc_price"
client = "btc_price"
method = "get_fiat_price"

Override client initialization parameters:

# Change fiat currency to USD
apiout run --config btcpriceticker.toml -p fiat=USD

# Change service to coingecko and lookback period to 7 days
echo '{"service": "coingecko", "days_ago": 7}' | apiout run --config btcpriceticker.toml

When init_params are overridden, apiout creates a new client instance with the updated parameters. This allows runtime customization without modifying configuration files.

Important: Interaction between method_params and init_params

When a parameter name appears in both init_params and method_params, the behavior is:

The parameter in init_params is NOT overridden by method params
The user-provided value is passed as a method argument instead
This allows the client to maintain its initialization state while the method receives different values

Example:

[clients.example]
module = "mymodule"
client_class = "Client"
init_params = {fiat = "EUR"}

[[apis]]
client = "example"
method = "get_data"
method_params = {fiat = "USD"}

Running with apiout run --config config.toml -p fiat=GBP:

Client is initialized with fiat="EUR" (from init_params)
Method is called as get_data("GBP") (from runtime params, overriding method_params default)

If you want runtime params to override init_params, do not include that parameter in method_params.

Benefits:

Cleaner syntax for complex parameter values
Easy integration with JSON-based tools and scripts
Support for nested objects and arrays
No need to escape special characters
Override default parameter values without modifying config files

Variable Substitution

apiout supports universal variable substitution using ${param_name} syntax in URLs, parameters, and headers. Variables are resolved from multiple sources with the following priority:

Runtime parameters (highest priority) - from -p flags or JSON stdin
method_params defaults - from configuration
Environment variables (fallback) - from system environment

URL Substitution

[[apis]]
name = "api_docs"
module = "requests"
client_class = "Session"
method = "get"
url = "https://api.example.com/v1/${library_id}?type=json&topic=${topic}&tokens=${tokens}"
method_params = {library_id = "", topic = "default", tokens = 1000}

Running with:

apiout run -c config.toml -p library_id=/vercel/next.js -p topic=hooks -p tokens=3000

Results in URL: https://api.example.com/v1/vercel/next.js?type=json&topic=hooks&tokens=3000

Parameter Substitution

[[apis]]
name = "weather_api"
module = "openmeteo_requests"
client_class = "Client"
method = "weather_api"
url = "https://api.open-meteo.com/v1/forecast"
method_params = {latitude = 52.52, longitude = 13.41}

[apis.params]
latitude = "${latitude}"
longitude = "${longitude}"
current = ["temperature_2m"]

Header Substitution

[[apis]]
name = "authenticated_api"
module = "requests"
client_class = "Session"
method = "get"
url = "https://api.example.com/data"
method_params = {api_key = ""}

[apis.headers]
Authorization = "Bearer ${api_key}"
Content-Type = "application/json"

Environment Variable Fallback

If a parameter is not provided via runtime or method_params, apiout falls back to environment variables:

export API_KEY="your-api-key-here"
export DEFAULT_TOPIC="general"

[[apis]]
name = "env_api"
module = "requests"
client_class = "Session"
method = "get"
url = "https://api.example.com/${DEFAULT_TOPIC}"
method_params = {api_key = "${API_KEY}"}

Advanced Usage

Variable substitution works with all string fields and supports:

Nested substitution: ${base_url}/${endpoint}
Default values: If no environment variable exists, the substitution fails gracefully
Multiple occurrences: Use the same variable multiple times in different fields

This feature makes configurations more dynamic and reusable across different environments and use cases.

2. Full JSON Configuration via stdin

Provide the entire configuration as JSON (without -c):

apiout run --json < config.json

This is useful for:

Converting TOML to JSON with tools like taplo
Dynamically generating configurations
Integration with JSON-based workflows

Example: Convert TOML to JSON

taplo get -f apis.toml -o json | apiout run --json

Example: Inline JSON

echo '{"apis": [{"name": "test", "module": "requests", "method": "get", "url": "https://api.example.com"}]}' | apiout run --json

The JSON structure matches the TOML format exactly:

{
  "apis": [
    {
      "name": "api_name",
      "module": "module_name",
      "client_class": "Client",
      "method": "method_name",
      "url": "https://api.url",
      "serializer": "serializer_ref",
      "params": {
        "key": "value"
      }
    }
  ],
  "serializers": {
    "serializer_name": {
      "fields": {
        "output_field": "InputAttribute"
      }
    }
  }
}

Output Formats

JSON Output

apiout run --config config.toml --json

Outputs valid JSON for piping to other tools:

{
  "api_name": [
    {
      "field1": "value1",
      "field2": "value2"
    }
  ]
}

Pretty Print (Default)

apiout run --config config.toml

Uses Rich console formatting for readable output.

Error Handling

apiout provides clear error messages for common issues:

Missing configuration file
Invalid TOML syntax
Missing required fields
Module import errors
API call failures

All errors are displayed with context to help diagnose issues quickly.

Migration Guide

This section covers breaking changes and how to migrate existing configurations.

Breaking Changes in Recent Versions

user_inputs/user_defaults → method_params

The old user_inputs (list) and user_defaults (dict) fields have been consolidated into a single method_params (dict) field.

Old Configuration:

[[apis]]
name = "example_api"
module = "mymodule"
method = "get_data"
user_inputs = ["param1", "param2"]
user_defaults = {param1 = "default1", param2 = "default2"}

New Configuration:

[[apis]]
name = "example_api"
module = "mymodule"
method = "get_data"
method_params = {param1 = "default1", param2 = "default2"}

CLI Changes

The CLI options have changed from --user-inputs/--user-defaults to --method-params:

# Old way
apiout run --config config.toml --user-inputs param1=value1 --user-defaults param2=value2

# New way
apiout run --config config.toml --param param1=value1 --param param2=value2

New Variable Substitution Feature

Take advantage of the new ${param_name} substitution syntax:

[[apis]]
name = "dynamic_api"
module = "requests"
method = "get"
url = "https://api.example.com/v1/${endpoint}/${id}"
method_params = {endpoint = "users", id = "123"}

[apis.params]
limit = "${limit}"
format = "${format}"

Run with runtime overrides:

apiout run -c config.toml -p endpoint=posts -p id=456 -p limit=10 -p format=json

Benefits of Migration

Simplified Configuration: Single method_params field instead of two separate fields
Variable Substitution: Dynamic URL and parameter building with ${} syntax
Better Priority Handling: Clear precedence: runtime > method_params > environment
Unified CLI: Single -p flag for all method parameters

Best Practices

Separate Concerns: Keep API configs and serializers in separate files for large projects
Use Descriptive Names: Give APIs and serializers clear, descriptive names
Start Without Serializers: Test API calls with default serialization first
Use Generator: Generate initial serializer configs, then refine manually
Version Control: Store config files in version control
Document Custom Serializers: Add comments to explain complex field mappings
Leverage Variable Substitution: Use ${} syntax for dynamic configurations
Environment Variables: Use environment variables for secrets and defaults