README

Receive JSON data via REST API, store it in custom database tables, and render it anywhere on your site with PHP templates and shortcodes.

Introduction

Data Importer & Visualizer is a WordPress plugin for teams that need to publish structured, frequently-updated data on a website without building a custom integration from scratch. You define one or more data sources, push JSON to a per-source REST endpoint, and display the records using flexible PHP templates rendered through a simple shortcode.

Typical use cases:

• Publishing real-time or scheduled data feeds (events, open data, listings, status boards).
• Bridging an external system (CRM, ERP, open-data API) with a WordPress front-end.
• Displaying tabular or card-based data that is maintained outside WordPress.

Requirements

Requirement	Minimum
WordPress	5.8
PHP	7.4
MySQL / MariaDB	any version supported by your WordPress install

No third-party services or external API subscriptions are required. All data is stored in the local WordPress database.

What's Possible

Data sources

• Create any number of independent data sources, each with its own slug, one or more API keys, and import settings.
• Choose an import mode per source:
  • replace — deletes all existing records then inserts the new payload (default).
  • append — adds incoming records without touching existing ones.
  • upsert — inserts or updates records matched by one or more key columns.
• Manually import JSON directly from the admin for testing.

REST API import

• Push JSON to a dedicated endpoint per source: POST /wp-json/data-importer/v1/import/{source-slug}
• Authenticate with a per-source API key sent in the X-API-Key header.
• Restrict imports with optional per-key IP or CIDR allowlists.
• Accepts both a single JSON object {} and arrays of objects [{}, {}].
• Override the source's import mode for a single request with the mode query parameter (replace, append, or upsert).

Templates

• Build multiple PHP templates per source.
• Each template has a wrapper before, a per-row template, and a wrapper after section.
• Reference any field in the imported JSON directly as a PHP variable inside the template.
• Attach custom CSS/JS assets to a template (loaded only on pages that use the shortcode).
• Preview a template against live data directly inside the admin.

Shortcode

• Render any source/template combination anywhere in WordPress with [data_importer].
• Filter records from the shortcode without changing the template: where_key, where_op, where_value with support for dot-notation for nested fields.
• Control pagination with limit, offset, and order attributes.
• Fetch a single record by database ID with id.

Security & operations

• API key authentication with hashed secrets stored at rest.
• Optional per-key IP/CIDR allowlists for import endpoints.
• Per-source rate limiting with configurable count/window.
• Payload size and max-record guardrails.
• Dedicated Log tab per source with three sections: Import Log (latest 20 imports), Security Log (latest 50 blocked/rejected attempts), and Template Error Log (latest 20 PHP template errors).
• Safe mode — disables all PHP template execution in the frontend with one click while you fix a broken template.
• All security controls are tunable via WordPress filter hooks without patching the plugin.

Installation

1. Upload the plugin folder to /wp-content/plugins/.
2. Activate through Plugins → Installed Plugins in wp-admin.
3. Go to Settings → Data Importer.
4. Click Add new source, give it a name and choose an import mode.
5. Copy the Endpoint URL and the newly created API Key from the API tab immediately, since the plaintext key is only shown once.

Uninstallation

Deactivate and delete the plugin through Plugins → Installed Plugins in wp-admin. The uninstall routine will remove:

• All custom database tables created by the plugin.
• All plugin options (source configuration, import/security/template error logs, safe mode setting).
• All plugin transients (rate-limit counters, API key reveal tokens, manual import drafts).

Note: Content generated by PHP templates — such as files or external records written by template code — is not tracked by the plugin and will not be removed automatically.

Testing

Run the integration suite from the plugin directory with:

composer test

For readable per-test output, use:

composer test:verbose

To list the discovered tests without running them, use:

composer test:list

The PHPUnit bootstrap loads the local WordPress install, then exercises the plugin's real REST, manual-import, shortcode, safe-mode, and logging flows.

Getting Started

1. Create a data source

Go to Settings → Data Importer → Add new source. Fill in:

• Name — human-readable label.
• Slug — used in the REST endpoint URL (auto-generated from name).
• Import mode — replace, append, or upsert.

2. Push data

Send a POST request with a JSON body to the source endpoint.

curl -X POST https://example.com/wp-json/data-importer/v1/import/my-source \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_API_KEY" \
  -d '[
    { "id": 1, "title": "First item", "status": "active" },
    { "id": 2, "title": "Second item", "status": "draft" }
  ]'

A successful import returns:

{
  "inserted": 2,
  "updated": 0,
  "deleted": 0
}

3. Build a template

Open the source, go to the Template tab and create a new template. Example per-row markup:

<div class="item item--<?php echo esc_attr( $vars['status'] ?? '' ); ?>">
  <h2><?php echo esc_html( $vars['title'] ?? '' ); ?></h2>
</div>

Add a wrapper in Wrapper before / Wrapper after:

<ul class="item-list">

</ul>

4. Add the shortcode to a page

[data_importer source="my-source"]

With a specific template:

[data_importer source="my-source" template="my-template"]

Shortcode Reference

[data_importer
  source="<slug>"
  template="<template-name>"
  limit="<int>"
  offset="<int>"
  order="ASC|DESC"
  id="<int>"
  where_key="<field>"
  where_op="<operator>"
  where_value="<value>"
]

Attribute	Default	Description
source	(first source)	Slug of the data source. May be omitted when exactly one source exists.
template	(default template)	Name of the template to use.
limit	0 (all)	Maximum number of records to render.
offset	0	Number of records to skip.
order	ASC	Sort direction: ASC or DESC.
id	0	Fetch a single record by its database row ID.
where_key	—	Field name to filter on. Supports dot-notation for nested fields, e.g. address.city.
where_op	—	Comparison operator (see table below).
where_value	—	Value to compare against.

Supported where_op values

Operator	Behaviour
eq	Exact match
neq	Not equal
contains	String contains (case-insensitive)
starts_with	String starts with
ends_with	String ends with
gt	Greater than (numeric)
lt	Less than (numeric)

Shortcode examples

Show all active items:

[data_importer source="my-source" where_key="status" where_op="eq" where_value="active"]

Filter by a nested field:

[data_importer source="my-source" where_key="address.city" where_op="contains" where_value="Alingsas"]

Paginate — second page of 10:

[data_importer source="my-source" limit="10" offset="10"]

Single record:

[data_importer source="my-source" id="42"]

Template Variables

Inside every template section (wrapper before, per-row, wrapper after), the following PHP variables are available:

Variable	Type	Description
$vars	array	Associative array of all top-level fields for the current row.
$record	array	Alias for $vars — the full record as an associative array.

Access fields via $vars:

<div class="item item--<?php echo esc_attr( $vars['status'] ?? '' ); ?>">
  <h2><?php echo esc_html( $vars['title'] ?? '' ); ?></h2>
</div>

Conditional rendering:

<?php if ( ( $vars['status'] ?? '' ) === 'active' ) : ?>
  <span class="badge badge--active">Active</span>
<?php endif; ?>

Optional variable extraction

By default, record fields are only accessible via $vars. You can opt in to also having each top-level key extracted as its own variable ($title, $status, …) using the data_importer_template_extract_vars filter:

add_filter( 'data_importer_template_extract_vars', '__return_true' );

Security note: Only enable extraction if you control the shape of all imported records. A record with a key named code, vars, or context can cause subtle or unexpected behaviour in the template scope.

Note: Template code is executed as PHP. Only trusted administrators should be allowed to edit templates. See SECURITY.md.

Import Modes

Mode	Behaviour
replace	Deletes all existing records for the source, then inserts the new payload. Use for full dataset syncs.
append	Inserts incoming records alongside existing ones. Use for log-style or additive feeds.
upsert	Inserts new records or updates existing ones matched by one or more key columns you define. Use for syncing keyed datasets without losing data between pushes.

For upsert, set the Update key field in the source settings to a comma-separated list of JSON field names that uniquely identify a record (e.g. id or event_id,date).

Overriding the import mode per request

You can override the source's configured import mode for a single request by adding a mode query parameter to the endpoint URL:

# Full replace — e.g. run once a week
curl -X POST "https://example.com/wp-json/data-importer/v1/import/my-source?mode=replace" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_API_KEY" \
  -d '[{ "id": 1, "title": "Full dataset" }]'

# Upsert — e.g. run every weekday
curl -X POST "https://example.com/wp-json/data-importer/v1/import/my-source?mode=upsert" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_API_KEY" \
  -d '[{ "id": 1, "title": "Updated item" }]'

Valid values for mode are replace, append, and upsert. Any other value is rejected with a 400 response. When the parameter is omitted, the mode configured in the source settings is used.

Security

See SECURITY.md for a full OWASP Top 10 mapping and all filter hooks.

Runtime filter hooks

These WordPress filters let you tune security policy without modifying the plugin:

Filter	Type	Default	Description
data_importer_capability	string	manage_options	Capability required to access Data Importer admin and edit templates.
data_importer_enable_php_templates	bool	true	Set to false to disable all PHP template execution globally.
data_importer_safe_mode_active	bool	—	Programmatically force safe mode.
data_importer_max_payload_bytes	int	1048576	Maximum accepted request body size in bytes (1 MB).
data_importer_max_records_per_import	int	1000	Maximum records accepted in a single import.
data_importer_max_template_bytes	int	524288	Maximum template source size in bytes (512 KB).
data_importer_rate_limit_count	int	60	Maximum imports allowed per rate-limit window.
data_importer_rate_limit_window	int	60	Rate-limit window in seconds.
data_importer_trust_proxy	bool	false	Trust X-Forwarded-For headers for IP resolution (enable behind a known reverse proxy only).
data_importer_require_json_content_type	bool	true	Require Content-Type: application/json on import requests. Set to false to accept other content types.

Rate limiting is best-effort. The built-in limiter uses WordPress transients. Without a persistent object cache (Redis, Memcached) transients are stored in wp_options, which can become a write bottleneck under high traffic. The counter increments are also non-atomic, so concurrent requests may slip past the limit before it is enforced. For production deployments that require reliable rate limiting, enforce limits at the infrastructure layer — an Nginx limit_req directive, a WAF rule, or a reverse-proxy rate limiter will be more accurate and more performant. | data_importer_template_error_details | bool | false | Expose PHP error details in the frontend (enable in development only). | | data_importer_blocked_template_functions | string[] | (see below) | Array of PHP function names that are forbidden inside templates. Add or remove entries to tune the blocklist. | | data_importer_blocked_template_constructs | string[] | (see below) | Array of PHP language constructs that are forbidden inside templates (backticks, include/require family). | | data_importer_template_extract_vars | bool | false | Re-enable legacy extract() behaviour so record keys are available as $name etc. See security note in Template Variables. |

Default blocked functions

The following functions are blocked in templates by default. Use the data_importer_blocked_template_functions filter to add or remove entries:

exec, system, passthru, shell_exec, proc_open, proc_close, proc_get_status,
proc_nice, proc_terminate, popen, pcntl_exec,
eval, assert, preg_replace, create_function,
call_user_func, call_user_func_array,
array_map, array_filter, array_walk, array_walk_recursive, array_reduce,
usort, uasort, uksort,
register_shutdown_function, register_tick_function, spl_autoload_register,
set_error_handler, set_exception_handler,
unlink, rmdir, file_put_contents, fputs, fwrite,
dl

The callable-accepting functions (call_user_func, array_map, etc.) are blocked because they can invoke arbitrary code when given a user-influenced callable. If a template genuinely needs one of them, remove it via the filter after reviewing how the callable argument is derived.

Default blocked constructs

Language constructs (which are not function calls and cannot be reached by the function blocklist regex) are covered by a separate data_importer_blocked_template_constructs filter:

backticks, include, include_once, require, require_once

backticks refers to the backtick shell-exec operator (`command`). The include / require family is blocked because it can execute code from a file path that may be attacker-influenced.

Example — add file_get_contents to the blocklist:

add_filter( 'data_importer_blocked_template_functions', function( array $fns ): array {
    $fns[] = 'file_get_contents';
    return $fns;
} );

Example — remove a function from the blocklist:

add_filter( 'data_importer_blocked_template_functions', function( array $fns ): array {
    return array_diff( $fns, array( 'file_put_contents' ) );
} );

Example — lower the rate limit and disable PHP templates on a staging environment:

add_filter( 'data_importer_rate_limit_count', fn() => 10 );
add_filter( 'data_importer_enable_php_templates', '__return_false' );

Safe Mode

If a broken PHP template causes a fatal error on the front-end:

1. In the Data Importer admin, click Activate safe mode in the notice bar.
2. While safe mode is active, no PHP templates are executed on the front-end.
3. Open the Template tab, fix the template, and save.
4. Click Deactivate safe mode to restore normal rendering.

Safe mode can also be forced programmatically:

// In wp-config.php or a mu-plugin — bypasses the database entirely.
define( 'DATA_IMPORTER_SAFE_MODE', true );

Or via filter:

add_filter( 'data_importer_safe_mode_active', '__return_true' );

When either the constant or a filter is in effect, the Data Importer admin shows an override in effect notice and hides the Disable Safe Mode button, so administrators understand the UI toggle is currently inert.

Frequently Asked Questions

Can I push a single object instead of an array?

Yes. Both {} and [{}, {}] are accepted. A single object is treated as one record.

Can I restrict which servers are allowed to import?

Yes. In the source API tab, enter a comma-separated list of allowed IP addresses. Leave empty to allow any IP.

How do I retrieve the available field names for a source?

Authenticated admin users can call:

GET /wp-json/data-importer/v1/fields/{source-slug}

This returns the unique field names found across all stored records.

Can I test an import without an external system?

Yes. Go to the source's Manual tab, paste a JSON payload, and click Import. The result is shown in the Log tab under Import Log.

How do I use multiple templates for the same source?

Create additional templates in the Template tab. Reference them by name in the shortcode:

[data_importer source="my-source" template="compact-view"]

What happens to old data when I push a new payload in replace mode?

All existing records for that source are deleted before the new records are inserted. Use append or upsert mode if you need to preserve existing data.

Can I use different import modes without changing the source settings?

Yes. Add a mode query parameter to the endpoint URL to override the configured mode for that single request. This is useful when the same source needs different import strategies on different schedules — for example a full replace once a week and an upsert every weekday:

POST /wp-json/data-importer/v1/import/my-source?mode=replace
POST /wp-json/data-importer/v1/import/my-source?mode=upsert

The source settings are not changed; the override only applies to that request.