README

Laravel Sharding is a toolkit for distributing data across multiple databases while keeping a familiar Eloquent workflow. The package powers production applications and provides pluggable strategies so each table can select the most appropriate sharding approach. Shards can run on MySQL, PostgreSQL, SQL Server, or SQLite as long as the connections are available to Laravel.

Requirements

• PHP ^8.2
• Laravel 12.x or any framework using Illuminate 12 components
• MySQL, PostgreSQL, SQL Server, or SQLite for shard connections
• Redis (optional) when using the Redis-backed strategy

Installation

Require the package via Composer:

composer require allnetru/laravel-sharding

The service provider is auto-discovered. Publish the configuration and optional migrations with:

php artisan vendor:publish --tag=laravel-sharding-config
php artisan vendor:publish --tag=laravel-sharding-migrations

Run the migrations to create metadata tables used by the built-in strategies:

php artisan migrate

Configuration

1. Define shard connections through the DB_SHARDS environment variable. Each entry follows the format name:host:port:database and multiple shards are separated by semicolons.
2. When preparing to migrate or remove shards, list them in DB_SHARD_MIGRATIONS. New writes are skipped for shards in this list until you finish rebalancing.
3. Review config/sharding.php to map tables to strategies, configure shard groups, and choose ID generators. Every shard-aware model should use the provided Shardable trait.

A minimal example stitches these pieces together:

# .env
DB_SHARDS="shard-1:10.0.0.10:3306:app_shard_1;shard-2:10.0.0.11:3306:app_shard_2;shard-archive:10.0.0.12:3306:app_archive"
DB_SHARD_MIGRATIONS="shard-legacy;shard-archive"

// config/sharding.php
return [
    'default' => 'hash',

    'strategies' => [
        'hash' => Allnetru\Sharding\Strategies\HashStrategy::class,
        'redis' => Allnetru\Sharding\Strategies\RedisStrategy::class,
        'range' => Allnetru\Sharding\Strategies\RangeStrategy::class,
        'db_range' => Allnetru\Sharding\Strategies\DbRangeStrategy::class,
        'db_hash_range' => Allnetru\Sharding\Strategies\DbHashRangeStrategy::class,
    ],

    'id_generator' => [
        'default' => 'snowflake',
        'strategies' => [
            'snowflake' => Allnetru\Sharding\IdGenerators\SnowflakeStrategy::class,
            'sequence' => Allnetru\Sharding\IdGenerators\TableSequenceStrategy::class,
        ],
        'sequence_table' => 'shard_sequences',
        // 'meta_connection' => 'pgsql', // use any connection name configured in database.php
    ],

    'connections' => [
        'shard-1' => ['weight' => 2],
        'shard-2' => ['weight' => 1],
        // 'shard-archive' => ['weight' => 1, 'replica' => true],
    ],

    'replica_count' => 1,

    'tables' => [
        // 'users' => [
        //     'strategy' => 'redis',
        //     'redis_connection' => 'shards',
        //     'redis_prefix' => 'user_shard:',
        //     'group' => 'user_data',
        // ],

        'users' => [
            'strategy' => 'db_hash_range',
            'slot_size' => 250_000,
            'connections' => [
                'shard-1' => ['weight' => 2],
                'shard-2' => ['weight' => 1],
            ],
            'meta_connection' => 'mysql',
            'group' => 'user_data',
        ],

        'profiles' => [
            // inherits the shard selected for the `users` table
            'group' => 'user_data',
            // 'id_generator' => 'sequence',
        ],

        'orders' => [
            'strategy' => 'db_range',
            'connections' => [
                'shard-1' => ['weight' => 2],
                'shard-2' => ['weight' => 1],
            ],
            'range_size' => 50_000,
            'meta_connection' => 'mysql',
            // 'range_table' => 'order_ranges',
        ],

        // 'payments' => [
        //     'strategy' => 'range',
        //     'ranges' => [
        //         ['start' => 1, 'end' => 1_000_000, 'connection' => 'shard-1'],
        //         ['start' => 1_000_001, 'end' => null, 'connection' => 'shard-2'],
        //     ],
        // ],
    ],

    'groups' => [
        'user_data' => ['users', 'profiles', 'orders'],
        // 'billing' => ['payments', 'refunds'],
    ],
];

Update config/database.php to merge the generated shard connections with your base definitions. The examples below use MySQL, but you can swap in any of Laravel's supported drivers:

// config/database.php (excerpt)

use Allnetru\Sharding\Support\Config\Shards;

return [
    'default' => env('DB_CONNECTION', 'mysql'),

    'connections' => array_merge([
        'mysql' => [
            'driver' => 'mysql',
            'url' => env('DB_URL'),
            'host' => env('DB_HOST', '127.0.0.1'),
            'port' => env('DB_PORT', '3306'),
            'database' => env('DB_DATABASE', 'forge'),
            'username' => env('DB_USERNAME', 'forge'),
            'password' => env('DB_PASSWORD', ''),
            // ... keep the rest of your base connection definition
        ],

        // other non-sharded connections...
    ], Shards::databaseConnections(env('DB_SHARDS', ''))),

    // ...
];

Note Passing the DB_SHARDS string ensures shard definitions are available while configuration files are still being evaluated. In other contexts you may call Shards::databaseConnections() without arguments and it will read the DB_SHARDS environment variable directly.

A full walkthrough is available in docs/en/sharding.md.

Usage

Creating sharded tables

Create tables with an unsigned big integer primary key and the is_replica flag to track replicated rows:

Schema::create('items', function (Blueprint $table) {
    $table->unsignedBigInteger('id')->primary();
    $table->boolean('is_replica')->default(false);
    $table->timestamps();
});

Then register the table inside config/sharding.php, select a strategy (hash, redis, range, db_range, or db_hash_range), and list the shard connections the table can use.

ID generation

The default snowflake generator creates sortable 64-bit identifiers. You can switch the global default or override per table to use a database-backed sequence generator or any other configured strategy.

Grouping related tables

Group tables so records that belong together end up on the same shard:

'groups' => [
    'user_data' => ['users', 'organizations', 'billing', 'transactions'],
],

When models belong to a group they reuse the shard selected for the group's primary table (for example, users).

Working with data

Models using the Shardable trait behave like standard Eloquent models:

$user = User::find(15);

$partners = Organization::where('status', OrganizationStatus::partner)
    ->paginate(50);

Insertions also resolve the target shard automatically. If you omit the primary key the configured ID generator assigns one before the record is saved.

Running under Swoole

When the PHP process is executed inside a Swoole coroutine context (for example, Laravel Octane with the Swoole engine), shard fan-out queries are dispatched concurrently. The package detects the coroutine runtime automatically and uses channels to aggregate results without blocking on each individual shard. When a request is not already inside a coroutine, the dispatcher boots a Swoole\Coroutine::run() scheduler so the queries still run in parallel. No additional configuration is required.

Custom coroutine drivers

The default behaviour can be overridden from config/sharding.php. The coroutines section accepts any class or closure that returns an implementation of Allnetru\Sharding\Support\Coroutine\CoroutineDriver, allowing you to disable coroutines entirely or integrate with an alternative runtime:

'coroutines' => [
    'default' => env('SHARDING_COROUTINE_DRIVER', 'swoole'),
    'drivers' => [
        'swoole' => Allnetru\Sharding\Support\Coroutine\Drivers\SwooleCoroutineDriver::class,
        'sync' => Allnetru\Sharding\Support\Coroutine\Drivers\SyncCoroutineDriver::class,
        'amphp' => App\Sharding\AmpCoroutineDriver::class,
    ],
],

Point the default driver to sync (or set SHARDING_COROUTINE_DRIVER=sync) to keep fan-out queries synchronous. Custom drivers may be resolved through the Laravel container, so you can bind them as singletons or expose factory closures for more advanced scenarios.

Console tooling

Use the bundled Artisan commands to inspect and maintain shards:

• php artisan shards:distribute {model} – backfill existing tables into shards in chunks once strategies are configured.
• php artisan shards:rebalance {table} – migrate rows between shards with optional --from, --to, --start, and --end filters.
• php artisan shards:migrate – run shard-specific migrations across every configured connection.

Testing

Clone the repository and install dependencies before running the test suite:

composer install
composer test

Contributing

Please review the CONTRIBUTING.md guide for details about our workflow, coding standards, and security policy.

Security

If you discover a security vulnerability, please follow the disclosure process described in CONTRIBUTING.md.

License

Laravel Sharding is open-sourced software licensed under the MIT license.