README

A package for using Google Datastore as a database driver.

By using this package, you can use query builder and eloquent to access data from datastore.

Installation

This package requires PHP 8.3 as a minimum.

This branch targets Laravel 12.x. For compatibility with other Laravel versions, please see the appropriate branch.

You can install the package via composer:

composer require affordablemobiles/eloquent-datastore

If you are using Laravel Package Auto-Discovery, you don't need you to manually add the ServiceProvider.

Without auto-discovery:

If you don't use auto-discovery, add the below ServiceProvider to the $providers array in config/app.php file.

AffordableMobiles\EloquentDatastore\DatastoreServiceProvider::class,

Roadmap

• Read data using query builder.
• Read data using Eloquent model.
• Insert data using Eloquent model.
• Update data using Eloquent model.
• Delete data.
• Keys only queries.
• Auto-generated primary key.
• Read multiple pages of data with Datastore cursors.
• Batch update from Eloquent collection.
• cursorPaginate (Datastore cursors are used for chunk and lazy, but full pagination is not yet implemented)
• Ancestor key relations.
• Datastore Namespaces (Multi-Tenancy).

Usage

You need to add datastore connection in config/database.php file.

'connections' => [
    ...
    'datastore' => [
        'driver' => 'datastore',
        'transport' => env('DATASTORE_TRANSPORT', 'grpc'),
    ],
    ...
],

Access using Eloquent Model

You need to extend AffordableMobiles\EloquentDatastore\Eloquent\Model class instead of Laravel's default Eloquent model class.

Example-

<?php

namespace App\Models;

use AffordableMobiles\EloquentDatastore\Eloquent\Model;

class Project extends Model
{
    // Your works here
}

Access using Query Builder

Example-

DB::connection('datastore')
    ->table('projects')
    ->where('project_id', '>', 5)
    ->skip(3)
    ->take(5)
    ->get();

It will return a collection.

Understanding Primary Keys: __key__ vs. $primaryKey vs. id

To use this driver effectively, it's important to understand how it handles Datastore's keys.

1. The "Ground Truth": Datastore's __key__

In Google Datastore, the true primary key for every entity is the __key__ property. This is a complex Key object that contains the Kind, the scalar identifier (a string name or numeric ID), and any ancestor information. All final database operations (lookups, saves, deletes) must use this Key object.

2. The Eloquent Layer (Recommended)

The Eloquent Model (e.g., App\Models\User) provides a high-level, model-aware abstraction.

• The $primaryKey is an ALIAS: The $primaryKey property on your model (which defaults to 'id') is just a convenient alias for the scalar identifier (the string/int) part of the __key__.
• Customization: You can change this alias. If you set protected $primaryKey = 'uuid'; on your User model, the driver will automatically handle all the mapping.
• Two-Way Mapping:
  • Query (Out): User::where('uuid', 'my-uuid-string')->first() is automatically translated by the driver into a ...WHERE __key__ = Key('User', 'my-uuid-string') query.
  • Hydration (In): When the driver fetches data, it automatically populates the uuid attribute on your model with the key's identifier.

// With: protected $primaryKey = 'uuid';

// GOOD: This all works as you'd expect.
$user = User::where('uuid', 'my-uuid-string')->first();
$user = User::find('my-uuid-string');
$user = User::firstOrCreate(['uuid' => 'my-uuid-string']);
$uuid = $user->uuid;

3. The Base Query Builder (The "Escape Hatch")

If you bypass Eloquent and use the base Query Builder (e.g., DB::table('users') or User::query()->toBase()), you are in a model-agnostic layer. This layer does not know about your model's $primaryKey = 'uuid' alias.

• Fixed Convention: To provide a consistent way to query keys at this level, the driver's query processor synthesizes the key's scalar identifier into a single, hardcoded property named id.
• You MUST use id: When using the base Query Builder, you must use 'id' to query the key's identifier, regardless of what your Eloquent model's $primaryKey is set to.

// GOOD: This works, even if the model's $primaryKey is 'uuid'.
$user = DB::table('users')->where('id', 'my-uuid-string')->first();

// BAD: This will NOT work.
// The query builder will look for a *data property* named 'uuid',
// not the entity's key, because it is model-agnostic.
$user = DB::table('users')->where('uuid', 'my-uuid-string')->first();

Caching & Performance

This package automatically integrates with laravel-eloquent-query-cache to cache results. By default, caching is opt-in per model. To enable it, add the $cacheFor property to your model:

class User extends Model
{
    public $cacheFor = 3600; // Cache queries for 1 hour
    // ...
}

Smart Lookup Optimization

Google Datastore "Lookups" (retrieving by Key) are significantly faster, cheaper, and more consistent than "Queries" (scanning indexes).

This driver automatically optimizes standard Eloquent queries into Datastore Lookups whenever possible. If you filter by the model's primary key (e.g., id, uuid, or __key__), the driver converts the operation into a direct lookup() call internally. This ensures that these operations hit the cache and use the most efficient Datastore operation.

Cached & Optimized Operations:

The following operations are automatically converted to Lookups and will use the cache:

• Model::find($id)
• Model::findOrFail($id)
• Model::where('id', $id)->first()
• Model::where('uuid', $uuid)->get() (assuming 'uuid' is your primary key)
• $user->posts()->find($postId) (Relation lookups - correctly handles ancestor keys)

Uncached Operations:

Complex queries that require an index scan are not cached by default (to prevent cache explosion and stale data issues with ranges).

• Model::where('email', 'test@example.com')->first()
• Model::where('age', '>', 18)->get()
• Model::all()

Key Normalization

Datastore IDs can be either integers or strings. To ensure consistent caching and prevent misses, this driver automatically normalizes all IDs in cache keys and tags to strings. This means you do not need to worry about casting IDs (e.g. (int) $id) when interacting with the cache; the driver handles it for you.

Safety & Limitations

Partial Updates Protection

Datastore's upsert operation replaces the entire entity. This poses a risk if you fetch a partial model (e.g., using select() or keysOnly()) and then try to save() it, as you would accidentally delete all the attributes you didn't select.

To prevent data loss, this driver blocks saving of partial models.

// Fetch only the name
$user = User::select('name')->find(1);
$user->name = 'New Name';

// ERROR: Throws LogicException
// "Cannot save a partial model... saving it would overwrite the full entity..."
$user->save(); 

If you need to update a record, ensure you fetch the full entity first:

$user = User::find(1); // Fetches all columns
$user->name = 'New Name';
$user->save(); // OK

Tested Builder Functions

• connection
• table
• from
• namespace (Datastore Namespace: Multi Tenancy)
• select (for projection query)
• kind (same as table)
• where (Available: = , > , < , >= , <= )
• limit / take
• skip
• orderBy
• distinct
• get
• pluck
• exists
• count (Note: This performs a keys-only query and counts the results; it is not a high-performance aggregation. Avoid using count() without where clauses on very large kinds, as it may be slow and costly.)
• simplePaginate
• paginate (works same as simplePaginate)
• first
• delete
• insert
• _upsert (different and incompatible with default upsert)
• find / lookup
• chunk / chunkMap / each
• lazy / cursor

Contribution Guide

You can contribute by reporting bugs, fixing bugs, submitting and reviewing pull requests.

Go to issues section, and you can start working on an issue immediately.

License

The MIT License (MIT). Please see License File for more information.