Google Drive

Drive ingestion brings project documents into the Supabase document spine and makes SABLE the owner of the searchable copy. The full ingestion pipeline is built and correct, but it is currently paused by an explicit runtime decision, not merely undeployed — even a confirm:true real run short-circuits before touching the service account.

Why it's paused

drive-sync checks the configured project-folder count before doing any Drive work:

• If no project folders are configured, it returns "paused" with the reason that project folders are link-only and selected evidence is uploaded manually.
• If project folders are configured, it still returns "paused," because legacy Drive document ownership requires a reviewed reconciliation before the crawler is allowed to run again.

Either way, health is reported healthy (not degraded) while paused, and the connector is marked required:false — a paused-by-design connector is not treated as a failing one.

How it works (when unpaused)

Function

drive-sync

Secret

GDRIVE_SA_JSON

Optionally set GOOGLE_DRIVE_ROOT_FOLDER_ID to the approved master-root folder ID. If it is omitted, the crawler refreshes the complete file set visible to the least-privilege service account. Calendar's mirror (calendar-sync) reuses this same service-account JSON with a different OAuth scope.

Purpose

Confirmed Drive runs:

• authenticate as the service account via JWT-bearer (RS256), scoped drive.readonly;
• list all folders once to walk multi-parent ancestry, then classify each file into contracts_sow, discovery, data_mapping, deliverables, or reference by folder/name pattern;
• export Google-native Docs, Sheets, and Slides to readable formats, and extract text from PDFs (via unpdf, or Claude-vision OCR for scanned PDFs), .docx (via mammoth), .pptx, images (vision OCR), .zip, and .msg files;
• store the raw blob in the Supabase documents Storage bucket;
• write or upgrade the documents catalog row, keyed idempotently per file (gdrive:<fileId>), upgrading a previously text-only row in place rather than duplicating it; and
• extract/index text into knowledge_chunks, with spreadsheet_rows for structured workbook data.

A poison-file guard skips a page after MAX_PAGE_RETRIES (3) failed attempts, so one bad file can't wedge the whole crawl — added after a 2026-07 outage caused by exactly that.

Targeted Re-Pull

The function supports targeted re-pull through fileIds, which is useful when a specific file or project needs refresh without reprocessing the newest-file tail.

{ "confirm": true, "fileIds": ["drive-file-id"] }

Use folderId when testing a shared folder before a broader run.

{ "dryRun": true, "folderId": "drive-folder-id" }

Full Recursive Refresh

fullRefresh lists the complete visible corpus, filters every descendant of the configured root, and returns a Google nextPageToken. The Trigger.dev sable.drive.refresh task persists and follows that cursor in one-file Edge batches, scheduling continuations (via driveRefreshContinuation) up to forty batches before stopping, so a timeout or retry doesn't reduce the scan to only the newest files.

{ "confirm": true, "fullRefresh": true, "limit": 1 }

This task exists but has no cron schedule. It's chained/manual-invoke only — comments in trigger/connectors.ts explain that since Drive ingestion is paused by decision, a live 6-hourly schedule would wake up, scan zero folders, and needlessly hold the single write-queue slot for up to 30 minutes doing nothing. It remains fully invokable by hand once the pause is lifted.

Unchanged files whose stored source_modified_at matches Drive are skipped. Workbooks larger than 750 KB use the lower-memory OOXML text path in the Edge worker: the original file is still owned in Storage and its text remains searchable, while row/table ETL is reserved for smaller workbooks.

Cron cadence

Task	Schedule	Notes
sable.drive.refresh (+ driveRefreshContinuation)	No cron — manual/chained invocation only	Batches of ≤40 files, one Edge invocation per file

connector_checkpoints.required is explicitly false for Drive (same descoping migration as Toggl), reflecting the paused state.

Table mapping

Table	What's written
documents	Title, category, restricted, mime_type, byte_size, storage_path, checksum_sha256, source_url, external_id, source_modified_at, indexed_status, metadata
Supabase Storage (documents bucket)	The raw file blob
knowledge_chunks	Extracted/indexed text
spreadsheet_rows	Structured workbook data (workbooks ≤750 KB only)
connector_checkpoints	Health bookkeeping, required:false

projects, clients, and project_external_refs (provider gdrive_folder) are read to classify files and resolve the configured root.

Related but separate: document storage purge

sable.document.storage-purge and sable.document.storage-purge-sweep (in trigger/document-purge.ts) delete SABLE's own Storage blobs for tombstoned documents rows. This is unrelated to the Drive connector itself — the code explicitly never calls Google Drive; it only cleans up SABLE's own copy of previously ingested files. It runs on its own sweep schedule independent of Drive's pause state.

Access Rules

Document access is enforced by /api/doc:

• Valid Supabase session required.
• Restricted documents require admin/lead.
• Non-leads must be project members.

What can go wrong

• Nothing new appears from Drive no matter what's shared with the service account. This is expected while ingestion is paused by design — check whether the pause decision has actually been reversed before troubleshooting service-account permissions.
• A specific file or folder needs to refresh even during the pause. Use the targeted paths (fileIds, folderId with dryRun) — they still execute the underlying pipeline; only the scheduled/full-corpus path is gated by the pause check in the way described above. Confirm with an operator before running a real (non-dry-run) targeted pull, since the pause exists for a reason.
• A large workbook's rows/tables aren't searchable as structured data. Anything over 750 KB intentionally uses the OOXML text-extraction path instead of row/table ETL — the text is still indexed and searchable, just not as spreadsheet_rows.
• A fullRefresh run seems to stop partway through a large corpus. Check whether it's still progressing via chained continuations (up to 40 batches) rather than assuming failure — and note that this whole path currently has no cron trigger, so nothing resumes it automatically unless something explicitly invokes sable.drive.refresh again.

Where the code lives

• supabase/functions/drive-sync/index.ts — full ingestion pipeline and the pause check
• trigger/connectors.ts — sable.drive.refresh / driveRefreshContinuation (no cron)
• trigger/document-purge.ts — SABLE-side Storage cleanup for tombstoned documents (not a Drive operation)
• web/api/doc.js — document access enforcement
• supabase/migrations/20260903170000_descope_toggl_drive_from_report_readiness.sql — checkpoint scoping