GEMINI LABJP
NANOLITE — Nano Banana 2 Lite is here: Google's fastest and most cost-efficient Gemini Image model, made for running lightweight image generation cheaplyOMNIFLASH — Gemini Omni Flash is in public preview, a natively multimodal model that lets enterprises and developers build custom, dynamic video workflowsAGENTS — Managed Agents expand with background: true for async server-side runs and polling, remote MCP server integration, and refreshing credentials across interactionsMEMORY — The Memory Bank IngestEvents API is generally available, decoupling event ingestion from memory generation so you can stream content continuouslyTHROUGHPUT — Provisioned Throughput now lets you submit up to seven pending orders for the same model and regionDEPRECATE — Image generation models shut down on August 17, and the Grok 4.1 family on the Gemini Enterprise Agent Platform on August 20NANOLITE — Nano Banana 2 Lite is here: Google's fastest and most cost-efficient Gemini Image model, made for running lightweight image generation cheaplyOMNIFLASH — Gemini Omni Flash is in public preview, a natively multimodal model that lets enterprises and developers build custom, dynamic video workflowsAGENTS — Managed Agents expand with background: true for async server-side runs and polling, remote MCP server integration, and refreshing credentials across interactionsMEMORY — The Memory Bank IngestEvents API is generally available, decoupling event ingestion from memory generation so you can stream content continuouslyTHROUGHPUT — Provisioned Throughput now lets you submit up to seven pending orders for the same model and regionDEPRECATE — Image generation models shut down on August 17, and the Grok 4.1 family on the Gemini Enterprise Agent Platform on August 20
Articles/API / SDK
API / SDK/2026-05-03Advanced

A Personal Secretary AI on Gemini API and Google Workspace — Auth, Orchestration, and Approval Gates

Designing a personal secretary AI across Gmail and Google Calendar with the Gemini API: OAuth2 scope design, Function Calling orchestration, approval gates for write tools, and ninety days of measured production cost, with full Python code.

gemini-api277google-workspace7function-calling20gmail5google-calendar2agent10python104

Premium Article

"I just want an AI that reads my Gmail and tells me what needs attention today" — you'd think that would be a weekend project. And then you spend three days fighting OAuth2 scopes, get confused by quota limits, and abandon the whole thing when the costs start climbing.

I've been there. Google Workspace APIs are genuinely powerful, but the infrastructure surrounding them — auth flows, scope management, quota types — adds enough complexity that you can exhaust yourself before writing a single line of AI logic.

This guide builds a personal secretary AI that spans Gmail, Google Calendar, and Google Drive using Gemini API Function Calling. The implementation is designed for production use: not a demo, but something you can actually run every morning and trust.

Why Secretary AI Projects Usually Fail

Before touching any code, here are three patterns that kill these projects.

Anti-pattern 1: Sequential pipeline design

Gmail → AI processes → writes to Calendar. This seems clean until you try to add Drive or Docs. The integrations become tangled, and the whole thing collapses under its own weight. The right approach from day one is to let Gemini Function Calling orchestrate tool selection — the AI decides what to call and when.

Anti-pattern 2: Mixing OAuth2 and service accounts

"OAuth2 for personal use, service accounts for teams" sounds reasonable until you need to debug a permissions error at 11pm. For personal secretary AI, pick OAuth2 and stay consistent. Service accounts are for domain-wide delegation and CI/CD pipelines.

Anti-pattern 3: Passing everything to the model without filtering

Sending your entire inbox to Gemini sounds like a good idea until you get a bill for several dollars from a single morning run. Filtering before the API call — not after — is the design constraint that keeps this usable at personal scale.

Google Workspace API Authentication Design

Two authentication approaches matter here:

OAuth2 (Authorization Code Flow): For accessing your own data — your Gmail, your Calendar, your Drive. Tokens expire hourly, so refresh token management is required. This is what you want for personal use.

Service Accounts: For domain-wide access in headless environments (CI/CD, server-to-server). Not needed for a personal secretary AI.

Here's a production-ready authentication module with automatic token refresh:

# workspace_auth.py
import logging
from pathlib import Path
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
 
logger = logging.getLogger(__name__)
 
# Principle of least privilege: only request what you actually need
SCOPES = [
    "https://www.googleapis.com/auth/gmail.readonly",
    "https://www.googleapis.com/auth/gmail.compose",
    "https://www.googleapis.com/auth/calendar.readonly",
    "https://www.googleapis.com/auth/drive.readonly",
]
 
TOKEN_PATH = Path("~/.config/secretary-ai/token.json").expanduser()
CREDENTIALS_PATH = Path("~/.config/secretary-ai/credentials.json").expanduser()
 
def get_credentials() -> Credentials:
    """
    Get or refresh OAuth2 credentials.
    First run launches a browser auth flow.
    Subsequent runs load from token file and auto-refresh if expired.
    """
    creds = None
 
    if TOKEN_PATH.exists():
        creds = Credentials.from_authorized_user_file(str(TOKEN_PATH), SCOPES)
 
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            try:
                creds.refresh(Request())
                logger.info("Access token refreshed successfully")
            except Exception as e:
                logger.warning(f"Token refresh failed, re-auth required: {e}")
                creds = None
 
        if not creds:
            if not CREDENTIALS_PATH.exists():
                raise FileNotFoundError(
                    f"credentials.json not found at: {CREDENTIALS_PATH}\n"
                    "Download an OAuth2 client ID from Google Cloud Console"
                )
            flow = InstalledAppFlow.from_client_secrets_file(
                str(CREDENTIALS_PATH), SCOPES
            )
            creds = flow.run_local_server(port=0)
 
        TOKEN_PATH.parent.mkdir(parents=True, exist_ok=True)
        TOKEN_PATH.write_text(creds.to_json())
        logger.info("Token saved to disk")
 
    return creds
 
def build_service(service_name: str, version: str):
    """Build a Google API service client"""
    creds = get_credentials()
    return build(service_name, version, credentials=creds)

The scope separation between gmail.readonly and gmail.compose is intentional. If a bug causes an unintended write operation, readonly prevents the damage. Apply least-privilege thinking to API scopes the same way you would to IAM policies.

Thank you for reading this far.

Continue Reading

What follows includes implementation code, benchmarks, and practical content we hope you'll find useful. This site runs without ads — server and development costs are supported entirely by members like you. If it's been helpful, we'd be truly grateful for your support.

WHAT YOU'LL LEARN
Splitting OAuth2 scopes by read and write, and gating only the write tools behind human approval
What changed in output and cost when the model behind gemini-flash-latest was swapped
How to decide whether to keep your own orchestrator now that Managed Agents are in preview
Secure payment via Stripe · Cancel anytime

Unlock This Article

Get full access to the rest of this article. Buy once, read anytime. This site is ad-free — your support goes directly toward keeping it running.

or
Unlock all articles with Membership →
Share

Thank You for Reading

Gemini Lab is ad-free, supported entirely by members like you. We publish practical guides daily with implementation code, benchmarks, and production-ready patterns. If you've found it useful, we'd love to have you on board.

  • Copy-paste ready implementation code
  • New advanced guides published daily
  • $5/mo or $10 for lifetime access
View Membership →

Related Articles

API / SDK2026-03-20
Build an AI Data Analysis Agent with Gemini API — Combining Code Execution, Function Calling, and Structured Output
Learn how to build a production-ready AI data analysis agent in Python that combines Gemini API's Code Execution, Function Calling, and Structured Output to automatically analyze CSV/Excel data, generate visualizations, and produce structured reports.
API / SDK2026-05-13
Building an AdMob Revenue Anomaly Detector with Gemini API Function Calling
Learn how to build an automatic AdMob revenue anomaly detection system using Gemini API Function Calling — with real Python code, practical tips from 10+ years of indie app development, and Slack alerting integration.
API / SDK2026-05-01
Why 'contents must alternate between user and model' Won't Go Away in the Gemini API — and How to Fix It
A focused guide to the Gemini API's 'contents must alternate between user and model' error — what really triggers it, why role names from OpenAI break it, and how to fix Function Calling and system_instruction pitfalls with copy-pasteable code.
📚RECOMMENDED BOOKS
Build a Large Language Model (From Scratch)
Sebastian Raschka
LLM Dev
Prompt Engineering for LLMs
Berryman & Ziegler
Prompting
AI Engineering
Chip Huyen
AI Eng
* Contains affiliate links
See all →