Setup and context
Gemini 2.5 Pro's multimodal capabilities enable document processing far beyond simple text extraction. Layout recognition, structured table extraction, cross-document analysis, and intelligent document understanding are now within reach for enterprise-grade applications.
Gemini's Document Understanding Capabilities
Advantages Over Traditional OCR
Traditional OCR focuses purely on character recognition, while Gemini understands image content contextually:
- Layout Understanding: Recognition of sections, columns, indentation, and visual hierarchy
- Table Structure Recognition: Accurate parsing of complex matrix layouts with merged cells
- Chart Interpretation: Extracting numeric values and trends from visual data
- Contextual Analysis: Comprehending meaning and relationships within document structure
Gemini API excels with complex structured documents like financial reports, legal contracts, and medical records where layout and context matter.
PDF Processing with File API
Upload and Processing Status
import google.generativeai as genai
import time
# Initialize client
client = genai.Client(api_key="YOUR_API_KEY")
def upload_and_process_pdf(file_path: str):
"""
Upload PDF and wait for processing completion
"""
print(f"Uploading file: {file_path}")
# Upload file
with open(file_path, "rb") as f:
pdf_file = client.files.upload(
file=f,
mime_type="application/pdf"
)
file_uri = pdf_file.uri
print(f"Upload complete: {file_uri}")
# Poll processing status
for attempt in range(30):
file = client.files.get(pdf_file.name)
if file.state.name == "ACTIVE":
print(f"File processing complete")
return file_uri
elif file.state.name == "FAILED":
raise Exception(f"File processing failed: {file.state.name}")
print(f"Processing... ({attempt + 1}/30)")
time.sleep(2)
raise TimeoutError("File processing timeout")
# Example usage
pdf_uri = upload_and_process_pdf("contract.pdf")Full Document Analysis and Text Extraction
Structure-Preserving Full Text Extraction
def analyze_full_pdf_content(pdf_uri: str):
"""
Extract complete PDF text preserving document structure
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Extract the complete content of this PDF document in the following structure:
# Document Structure Analysis
## Metadata
- Title:
- Date Published:
- Total Pages:
## Section-by-Section Content
Organize sections in hierarchical structure
### [Section 1 Title]
Content...
### [Section 2 Title]
Content...
## Key Terms and Named Entities
- Term 1: Definition
- Term 2: Definition
## Numerical Data
- Value 1: Amount
- Value 2: Amount
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": pdf_uri
}
])
return response.text
# Implementation example
full_content = analyze_full_pdf_content(pdf_uri)
print(full_content)Section Identification
def identify_pdf_sections(pdf_uri: str):
"""
Automatically identify document section structure
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Extract the document's section structure in JSON format:
{
"sections": [
{
"section_id": 1,
"title": "Section Title",
"page_numbers": [1, 2],
"subsections": [
{
"title": "Subsection Title",
"content_summary": "Brief summary"
}
]
}
]
}
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": pdf_uri
}
])
import json
return json.loads(response.text)Table Extraction to Structured Format
Converting Tables to JSON
def extract_tables_to_json(pdf_uri: str):
"""
Extract all tables from PDF as JSON structures
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Extract all tables in this PDF in JSON format:
{
"tables": [
{
"table_id": 1,
"title": "Table Title",
"location": "Page X",
"headers": ["Column 1", "Column 2", "Column 3"],
"rows": [
["Data11", "Data12", "Data13"],
["Data21", "Data22", "Data23"]
],
"notes": "Any footnotes"
}
]
}
Guidelines:
- Empty cells represented as null
- Handle merged cells appropriately
- Include units with values
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": pdf_uri
}
])
import json
return json.loads(response.text)
# Example usage
tables = extract_tables_to_json(pdf_uri)
for table in tables["tables"]:
print(f"Table: {table['title']}")
print(f"Location: {table['location']}")
for row in table["rows"]:
print(row)Handling Complex Tables
def extract_complex_tables_with_context(pdf_uri: str):
"""
Process complex tables with multi-level headers and merged cells
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Analyze all tables in this document carefully:
1. Capture multi-level header structures precisely
2. Record cell merging and splitting
3. Include units and supplementary information
4. Identify subtotals and summary rows
Return as JSON:
{
"tables": [
{
"table_id": 1,
"structure": {
"header_rows": 2,
"header_columns": 1,
"data_rows": 10
},
"columns": [
{
"name": "Column Name",
"type": "string|number|date",
"unit": "Unit"
}
],
"data": [...]
}
]
}
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": pdf_uri
}
])
import json
return json.loads(response.text)Chart and Graph Interpretation
Extracting Numeric Values from Charts
def analyze_charts_and_graphs(pdf_uri: str):
"""
Extract numeric data from all charts and graphs in PDF
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Analyze all charts, graphs, and visual data in this document.
Return in JSON format:
{
"charts": [
{
"chart_id": 1,
"type": "bar|line|pie|scatter",
"title": "Chart Title",
"x_axis": {
"label": "X Axis Label",
"values": ["Value1", "Value2"]
},
"y_axis": {
"label": "Y Axis Label",
"scale": "linear|logarithmic"
},
"series": [
{
"name": "Series Name",
"data": [100, 150, 120]
}
],
"insights": "Key trends and findings visible in chart"
}
]
}
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": pdf_uri
}
])
import json
return json.loads(response.text)Multi-Document Comparison
Comparative Document Analysis
def compare_multiple_documents(pdf_uris: list):
"""
Compare and analyze multiple PDF documents
"""
model = genai.GenerativeModel("gemini-2.5-pro")
# Build prompt
prompt = """
Analyze and compare the following documents, identifying similarities and differences:
Return JSON:
{
"comparison": {
"document_count": N,
"similarities": [
{
"aspect": "Comparison Point",
"details": "Description of commonalities"
}
],
"differences": [
{
"aspect": "Comparison Point",
"document_1": "Value 1",
"document_2": "Value 2",
"significance": "Importance of difference"
}
],
"summary": "Overall assessment"
}
}
"""
# Construct content
content = [prompt]
for pdf_uri in pdf_uris:
content.append({
"mime_type": "application/pdf",
"data": pdf_uri
})
response = model.generate_content(content)
import json
return json.loads(response.text)
# Example usage
comparison = compare_multiple_documents([pdf_uri_1, pdf_uri_2])
print(comparison)Automated Contract Review
Extracting Contract Terms
def review_contract_automatically(contract_uri: str):
"""
Automatically extract and analyze key contract terms
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Analyze this contract from a legal perspective.
Return as JSON:
{
"contract_analysis": {
"contract_type": "Type of contract",
"parties": [
{
"name": "Party Name",
"role": "Role"
}
],
"key_terms": {
"effective_date": "Start date",
"duration": "Contract period",
"termination_clause": "Termination conditions"
},
"financial_terms": {
"payment_amount": "Amount",
"currency": "Currency",
"payment_schedule": "Payment schedule",
"penalties": "Penalty clauses"
},
"obligations": [
{
"party": "Party",
"obligation": "Obligation description"
}
],
"risk_factors": [
{
"risk": "Risk description",
"severity": "high|medium|low",
"mitigation": "Mitigation strategy"
}
],
"compliance_requirements": [
"Compliance requirement"
],
"recommendations": "Reviewer recommendations"
}
}
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": contract_uri
}
])
import json
return json.loads(response.text)
# Example usage
contract_review = review_contract_automatically(contract_uri)
print(f"Risk factors: {contract_review['contract_analysis']['risk_factors']}")Detecting Unfavorable Clauses
def detect_unfavorable_clauses(contract_uri: str, company_context: str):
"""
Automatically identify clauses that are unfavorable to your company
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = f"""
Company context: {company_context}
Identify clauses in this contract that are unfavorable or concerning to our company:
{{
"unfavorable_clauses": [
{{
"clause_location": "Article X Section Y",
"clause_text": "Original clause text",
"concern": "Specific concern",
"impact": "Impact on our company",
"suggested_revision": "Recommended modification"
}}
],
"negotiation_priorities": [
{{
"priority": 1,
"clause": "Clause to modify",
"rationale": "Reason for modification"
}}
]
}}
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": contract_uri
}
])
import json
return json.loads(response.text)Invoice Processing
Structured Data Extraction from Invoices
def process_invoice(invoice_uri: str):
"""
Extract financial data from invoice in structured format
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Extract the following information from this invoice as JSON:
{
"invoice_metadata": {
"invoice_number": "Invoice number",
"invoice_date": "Date issued",
"due_date": "Due date",
"currency": "Currency"
},
"parties": {
"vendor": {
"name": "Vendor name",
"address": "Address",
"contact": "Contact info"
},
"customer": {
"name": "Customer name",
"address": "Address"
}
},
"line_items": [
{
"description": "Item description",
"quantity": 10,
"unit_price": 100.00,
"line_total": 1000.00
}
],
"summary": {
"subtotal": Subtotal,
"tax_amount": Tax amount,
"tax_rate": "Tax rate",
"total": Total amount,
"payment_terms": "Payment terms"
},
"payment_information": {
"bank_account": "Bank account",
"payment_method": "Payment method"
}
}
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": invoice_uri
}
])
import json
return json.loads(response.text)
# Example usage
invoice_data = process_invoice(invoice_uri)
print(f"Total amount: {invoice_data['summary']['total']}")Financial Statement Analysis
Automated Key Metrics Extraction
def analyze_financial_statement(statement_uri: str):
"""
Extract key metrics from financial reports and statements
"""
model = genai.GenerativeModel("gemini-2.5-pro")
prompt = """
Analyze this financial statement and return in JSON format:
{
"statement_info": {
"company_name": "Company name",
"fiscal_year": "Fiscal year",
"statement_type": "balance_sheet|income_statement|cash_flow"
},
"balance_sheet": {
"assets": {
"current_assets": Amount,
"non_current_assets": Amount,
"total_assets": Amount
},
"liabilities": {
"current_liabilities": Amount,
"non_current_liabilities": Amount,
"total_liabilities": Amount
},
"equity": {
"total_equity": Amount
}
},
"income_statement": {
"revenue": Revenue,
"cost_of_revenue": Cost,
"gross_profit": Profit,
"operating_expenses": Expenses,
"operating_income": Income,
"net_income": Net income
},
"key_ratios": {
"current_ratio": Current ratio,
"debt_to_equity": Debt to equity,
"profit_margin": "Profit margin",
"return_on_assets": "ROA"
},
"year_over_year_analysis": {
"revenue_change": "Revenue growth rate",
"profit_trend": "Profit trend",
"key_observations": "Key findings"
}
}
"""
response = model.generate_content([
prompt,
{
"mime_type": "application/pdf",
"data": statement_uri
}
])
import json
return json.loads(response.text)Context Caching for Efficiency
Document Caching Implementation
def process_document_with_caching(pdf_uri: str, queries: list):
"""
Run multiple queries against the same document with caching
"""
model = genai.GenerativeModel("gemini-2.5-pro")
# Cached document reference
cached_document = {
"mime_type": "application/pdf",
"data": pdf_uri
}
results = []
for i, query in enumerate(queries):
print(f"Query {i + 1}/{len(queries)}")
content = [
query,
cached_document
]
# Caching optimization (from 2nd request onwards)
if i > 0:
# Caching headers applied automatically by SDK
pass
response = model.generate_content(content)
results.append({
"query": query,
"response": response.text
})
return results
# Example usage
queries = [
"What are the key findings in this document?",
"What are the main financial trends?",
"What are the identified risk factors?"
]
cached_results = process_document_with_caching(pdf_uri, queries)Context caching significantly reduces API costs and improves performance. Use it for frequently accessed documents and multiple queries against the same PDF.
Batch Processing
Efficient Processing of Large Document Sets
from concurrent.futures import ThreadPoolExecutor
import time
class BatchDocumentProcessor:
def __init__(self, max_workers=5):
self.max_workers = max_workers
self.results = []
def process_batch(self, document_uris: list, task_type: str):
"""
Process multiple documents in parallel
"""
def process_single_document(uri, task_type):
if task_type == "contract_review":
return review_contract_automatically(uri)
elif task_type == "invoice":
return process_invoice(uri)
elif task_type == "financial":
return analyze_financial_statement(uri)
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
futures = [
executor.submit(process_single_document, uri, task_type)
for uri in document_uris
]
for i, future in enumerate(futures):
try:
result = future.result(timeout=300)
self.results.append({
"document_index": i,
"status": "success",
"result": result
})
print(f"Processed: Document {i + 1}/{len(document_uris)}")
except Exception as e:
self.results.append({
"document_index": i,
"status": "failed",
"error": str(e)
})
return self.results
# Example usage
processor = BatchDocumentProcessor(max_workers=5)
results = processor.process_batch(pdf_uris, "contract_review")
success_count = sum(1 for r in results if r["status"] == "success")
print(f"Success rate: {success_count}/{len(pdf_uris)}")Cost Management
class DocumentProcessingCostManager:
def __init__(self):
# Gemini API pricing (March 2026)
self.input_cost_per_million = 0.075 # USD
self.output_cost_per_million = 0.3
def estimate_cost(self, num_documents, avg_pages_per_doc, queries_per_doc=1):
"""
Estimate document processing costs
"""
# Convert pages to tokens (1 page ≈ 2000 tokens)
avg_tokens_per_page = 2000
total_input_tokens = (
num_documents * avg_pages_per_doc * avg_tokens_per_page *
queries_per_doc
)
# Estimate output tokens (≈20% of input)
total_output_tokens = int(total_input_tokens * 0.2)
input_cost = (total_input_tokens / 1_000_000) * self.input_cost_per_million
output_cost = (total_output_tokens / 1_000_000) * self.output_cost_per_million
total_cost = input_cost + output_cost
return {
"num_documents": num_documents,
"estimated_input_tokens": total_input_tokens,
"estimated_output_tokens": total_output_tokens,
"input_cost": f"${input_cost:.2f}",
"output_cost": f"${output_cost:.2f}",
"total_cost": f"${total_cost:.2f}",
"cost_per_document": f"${total_cost / num_documents:.4f}"
}
# Example usage
cost_mgr = DocumentProcessingCostManager()
estimate = cost_mgr.estimate_cost(
num_documents=1000,
avg_pages_per_doc=10,
queries_per_doc=3
)
print(f"Estimated cost: {estimate['total_cost']}")
print(f"Per document: {estimate['cost_per_document']}")Document Processing Patterns
Pattern: Invoice Auto-Classification and Reconciliation
def classify_and_process_invoices(invoice_uris: list):
"""
Classify invoices by vendor and process in batches
"""
model = genai.GenerativeModel("gemini-2.5-pro")
classified = {"vendors": {}, "failed": []}
for uri in invoice_uris:
# Classify by vendor
classify_prompt = "Extract vendor name and amount from this invoice"
classify_response = model.generate_content([
classify_prompt,
{"mime_type": "application/pdf", "data": uri}
])
vendor_name = extract_vendor_name(classify_response.text)
# Organize by vendor
if vendor_name not in classified["vendors"]:
classified["vendors"][vendor_name] = []
classified["vendors"][vendor_name].append(uri)
# Process by vendor
for vendor, uris in classified["vendors"].items():
print(f"\nVendor: {vendor} ({len(uris)} invoices)")
for uri in uris:
invoice_data = process_invoice(uri)
print(f" Amount: {invoice_data['summary']['total']}")
return classifiedConclusion
Gemini API's multimodal capabilities enable enterprise-grade document processing. Key takeaways:
- File API: Efficient handling of large PDFs with structured processing
- Structured Extraction: JSON-format output for reliable data capture
- Multi-Document Analysis: Cross-document comparison and relationship detection
- Batch Processing: Efficient large-scale document handling
- Context Caching: Cost reduction and performance optimization
By combining these techniques, you can build high-accuracy, cost-effective document processing systems that rival traditional specialized software while offering superior flexibility and integration.
A note from the field
Automating the Dolice Labs operations as an indie developer, the hard part of document processing was never accuracy — it was deciding how extraction fails. I attach a confidence score to critical fields and route anything below threshold to a human queue rather than silently returning blanks, and I screen out unreadable scans up front so the downstream model costs stay predictable.