Unity × Gemini API: Intelligent NPC Conversations

Talk to a villager three times, hear the same line come back, and the world flattens into painted scenery. No amount of extra dialogue branches quite repairs that feeling.

An NPC that answers in its own words, reads the state of the world, and holds its personality across a conversation is a different experience altogether — and wiring Gemini API into Unity puts it within reach. Here's the build, end to end.

Why Gemini for Game NPCs

Ultra-Fast Responses (1.2 sec latency ideal for gaming)
Multimodal Capable (future: face expression recognition)
Cost-Effective (low API costs per response)
Character Consistency (system prompts maintain personality)

Gemini API Setup

Google Cloud Configuration

Visit Google Cloud Console
Create new project ("UnityGameAI")
Enable Gemini API
Generate API key (Credentials → New Key)

Unity API Key Management

using UnityEngine;
 
[CreateAssetMenu(menuName = "AI/Gemini Config")]
public class GeminiConfig : ScriptableObject
{
    [SerializeField]
    private string apiKey;
 
    public string ApiKey => apiKey;
}

Important: In production, access APIs through backend server, not client directly.

Gemini API Client Implementation

using UnityEngine;
using UnityEngine.Networking;
using System.Collections;
using System.Text;
using Newtonsoft.Json;
 
public class GeminiAPIClient : MonoBehaviour
{
    [SerializeField]
    private GeminiConfig config;
 
    private const string API_URL =
        "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent";
 
    public IEnumerator SendMessage(string userMessage, System.Action<string> onResponse)
    {
        var request = new GeminiRequest
        {
            contents = new[] {
                new Content {
                    role = "user",
                    parts = new[] { new Part { text = userMessage } }
                }
            }
        };
 
        string jsonData = JsonConvert.SerializeObject(request);
        byte[] postData = Encoding.UTF8.GetBytes(jsonData);
 
        string url = $"{API_URL}?key={config.ApiKey}";
        using (var www = new UnityWebRequest(url, "POST"))
        {
            www.uploadHandler = new UploadHandlerRaw(postData);
            www.downloadHandler = new DownloadHandlerBuffer();
            www.SetRequestHeader("Content-Type", "application/json");
 
            yield return www.SendWebRequest();
 
            if (www.result == UnityWebRequest.Result.Success)
            {
                var response = JsonConvert.DeserializeObject<GeminiResponse>(www.downloadHandler.text);
                string responseText = response.candidates[0].content.parts[0].text;
                onResponse?.Invoke(responseText);
            }
            else
            {
                Debug.LogError($"Error: {www.error}");
                onResponse?.Invoke("Error occurred");
            }
        }
    }
 
    // Serializable structures for API
    [System.Serializable]
    public class GeminiRequest { public Content[] contents; }
 
    [System.Serializable]
    public class Content
    {
        public string role;
        public Part[] parts;
    }
 
    [System.Serializable]
    public class Part { public string text; }
 
    [System.Serializable]
    public class GeminiResponse { public Candidate[] candidates; }
 
    [System.Serializable]
    public class Candidate { public Content content; }
}

NPC Character System

Base NPC Class

using UnityEngine;
using System.Collections.Generic;
 
public class NPCCharacter : MonoBehaviour
{
    [SerializeField]
    private string npcName;
 
    [SerializeField]
    private string characterPrompt;
 
    [SerializeField]
    private GeminiAPIClient apiClient;
 
    private List<(string role, string message)> conversationHistory;
 
    private void Start()
    {
        conversationHistory = new List<(string, string)>();
    }
 
    public void SendMessage(string userMessage)
    {
        conversationHistory.Add(("user", userMessage));
 
        var sb = new System.Text.StringBuilder();
        sb.AppendLine(characterPrompt);
        sb.AppendLine("\n【Conversation History】");
 
        foreach (var (role, msg) in conversationHistory)
        {
            sb.AppendLine($"{(role == "user" ? "Player" : npcName)}: {msg}");
        }
 
        StartCoroutine(apiClient.SendMessage(sb.ToString(), OnResponseReceived));
    }
 
    private void OnResponseReceived(string response)
    {
        conversationHistory.Add(("npc", response));
        // Display in UI
        DisplayDialogue(response);
    }
 
    private void DisplayDialogue(string text)
    {
        // Show NPC response in dialogue panel
        var panel = GetComponent<DialoguePanel>();
        panel.ShowMessage(npcName, text);
    }
}

Character Personality Definition

Key to realistic NPCs: Detailed system prompts

You are "Frodia," a wise 150-year-old wizard living in the eastern tower.

Personality: Calm, thoughtful, occasionally humorous
Catchphrase: "Hmm, I see..." "Do you understand?"

World Knowledge:
- Magic uses four elements (fire, water, wind, earth)
- Humans and elves coexist but have tensions
- Dark magic is forbidden

Conversation Rules:
1. Always respond in character
2. Use world knowledge
3. Keep responses 1-3 sentences
4. Answer player questions helpfully

Example:
Player: "I want to learn fire magic"
Frodia: "Good intention, young one. Fire magic harnesses flame element.
        First learn the basics. Will you practice?"

This level of detail ensures consistent, contextual responses.

Cost Optimization

Conversation History Management

private const int MAX_HISTORY_PAIRS = 5;
 
private string BuildConversationContext(string userMessage)
{
    // Keep only last 5 exchanges to limit tokens
    int startIndex = Mathf.Max(0, conversationHistory.Count - MAX_HISTORY_PAIRS * 2);
 
    var sb = new System.Text.StringBuilder();
    sb.AppendLine(characterPrompt);
 
    for (int i = startIndex; i < conversationHistory.Count; i++)
    {
        var (role, msg) = conversationHistory[i];
        sb.AppendLine($"{(role == "user" ? "Player" : npcName)}: {msg}");
    }
 
    sb.AppendLine($"Player: {userMessage}");
    return sb.ToString();
}

Response Caching

private Dictionary<string, string> responseCache = new();
 
public IEnumerator SendMessage(string userMessage, System.Action<string> onResponse)
{
    if (responseCache.TryGetValue(userMessage, out string cached))
    {
        onResponse?.Invoke(cached);
        yield break;
    }
 
    // API call... then cache result
    responseCache[userMessage] = response;
}

Fallback Responses

private string GetFallbackResponse(string userMessage)
{
    if (userMessage.Contains("hello"))
        return "Hello there, traveler. What brings you?";
    else if (userMessage.Contains("quest"))
        return "Quests... check the bulletin board.";
    else
        return "Hmm... (contemplating)";
}

Practical Examples

Merchant NPC

public class MerchantNPC : NPCCharacter
{
    [SerializeField]
    private List<ItemData> inventory;
 
    protected override string BuildCharacterPrompt()
    {
        var sb = new System.Text.StringBuilder();
        sb.AppendLine("You are Marco, a merchant. Shrewd, business-minded.");
        sb.AppendLine("\nCurrent Inventory:");
 
        foreach (var item in inventory)
            sb.AppendLine($"- {item.name}: {item.price} gold");
 
        sb.AppendLine("\nRecommend items to the customer.");
        return sb.ToString();
    }
}

Quest NPC

public class QuestGiverNPC : NPCCharacter
{
    [SerializeField]
    private List<QuestData> availableQuests;
 
    // Similar implementation:  Dynamic quest offers based on inventory
}

UI Integration

public class DialoguePanel : MonoBehaviour
{
    [SerializeField] private Text characterNameText;
    [SerializeField] private Text dialogueText;
 
    public void ShowMessage(string characterName, string message)
    {
        characterNameText.text = characterName;
        StartCoroutine(TypeMessage(message));
    }
 
    private IEnumerator TypeMessage(string message)
    {
        dialogueText.text = "";
        foreach (char c in message)
        {
            dialogueText.text += c;
            yield return new WaitForSeconds(0.05f);
        }
    }
}

Best Practices

Character Consistency: Detailed, specific system prompts
Response Length: Limit to 1-3 sentences for game pacing
Token Management: Cache, limit history, use fallbacks
Error Handling: Network failures need graceful degradation
Testing: Thoroughly test NPC personalities before deployment

Security Notes

Never embed API keys in client code
Use backend proxy for production
Rate limit API calls per player
Validate all inputs before API calls

Wrapping up

Gemini API-powered NPCs transform game immersion. With proper configuration, your NPCs become genuinely intelligent conversationalists, not dialogue tree puppets.

Unity × Gemini API: Give NPCs Real Intelligence — Game Development Guide