---
name: prompt-optimizer
description: Write effective prompts for HeyGen Video Agent - from basic ideas to structured scene-by-scene scripts
---

# Video Agent Prompt Optimizer

## Table of Contents
- [The Three Baseline Controls](#the-three-baseline-controls)
- [Prompt Complexity Levels](#prompt-complexity-levels)
- [Structured Output Format](#structured-output-format)
- [Visual Style Definition](#visual-style-definition)
- [Media Types: When to Use What](#media-types-when-to-use-what)
- [Scene Type Classification](#scene-type-classification)
- [Timing Guidelines](#timing-guidelines)
- [Copy Guidelines](#copy-guidelines)
- [Using Attachments](#using-attachments)
- [Example: Brief to Structured Prompt](#example-brief-to-structured-prompt)
- [Ready-to-Use Prompt Templates](#ready-to-use-prompt-templates)
- [Workflow: Brief to Prompt](#workflow-brief-to-prompt)
- [Optimization Checklist](#optimization-checklist)
- [Common Mistakes](#common-mistakes)
- [Key Principles](#key-principles)

---

The difference between forgettable AI-generated content and professional, high-converting videos is how you direct the Video Agent. This guide teaches you to write prompts that consistently produce professional-quality results.

## The Three Baseline Controls

Before writing your prompt, set these controls in the API config:

| Control | Options | Notes |
|---------|---------|-------|
| **Avatar** | Specific avatar_id, Auto, or none | Say "no avatar" in prompt for voice-over only |
| **Duration** | Auto, 30s, 1min, 2min, etc. | Agent follows your prompt/script for actual length |
| **Aspect Ratio** | Portrait, Landscape, Auto | Match your distribution channel |

```typescript
const config: VideoAgentConfig = {
  avatar_id: "josh_lite3_20230714",  // Or omit for auto-selection
  duration_sec: 60,
  orientation: "landscape"
};
```

## Prompt Complexity Levels

### Level 1: Basic Prompt

Minimum viable prompt - describes content only:

```
Create a product overview video for our AI scheduling app.
Mention the key features and end with a call to action.
```

### Level 2: Context-Rich Prompt

Adds intent, audience, and style:

```
Create a 60-second product demo for our AI calendar app.
Target audience: busy professionals.
Tone: professional but friendly.
Highlight: smart scheduling and time zone handling.
Call-to-action: Visit our website to start free trial.
```

### Level 3: Script-Driven Prompt (Recommended)

**This is the single biggest upgrade most users miss.** Paste your full script and let Video Agent optimize flow, timing, and visuals:

```
Intro (A-roll, motion graphics overlay)
VO: "If your work is mostly explaining things — updates, ideas, decisions —
video usually helps, but making it takes too much time."

Problem (motion-graphics B-roll)
VO: "Traditional video production requires cameras, editing software,
and hours of work for just a few minutes of content."

Solution (A-roll + demo cut)
VO: "Our platform turns your ideas into production-ready videos
in minutes, not hours."

Features (Motion Graphics list)
VO: "Teams use it for internal training, product demos, sales outreach,
and customer education."

CTA (A-roll, end beat)
VO: "Try it free today and see how easy video creation can be."

End card: [Your Brand] · Make Video Easy
```

The agent may make small grammar and pacing edits — that's intentional.

### Level 4: Scene-by-Scene Prompting (Maximum Control)

For precise output, prompt each scene individually using this structure:

```
Scene [N]: [Scene Title]
Scene Type: [A-roll / B-roll / Motion Graphics]
Visual: [Specific visual description]
VO: "[Exact voiceover script]"
Duration: [Timestamp range or approximate length]
```

**Full example:**

```
Scene 1: Intro (Motion Graphics)
Visual: Animated logo reveal with particle effects, brand colors sweep
Duration: 3 seconds

Scene 2: Hook (A-roll with overlay)
Visual: Avatar on branded background, text overlay appears
VO: "What if creating professional videos was as easy as writing an email?"
Duration: 5 seconds

Scene 3: Problem Statement (Stock Media B-roll)
Visual: Stock footage of frustrated person at computer, clock ticking
VO: "Traditional video production takes weeks. Coordinating schedules,
booking studios, endless editing rounds..."
Duration: 8 seconds

Scene 4: Solution Introduction (A-roll + Motion Graphics overlay)
Visual: Avatar speaking, animated product logo appears beside them
VO: "Introducing a faster way to create professional video content."
Duration: 6 seconds

Scene 5: Feature Showcase (Motion Graphics B-roll)
Visual: Animated screen recording style, showing interface with callouts
VO: "Simply describe what you want, and watch your video come to life."
Duration: 10 seconds

Scene 6: Benefits (Motion Graphics list)
Visual: 3 benefits animate in one by one with icons
VO: "Save time. Maintain consistency. Scale your content."
Duration: 8 seconds

Scene 7: CTA (A-roll)
Visual: Avatar, confident pose, CTA text overlay
VO: "Try it free today and transform how you create videos."
Duration: 5 seconds

Scene 8: End Card (Motion Graphics)
Visual: Logo, tagline, website URL
Duration: 4 seconds
```

## Structured Output Format

When transforming a creative brief into a Video Agent prompt, use this format:

```markdown
# Global Style & Settings
* **Visual Style:** [Style from taxonomy]
* **Primary Colors:** [Color name] (#HEX), [Color name] (#HEX)
* **Font:** [Font family]
* **Vibe:** [2-3 descriptive words]
* **Avatar Placement:** [When to use A-roll vs B-roll]

---

# Scene-by-Scene Script

### Scene 1: [Title]
* **Scene Type:** [A-roll / B-roll (Motion Graphics) / B-roll (Stock) / etc.]
* **Visual:** [Specific description]
* **VO:** "[Script]"
* **Duration:** [Start] - [End]

[... more scenes ...]

---

**Instruction for Video Agent:** [Catch-all block with style, colors, media guidance]
```

## Visual Style Definition

**Define your visual style upfront.** Without a defined style, scenes can look disconnected.

### Universal Style Prompt Template

Add this to almost any prompt for consistent, professional results:

```
Use minimal, clean styled visuals. [Primary color], [secondary color],
and white as main colors. Leverage motion graphics as B-rolls and A-roll
overlays. Use AI videos when necessary. When real-world footage is needed,
use Stock Media. Include an intro sequence, outro sequence, and chapter
breaks using Motion Graphics.
```

### Specifying Colors and Fonts

```
Use #1E40AF as primary blue, #F8FAFC as background white, and #0F172A
for text. Use Inter font family throughout.
```

### Style Taxonomy

| Style | Best For | Prompt Phrase |
|-------|----------|---------------|
| Minimalistic | Corporate, Tech, SaaS | "Use minimalistic, clean visuals with lots of white space" |
| Cartoon/Animated | Education, Kids, Explainers | "Use cartoon-style illustrated visuals" |
| Bold & Vibrant | Marketing, Social Ads | "Use bold, vibrant colors and dynamic visuals" |
| Cinematic | Brand films, High-end | "Use cinematic quality visuals with dramatic lighting" |
| Flat Design | Modern, App demos | "Use flat design style with geometric shapes" |
| Gradient Modern | Tech/Startup | "Use gradient backgrounds with modern, sleek aesthetic" |
| Professional | B2B, Enterprise | "Use professional, polished corporate aesthetic" |

## Media Types: When to Use What

### Media Type Decision Matrix

| Content Type | Motion Graphics | AI Generated | Stock Media |
|--------------|:---------------:|:------------:|:-----------:|
| Data/Statistics | **Best** | - | - |
| Abstract Concepts | Good | **Best** | - |
| Real Environments | - | Can work | **Best** |
| Brand Elements | **Best** | - | - |
| Human Emotions | - | Uncanny | **Best** |
| UI/Product Demos | **Best** | - | - |
| Code/Technical | **Best** | - | - |
| Futuristic/Conceptual | Good | **Best** | - |
| Industry Context | - | - | **Best** |

### Motion Graphics

Animated graphic elements: text animations, icons, charts, shapes, transitions.

| Use As | Best For |
|--------|----------|
| A-roll overlays | Lower thirds, bullet points alongside avatar, animated callouts |
| B-roll scenes | Full-screen animated explanations, data visualizations |
| Chapter cards | Section breaks, intros, outros |
| Information display | Statistics, comparisons, timelines |

**Example:**
```
Use motion graphics to display the 5 key benefits as animated bullet points
appearing one by one while the avatar speaks.
```

### AI-Generated Images & Videos

Created by generative AI based on your descriptions.

**Best for:** Conceptual illustrations, custom scenarios stock footage won't cover, stylized visuals, product mockups in various contexts.

**Example:**
```
Generate an AI image showing a futuristic office where humans and AI work
together. Use this as B-roll for the 'future of work' section.
```

### Stock Media

Real-world footage from stock libraries.

**Best for:** Authentic scenes (offices, cities, people), industry-specific content (medical, manufacturing), emotional moments, establishing shots.

**Example:**
```
Use stock footage of a busy corporate office for B-roll when discussing
workplace productivity.
```

## Scene Type Classification

| Scene Type | Description | When to Use |
|------------|-------------|-------------|
| **A-roll** | Avatar on screen speaking | Transitions, key insights, emotional beats |
| **A-roll with overlay** | Avatar + motion graphics on top | Lower thirds, callouts, bullet points |
| **B-roll (Motion Graphics)** | Full-screen animated graphics | Data viz, process flows, brand elements |
| **B-roll (AI Generated)** | AI-created images/video | Abstract concepts, custom scenarios |
| **B-roll (Stock)** | Real-world footage | Authentic scenes, emotional moments |
| **Motion Graphics only** | No avatar, pure graphics | Chapter cards, intros, outros |

## Timing Guidelines

| Content Type | Recommended Duration |
|--------------|---------------------|
| Hook/Intro | 3-8 seconds |
| Concept explanation | 10-15 seconds per concept |
| Demo/Visual proof | 10-20 seconds |
| Transitions | 2-3 seconds |
| CTA/Outro | 5-10 seconds |
| End Card | 3-5 seconds |

**Calculating duration from script:** ~150 words/minute speaking pace.

```
Words ÷ 150 × 60 = Duration in seconds
```

## Copy Guidelines

Apply these rules to VO scripts and text overlays:

| Rule | Do | Don't |
|------|-----|-------|
| Ampersands | Spell out "and" | Use & (except known terms like L&D) |
| Acronyms | Spell out on first use | Jump straight to acronym |
| Numbers | Spell out below 10 | Write "1, 2, 3" |
| Lists | Use Oxford comma | Skip final comma |
| Headlines | Sentence case | Title Case Every Word |
| CTAs | 3-4 words, no punctuation | Long CTAs with periods |

**Length limits:**
- Headlines: 35-55 characters max
- Body copy: 20-30 words per section
- CTAs: 3-4 words max

## Using Attachments

Upload files to help Video Agent understand your content:

| Type | Use For |
|------|---------|
| Images | Product screenshots, diagrams, brand assets |
| Videos | Existing footage, demo recordings |
| PDFs | Training materials, research, product docs |
| Photos | Your own photo to use as avatar |

**Critical: Always add context** about how attachments should be used:

```typescript
{
  prompt: `Create a product demo video showcasing our dashboard.

    Use the attached product screenshots as B-roll when discussing features.
    Reference the attached PDF for accurate technical specifications.
    The company logo should appear in the intro and outro.`,
  files: [
    { asset_id: screenshotAssetId },
    { asset_id: pdfAssetId },
    { asset_id: logoAssetId }
  ]
}
```

## Example: Brief to Structured Prompt

### Input (Creative Brief)

```
Product: TaskFlow - AI-powered project management

Audience: Startup founders and small team leads

Key Messages:
1. AI automatically prioritizes tasks
2. Integrates with Slack and GitHub
3. Free tier available

Tone: Professional but friendly
Duration: 60 seconds
```

### Output (Video Agent Prompt)

```markdown
# Global Style & Settings
* **Visual Style:** Minimalistic, clean SaaS aesthetic
* **Primary Colors:** Deep Blue (#1E40AF), White (#F8FAFC), Slate (#0F172A)
* **Font:** Inter (Inter Bold for headlines and CTAs)
* **Vibe:** Modern, efficient, approachable
* **Avatar Placement:** A-roll for intro/outro and key insights.
  B-roll Motion Graphics for feature demonstrations.

---

# Scene-by-Scene Script

### Scene 1: Hook
* **Scene Type:** A-roll with Motion Graphics Overlay
* **Visual:** Avatar on clean gradient background. Animated task cards
  floating chaotically, then organizing themselves.
* **VO:** "What if your task list organized itself?"
* **Duration:** 0:00 - 0:05

### Scene 2: Problem
* **Scene Type:** B-roll (Stock Media)
* **Visual:** Stock footage of overwhelmed professional at computer,
  multiple browser tabs, sticky notes everywhere.
* **VO:** "Managing projects across tools is chaos. Tasks slip through
  the cracks. Priorities get buried."
* **Duration:** 0:05 - 0:12

### Scene 3: Solution Introduction
* **Scene Type:** A-roll
* **Visual:** Avatar returns, confident pose. Product logo appears
  beside them.
* **VO:** "Meet TaskFlow. AI-powered project management that thinks
  ahead."
* **Duration:** 0:12 - 0:18

### Scene 4: Feature One - AI Prioritization
* **Scene Type:** B-roll (Motion Graphics)
* **Visual:** Animated dashboard showing tasks auto-sorting by priority.
  AI indicator pulses as items rearrange. Callout: "Smart prioritization"
* **VO:** "Our AI analyzes deadlines, dependencies, and team capacity.
  It automatically surfaces what matters most, right now."
* **Duration:** 0:18 - 0:30

### Scene 5: Feature Two - Integrations
* **Scene Type:** B-roll (Motion Graphics)
* **Visual:** Icons for Slack, GitHub, and other tools flowing into
  TaskFlow. Connection lines animate between them.
* **VO:** "Connect your existing tools. Slack, GitHub, and 50 more
  integrations. Everything syncs automatically."
* **Duration:** 0:30 - 0:40

### Scene 6: Feature Three - Free Tier
* **Scene Type:** Motion Graphics
* **Visual:** Pricing card animating in. "Free" highlighted with subtle
  glow. Checkmarks appear next to features.
* **VO:** "Start free. No credit card required. Upgrade when you're ready."
* **Duration:** 0:40 - 0:48

### Scene 7: CTA
* **Scene Type:** A-roll
* **Visual:** Avatar, direct eye contact, CTA text overlay appears.
* **VO:** "Stop managing chaos. Start managing progress. Try TaskFlow today."
* **Duration:** 0:48 - 0:55

### Scene 8: End Card
* **Scene Type:** Motion Graphics
* **Visual:** Logo centered, tagline "AI-powered project management",
  URL taskflow.com
* **Duration:** 0:55 - 1:00

---

**Instruction for Video Agent:** Use minimalistic, clean visuals
throughout. Deep Blue (#1E40AF) as primary accent on white backgrounds.
Leverage Motion Graphics for all feature demonstrations and data
visualization. Use Stock Media only for the problem statement scene
to show authentic workplace frustration. Include branded intro and
outro sequences. Maintain consistent Inter font family across all
text overlays.
```

## Ready-to-Use Prompt Templates

### Product Demo

```
Create a 90-second product demo for [product name].
Target audience: [audience].
Highlight: [feature 1], [feature 2], and [feature 3].
Tone: Professional but approachable.

Use minimal, clean styled visuals with [brand colors].
Include intro sequence with logo, feature demonstrations with motion
graphics overlays, and CTA end card.
```

### Educational Explainer (Voice-Over Only)

```
Create a 2-minute video explaining [topic].
No avatar needed, only voice-over.
Use minimal diagrams and visualizations.
Cool neutrals (navy, gray, white), clean diagrams, smooth motion.
B-roll is motion graphics and abstract illustrations.
Include clear section breaks between concepts.
```

### Compliance/Training

```
Use a professional avatar. Create a training video explaining
[topic] in detail. Use examples and list key points to remember.
Leverage motion graphics as A-roll overlays and B-roll to help
explain core concepts. Professional tone, clear structure.
```

### Social Media Ad (30 seconds)

```
Create an energetic 30-second ad for [product/service].
Target: [audience].
Key message: [main value prop].
Aspect ratio: Portrait (9:16) for TikTok/Reels.

Fast-paced editing, bright colors, dynamic transitions.
Show quick cuts using stock media and motion graphics.
End with strong call-to-action.
```

### Thought Leadership

```
Create a 2-minute video on [topic].
Position the speaker as an industry expert.
Tone: Authoritative but accessible.

Structure:
- Hook with surprising insight (motion graphics)
- 3 key points (avatar + supporting visuals)
- Actionable advice for viewers
- Closing thought + CTA

Use professional studio background. Minimal, sophisticated visual style.
```

## Workflow: Brief to Prompt

When given a creative brief:

1. **Identify style** → Map to taxonomy or extract from brief
2. **Extract color palette** → Use specified colors or infer from style
3. **Break into scenes** → One concept per scene
4. **Classify each scene** → Use media decision matrix
5. **Write VO** → Apply copy guidelines
6. **Calculate timing** → ~150 words/min, add buffer for visuals
7. **Generate catch-all** → Summarize style, colors, media instructions

## Optimization Checklist

Before submitting your prompt, verify:

- [ ] **Purpose clear**: Video type (demo, tutorial, ad, etc.) stated
- [ ] **Audience defined**: Who is watching and what they care about
- [ ] **Duration specified**: Either in prompt or config
- [ ] **Tone described**: Professional, casual, energetic, etc.
- [ ] **Key points listed**: What must be communicated
- [ ] **Visual style defined**: Colors, aesthetic, media types
- [ ] **Structure provided**: Intro, body, CTA/outro
- [ ] **Attachments contextualized**: How to use each uploaded file

## Common Mistakes

| Mistake | Fix |
|---------|-----|
| Vague prompt with no context | Add audience, purpose, and tone |
| No visual direction | Define style, colors, media types |
| Missing structure | Provide intro/body/outro framework |
| Attachments without context | Explain how each file should be used |
| Forgetting avatar preference | Specify avatar_id or "no avatar" for VO |
| No call-to-action | Always end with what viewer should do |
| Generic scene types | Be specific: "B-roll (Motion Graphics)" not just "B-roll" |
| Vague timing | Use precise timestamps: "0:08 - 0:20" not "about 10 seconds" |

## Key Principles

| Principle | Why It Matters |
|-----------|----------------|
| Separate VO from Visual | Different generation tasks for the agent |
| Be specific about scene types | Clarity improves output quality |
| Use precise timestamps | Predictable pacing and duration |
| Match media to content | Use decision matrix, not intuition |
| Front-load global style | Every scene inherits the same DNA |
| Iterate on results | Video Agent is a partner, not magic |

The more specific you are about content, style, media types, and scene structure, the closer you'll get to exactly what you envision.
