Documentation menu

A/B Test Design Variants User Guide

Learn how to create and manage A/B tests with optional design variations to optimize your email campaigns.

Overview

PostBeagle's A/B testing feature helps you determine which version of your email performs better. You can test:

  • Subject lines - Which subject gets more opens?
  • Preview text - Which preview drives more engagement?
  • From names - Which sender name builds more trust?
  • Email designs - Which layout drives more clicks? (Optional)

By default, both variants share the same email design. This guide shows you how to optionally create different designs for variant B to test content variations alongside metadata.

When to Use Design Variants

Use Different Designs When:

✅ Testing Content Hierarchy - Compare different layouts or content organization
✅ Testing Visual Elements - Compare different images, colors, or styling
✅ Testing CTA Placement - Compare different call-to-action button positions
✅ Testing Email Length - Compare short vs. long-form content
✅ Testing Content Types - Compare text-heavy vs. image-heavy designs

Use Same Design When:

✅ Subject Line Testing - Focus purely on subject line optimization
✅ From Name Testing - Test sender name variations
✅ Preview Text Testing - Optimize preview text
✅ Timing Tests - Test send times with identical content

Creating an A/B Test with Same Design

This is the default and simplest approach - both variants share the same email design.

Step 1: Navigate to Campaigns

  1. Click Campaigns in the sidebar
  2. Click Create New Campaign
  3. Select A/B Test Campaign

Step 2: Configure Variant A

  1. Subject Line: Enter your first subject line option
  2. Preview Text: Add preview text (optional)
  3. From Name: Enter sender name
  4. From Email: Select verified sender email
  5. Design Email: Click to open the email builder
  6. Create your email design using GrapesJS or the new builder
  7. Save your design

Step 3: Configure Variant B

  1. Subject Line: Enter your second subject line option
  2. Preview Text: Add different preview text (optional)
  3. From Name: Keep same or change
  4. From Email: Keep same or change
  5. Design for Variant B: Select "Use same design as Variant A" (default)

Step 4: Configure Test Settings

  1. Test Duration: How long to run the test (e.g., 24 hours)
  2. Test Percentage: What percentage of your audience receives the test (e.g., 20%)
  3. Winning Metric: Choose Open Rate or Click Rate

Step 5: Select Recipients and Send

  1. Choose your email list or segment
  2. Review both variants
  3. Send test emails to yourself
  4. Launch the A/B test

Creating an A/B Test with Different Designs

When you want to test different email layouts or content variations.

Step 1: Create Variant A

Follow the same steps as above to create Variant A with your first design.

Step 2: Choose Different Design for Variant B

In the Variant B section:

  1. Design for Variant B: Select "Use different design for Variant B"
  2. You'll see two options:
    • Clone from Variant A - Start with a copy of Variant A's design
    • Start from Scratch - Create a completely new design

Option A: Clone from Variant A

This is recommended when you want to make small modifications to the original design.

  1. Click "Clone from Variant A"
  2. The system copies Variant A's design to Variant B
  3. The email builder opens with the cloned design
  4. Make your modifications:
    • Change headlines
    • Adjust images
    • Modify button placement
    • Update content sections
  5. Save your changes

Option B: Start from Scratch

This is recommended when you want to test completely different design approaches.

  1. Click "Start from Scratch"
  2. Choose to start with:
    • A blank canvas
    • A different template
  3. Design your email from the ground up
  4. Save your design

Step 3: Review Both Designs

Before sending, review both variants:

  1. Click the preview icon next to each variant
  2. Check how they look on desktop and mobile
  3. Verify all links work correctly
  4. Ensure images load properly

Step 4: Configure and Send

Complete the test settings and send as described in the same design workflow.

Editing Designs After Creation

Editing Variant A's Design

  1. Open your A/B test campaign
  2. In the Variant A section, click "Edit Design"
  3. Make your changes in the email builder
  4. Save your changes

Editing Variant B's Design

  1. Open your A/B test campaign
  2. In the Variant B section, click "Edit Design" (only visible if using different design)
  3. Make your changes in the email builder
  4. Save your changes

Switching Between Same and Different Designs

Switching to Different Design

If you created an A/B test with the same design but want to add a different design for Variant B:

  1. Open your A/B test campaign
  2. In the Variant B section, change the radio button to "Use different design for Variant B"
  3. Choose to clone or start from scratch
  4. Create your Variant B design
  5. Save your changes

Switching to Same Design

If you want to go back to using the same design for both variants:

  1. Open your A/B test campaign
  2. In the Variant B section, change the radio button to "Use same design as Variant A"
  3. A confirmation dialog appears warning that Variant B's design will be deleted
  4. Click "Confirm" to proceed

Understanding the Results

Viewing A/B Test Results

After your test completes:

  1. Navigate to Campaigns
  2. Click on your A/B test campaign
  3. View the Results Dashboard

Results for Same Design Tests

You'll see:

  • Variant A Stats: Opens, clicks, open rate, click rate
  • Variant B Stats: Opens, clicks, open rate, click rate
  • Winner Badge: Indicates which variant won
  • Design Note: "Both variants use the same design"
  • Single Design Preview: One preview thumbnail (since both use the same design)

Results for Different Design Tests

You'll see:

  • Variant A Stats: Opens, clicks, open rate, click rate
  • Variant A Design Preview: Thumbnail of Variant A's design
  • Variant B Stats: Opens, clicks, open rate, click rate
  • Variant B Design Preview: Thumbnail of Variant B's design
  • Winner Badge: Indicates which variant won
  • Winner Design Preview: Thumbnail of the winning variant's design

Best Practices for Design Testing

1. Make Meaningful Differences

Don't test tiny variations that recipients won't notice. Make changes significant enough to potentially impact behavior:

✅ Good: Testing a single-column vs. two-column layout
❌ Bad: Testing slightly different shades of the same color

2. Test One Design Element at a Time

If possible, isolate what you're testing:

✅ Good: Same content, different CTA button placement
❌ Bad: Different content, layout, images, and CTA all at once

3. Use Larger Sample Sizes for Design Tests

Design variations often have smaller effect sizes than subject line tests:

  • Subject line tests: 10-20% test percentage is usually sufficient
  • Design tests: Consider 20-30% test percentage for better statistical significance

4. Allow Sufficient Test Duration

Give recipients time to engage with your email:

  • Subject line tests: 4-8 hours may be sufficient
  • Design tests: 24-48 hours recommended (people may click later)

5. Consider Mobile Optimization

Ensure both designs work well on mobile devices:

  1. Use the preview feature to check mobile rendering
  2. Test on actual mobile devices before sending
  3. Keep mobile-first design principles in mind

6. Document Your Learnings

Keep track of what works:

  • Screenshot winning designs
  • Note what elements performed best
  • Build a library of successful patterns
  • Share insights with your team

Common Workflows

Workflow 1: Subject Line Optimization

Goal: Find the best subject line without changing design

  1. Create A/B test with same design
  2. Variant A: "Your Weekly Newsletter"
  3. Variant B: "This Week's Top Stories"
  4. Test duration: 8 hours
  5. Test percentage: 20%
  6. Winning metric: Open Rate

Result: The subject line with higher open rate wins and is sent to remaining 80%

Workflow 2: CTA Button Testing

Goal: Test different call-to-action button placements

  1. Create A/B test with different designs
  2. Clone Variant A to Variant B
  3. In Variant B, move CTA button from bottom to top
  4. Keep subject lines identical
  5. Test duration: 24 hours
  6. Test percentage: 30%
  7. Winning metric: Click Rate

Result: The design with higher click rate wins

Workflow 3: Content Length Testing

Goal: Test short vs. long-form content

  1. Create A/B test with different designs
  2. Variant A: Short, concise email (3 paragraphs)
  3. Variant B: Start from scratch, create detailed email (10 paragraphs)
  4. Keep subject lines identical
  5. Test duration: 48 hours
  6. Test percentage: 30%
  7. Winning metric: Click Rate

Result: Discover whether your audience prefers brief or detailed content

Workflow 4: Visual vs. Text-Heavy

Goal: Test image-heavy vs. text-heavy design

  1. Create A/B test with different designs
  2. Variant A: Multiple images, minimal text
  3. Variant B: Start from scratch, text-focused with one hero image
  4. Keep subject lines identical
  5. Test duration: 24 hours
  6. Test percentage: 25%
  7. Winning metric: Click Rate

Result: Learn which content type resonates with your audience

Troubleshooting

I can't see the "Edit Design" button for Variant B

Cause: You're using the same design for both variants

Solution: Change the radio button to "Use different design for Variant B" first

My cloned design looks different in the editor

Cause: Different editors may render designs slightly differently

Solution: This is normal. The HTML output will be identical when sent. Preview the email to see the actual rendering.

I accidentally switched to same design and lost my Variant B design

Cause: The confirmation dialog was accepted

Solution: Unfortunately, this action cannot be undone. You'll need to recreate the design. Consider keeping backups of important designs.

Both variants are sending the same design even though I created different designs

Cause: The hasDifferentDesign flag may not be set correctly

Solution:

  1. Check that you saved Variant B's design
  2. Verify the "Use different design" option is selected
  3. Contact support if the issue persists

The winner is using the wrong design

Cause: This should not happen if configured correctly

Solution: Contact support immediately with your campaign ID

Tips for Success

Start Simple

If you're new to A/B testing:

  1. Start with subject line tests (same design)
  2. Build confidence with the process
  3. Graduate to design testing once comfortable

Build a Testing Calendar

Plan your tests strategically:

  • Week 1: Subject line test
  • Week 2: Preview text test
  • Week 3: Design layout test
  • Week 4: CTA placement test

Segment Your Tests

Different audiences may respond differently:

  • Test designs separately for engaged vs. inactive subscribers
  • Test different approaches for different customer segments
  • Use insights to personalize future campaigns

Combine with Other Features

Enhance your A/B tests:

  • Use segments to target specific audiences
  • Use scheduling to send at optimal times
  • Use analytics to dive deeper into results

Advanced Techniques

Sequential Testing

Run multiple tests in sequence to optimize different elements:

  1. Test 1: Find best subject line (same design)
  2. Test 2: Using winning subject, test designs
  3. Test 3: Using winning design, test CTA copy
  4. Test 4: Using winning CTA, test send time

Multi-Variant Testing

While PostBeagle supports A/B testing (2 variants), you can simulate multi-variant testing:

  1. Run Test 1: A vs. B
  2. Run Test 2: Winner vs. C
  3. Run Test 3: Winner vs. D

Seasonal Design Testing

Test different design approaches for different seasons:

  • Holiday-themed designs
  • Seasonal color schemes
  • Weather-appropriate imagery

Next Steps

Now that you understand A/B test design variants: