Plugin System
Build interactive plugins with HTML, CSS, and JavaScript. 6 plugin types across 12 categories.
Overview#
Plugins add interactive UI elements to your video. Each plugin is a self-contained component with HTML, CSS, and JavaScript that runs in a sandboxed iframe. Plugins communicate with the player through the mp4e bridge API.
You can create plugins directly in the Studio Plugin Editor (a built-in code editor with live preview), install them from the Marketplace, or build them externally and import them.
Plugin Types#
MP4E has 6 plugin types, each designed for different interaction patterns:
| Type | Description | Default Size | Has UI |
|---|---|---|---|
| overlay | Fixed position UI element placed in video layers | 400 x 300 | Yes |
| object-display | Shows when user interacts with detected objects (hover, click) | 300 x 200 | Yes |
| modal | Fullscreen dialog that pauses video playback | 420 x 600 | Yes |
| subtitle | Time-synced subtitle text with special rendering | 800 x 100 | Yes |
| controls | Video player controls (play/pause/seek) | 400 x 80 | Yes |
| service | Always-alive plugin that provides custom actions and variables | Hidden | No (background) |
Overlay
Fixed position UI element placed in video layers
// Show overlay via action
mp4e.executeActions([{
type: 'showCustomOverlay',
pluginId: 'plugin_cta',
position: 'bottom-right',
margin: 16
}]);
// Hide overlay
mp4e.executeActions([{
type: 'hideCustomOverlay',
pluginId: 'plugin_cta'
}]);Object Display
Shows when user interacts with detected objects (hover, click)
// Configured in objectGroup displaySettings:
{
"displaySettings": {
"eventBindings": [{
"event": "clicked",
"enabled": true,
"display": {
"pluginType": "mp4e:product-card",
"config": {
"title": "{{object.userLabel}}",
"price": "{{object.data.price}}"
},
"position": { "position": "above", "offsetY": 10 }
}
}]
}
}Modal
Fullscreen dialog that pauses video playback
// Show modal from JavaScript
mp4e.executeActions([{
type: 'showCustomModal',
pluginId: 'plugin_checkout',
pauseVideo: true,
config: { title: 'Complete Purchase' }
}]);
// Close modal from inside the plugin
mp4e.close();Subtitle
Time-synced subtitle text with special rendering
// Subtitle plugins receive word-level timing data:
// - Words with start/end times for highlighting
// - Current word index for karaoke effects
// Access via mp4e.getData() or the data context
// Listen for word clicks
mp4e.onSubtitleWordClick((wordData) => {
console.log('Clicked word:', wordData.text);
});Controls
Video player controls (play/pause/seek)
// Controls plugin receives video state via events:
mp4e.on('mp4e:video:timeupdate', (state) => {
updateProgressBar(state.currentTime / state.duration);
});
mp4e.on('mp4e:video:play', () => setPlayIcon('pause'));
mp4e.on('mp4e:video:pause', () => setPlayIcon('play'));
// Control playback
document.getElementById('play-btn')
.addEventListener('click', () => mp4e.playVideo());Servicebackground
Always-alive plugin that provides custom actions and variables
// Service plugins provide variables, actions, and events
// to other plugins — no visible UI.
// Example: Shopping cart service
// Variables: cart:items, cart:itemCount, cart:total
// Actions: addItem, removeItem, clearCart
// Events: cartUpdated, itemAdded
// Other plugins call service actions:
mp4e.executeActions([{
type: 'pluginAction',
pluginId: 'cart',
actionId: 'addItem',
config: { productId: '{{object.id}}', price: '{{object.data.price}}' }
}]);Plugin Editor#
The Studio includes a built-in Plugin Editor — a full code editor with live preview for creating and editing plugins. Open it from any overlay's settings.
The editor has a two-panel layout with 12 tabs you can arrange between the left and right panels:
Cmd/Ctrl and click a tab to open it in the right panel. This lets you edit HTML on the left while seeing the live Preview on the right, or write Script while referencing the API Docs.Plugin Structure#
Every plugin consists of HTML, CSS, JavaScript, and a configuration schema:
{
id: 'plugin_product_card',
name: 'Product Card',
type: 'object-display', // overlay | object-display | modal | subtitle | controls | service
// Content — the actual plugin code
content: {
html: '<div class="card"><h3>{{object.userLabel}}</h3></div>',
css: '.card { padding: 16px; background: white; border-radius: 8px; }',
script: 'const config = mp4e.getConfig(); /* ... */'
},
// Configuration schema — generates form UI in Studio
configSchema: {
type: 'object',
properties: {
showPrice: { type: 'boolean', title: 'Show Price', default: true },
accentColor: { type: 'string', format: 'color', title: 'Accent Color', default: '#3b82f6' }
}
},
// Events the plugin can emit (available as rule triggers)
emits: {
purchased: {
label: 'Item Purchased',
description: 'Fired when user completes purchase',
dataSchema: { type: 'object', properties: { productId: { type: 'string' } } }
}
},
// Actions other plugins/rules can call on this plugin
actions: [
{ id: 'highlight', label: 'Highlight', description: 'Flash highlight effect' }
],
// Variables auto-created when plugin is installed
variables: [
{ id: 'viewCount', name: 'View Count', type: 'number', default: 0 }
]
}Plugin API#
The mp4e global provides methods for interacting with the player. All methods communicate via postMessage from the sandboxed iframe.
// ═══════════════════════════════════════════
// CONFIG & DATA (Sync)
// ═══════════════════════════════════════════
const config = mp4e.getConfig(); // Plugin configuration values
const data = mp4e.getData(); // Data object (object data for group-bound)
const pluginId = mp4e.getPluginId(); // This plugin's ID
// ═══════════════════════════════════════════
// VARIABLES (Async)
// ═══════════════════════════════════════════
const score = await mp4e.getVariable('score');
await mp4e.setVariable('score', score + 10);
mp4e.onVariableChange((variables, changedIds) => {
// changedIds = array of changed variable IDs, or undefined for full broadcast
if (!changedIds || changedIds.includes('score')) {
updateScoreUI(variables.score);
}
});
// ═══════════════════════════════════════════
// PLAYBACK (Sync getters, Async setters)
// ═══════════════════════════════════════════
const time = mp4e.getCurrentTime(); // Current time in seconds
const frame = mp4e.getCurrentFrame(); // Current frame number
const dur = mp4e.getDuration(); // Video duration
const playing = mp4e.isPlaying(); // Playback state
await mp4e.playVideo(); // Play
await mp4e.pauseVideo(); // Pause
await mp4e.seekToTime(30); // Seek to 30s
await mp4e.setVolume(0.5); // Volume (0-1)
await mp4e.setPlaybackRate(1.5); // Playback speed
// ═══════════════════════════════════════════
// EVENTS (Sync)
// ═══════════════════════════════════════════
mp4e.emit('purchased', { productId: '123' }); // Emit custom event
mp4e.on('mp4e:video:play', () => { /* ... */ });
mp4e.on('mp4e:video:pause', () => { /* ... */ });
mp4e.on('mp4e:video:timeupdate', (data) => { /* ... */ });
// ═══════════════════════════════════════════
// ACTIONS
// ═══════════════════════════════════════════
mp4e.executeActions([
{ type: 'showNotification', message: 'Added!', variant: 'success' },
{ type: 'setVariable', variableId: 'cartCount', operation: 'increment', value: 1 }
]);
mp4e.close(); // Close this overlay
await mp4e.openUrl('https://...', '_blank'); // Open URL
// ═══════════════════════════════════════════
// INTER-PLUGIN COMMUNICATION
// ═══════════════════════════════════════════
mp4e.onAction('highlight', (config) => { /* handle action from rules */ });
await mp4e.callAction('cart-plugin', 'addItem', { sku: '123' });
// ═══════════════════════════════════════════
// STORAGE (Async, sandboxed per plugin)
// ═══════════════════════════════════════════
await mp4e.storage.set('lastSeen', Date.now());
const val = await mp4e.storage.get('lastSeen');
// ═══════════════════════════════════════════
// SCALING & VIEWPORT (Sync)
// ═══════════════════════════════════════════
const scale = mp4e.getScaleFactor();
const px = mp4e.scale(16); // Scale-adjusted pixels
const vp = mp4e.getViewport(); // { width, height, scaleFactor, hasTouch }Categories#
Plugins are organized into 12 semantic categories for the marketplace:
Lifecycle#
Understanding the plugin lifecycle helps write robust plugins:
// 1. IFRAME CREATED
// - Engine generates srcDoc with bridge script + your HTML/CSS/JS
// - {{config.field}} values interpolated in HTML/CSS at generation time
// - {{variable}} patterns preserved for runtime interpolation
// 2. BRIDGE READY
// - mp4e global object available
// - mp4e.getConfig() returns current configuration
// - window.object available (for group-bound overlays)
// - mp4e.ready() auto-called on DOMContentLoaded
// 3. INITIAL DATA
// - mp4e-variable-update message delivers all variable values
// - mp4e-interpolated message resolves remaining {{variable}} patterns
// - onVariableChange callbacks fire with changedIds = undefined (full broadcast)
// 4. RUNTIME
// - onVariableChange fires when variables change (with changedIds array)
// - onConfigUpdate fires when overlay config changes
// - Video events (play, pause, timeupdate) delivered via mp4e.on()
// - Plugin can call any mp4e API method
// 5. CLEANUP
// - Plugin closes via: mp4e.close(), backdrop click, hideOverlay action,
// or overlay visibility change
// - Iframe is destroyed — no manual cleanup needed
// - Tip: clear intervals/timeouts to prevent errors during teardown