Analyze and generate media with ak:ai-multimodal

Use ak:ai-multimodal to analyze images, transcribe audio or video, extract structured data, convert documents, and generate image, video, speech, or music assets. The Skill orchestrates the exact-pinned @mrgoonie/multix@0.2.0 CLI; provider catalogs remain the authority for current models, limits, retention, features, and pricing.

Choose provider-backed multimodal work

Use ak:ai-multimodal when

• You need OCR, visual evidence, layout understanding, or cross-image comparison.
• You need a timestamped transcript, scene map, summary, or structured extraction from audio or video.
• You need to convert a supported document to Markdown.
• You need provider-generated images, video, speech, or music with an explicit model and review step.
• You need to optimize or split media before a verified provider upload limit.

Choose another workflow when

• You want curated prompt search for a new visual concept. Use ak:ai-artist.
• The work is deterministic local resizing, encoding, compositing, metadata, or background removal. Use ak:media-processing.
• You cannot upload the source media or prompt to a third party.
• A critical OCR, identity, measurement, legal, medical, or safety conclusion cannot be independently checked against the original.

Prepare inputs, tools, and provider access

Before you start:

• Complete Onboarding, and confirm Engineer Kit is installed for the current runtime and scope.
• Use Node.js 20 or newer and the pinned invocation npx -y -p @mrgoonie/multix@0.2.0 multix; do not substitute a floating global multix installation.
• Run multix --version through that invocation, then multix check --verbose. First use can download the npm package and later use the npm cache.
• Configure only the provider needed: GEMINI_API_KEY, optional OPENROUTER_API_KEY, or optional MINIMAX_API_KEY. The pinned CLI can discover keys from process environment, project .env, or ~/.multix/.env.
• Inspect the relevant pinned subcommand with --help, then verify a current provider model, supported inputs and controls, retention, quota, and pricing.
• Name every input, intended output, allowed upload, redaction need, and cleanup requirement. Install or approve FFmpeg only when the selected operation needs it.

Run a bounded analysis

/ak:ai-multimodal "Analyze ./assets/receipt.png, extract merchant, date, currency, total, and line items to ./artifacts/receipt.json, separate unreadable fields from inferred values, and do not upload any other file"

/ak:ai-multimodal "Analyze ./assets/receipt.png, extract merchant, date, currency, total, and line items to ./artifacts/receipt.json, separate unreadable fields from inferred values, and do not upload any other file"

$ak:ai-multimodal "Analyze ./assets/receipt.png, extract merchant, date, currency, total, and line items to ./artifacts/receipt.json, separate unreadable fields from inferred values, and do not upload any other file"

Match the operation to the evidence

Do not replace <verified-model-id> with a value copied from this page. Model IDs and controls must be resolved against the live provider before generation.

Observe the processing stages

1. The Skill defines the evidence contract. It identifies source files, analysis or generation goal, output format, uncertainty, and privacy boundary.
2. It checks the pinned backend. Node, npm cache, provider keys, FFmpeg when applicable, and the relevant Multix command are verified.
3. It resolves live provider facts. The model, supported modality, input method, size or duration limit, controls, retention, quota, and price are checked before a request.
4. It prepares media only as needed. Oversized inputs are compressed or split after the verified boundary is known; source detail needed for OCR or fine visual inspection is preserved.
5. It processes or generates. The pinned CLI uploads approved inputs and writes the requested local artifact.
6. It validates against the source or brief. Names, numbers, timestamps, visible evidence, prompt adherence, continuity, pronunciation, and artifacts are checked for the selected modality.
7. It reports and cleans up. Provider, model, commands, files, segment status, limitations, and required deletion of provider-hosted inputs are recorded.

Control uploads, privacy, rights, and cost

• Never place real provider keys in prompts, committed .env files, logs, or generated reports. Preserve provider errors but redact credentials.
• Obtain consent and appropriate rights for voices, likenesses, recordings, music, documents, reference images, and generated output before use or publication.
• Generation and analysis can consume billable tokens, media seconds, storage, or provider jobs. Generate short review samples first and approve expensive retries.
• npx can download code and use the npm cache. FFmpeg optimization creates new files and consumes local CPU, memory, disk, and process time.
• Splitting media can lose cross-segment context. Record chunk boundaries and do not invent transcript or narrative continuity when merging.
• The Skill writes artifacts locally but does not publish, commit, deploy, or delete originals without separate authority.

Verify the result

A complete result should include:

• Pinned Multix version, exact command family, provider, explicit model when applicable, and live capability checks.
• Input paths, files actually uploaded, any compression or segmentation, and provider cleanup status.
• Output paths and formats, plus timestamps, schema, prompt, or settings needed to interpret each artifact.
• Source-grounded validation of names, numbers, text, timing, visual evidence, motion, sound, pronunciation, or prompt adherence as relevant.
• Provider errors, uncertain regions, failed chunks, rights or privacy review, and current limits that constrained the result.

Stable and beta package identical ak:ai-multimodal content and the same exact Multix pin. Provider catalogs are live external state; the release does not prove current model availability, pricing, retention, or cross-runtime parity.

Troubleshoot and interpret limits