The model card for MetaEncoder v1, the model’s user guide, appeared on 12 November in the Hugging Face repository facebook/meta-encoder-v1. The Deploy Log, which lists AI releases, names Meta as the publisher. It cites the same Hugging Face page and offers no information of its own.

The instruction changes the embedding

An embedding is a sequence of numbers that represents the meaning of content. Similar texts or images get sequences that lie close to each other, so they can be searched, compared and clustered. MetaEncoder produces a vector of 6,656 numbers for each input. The input can be text, an image, video frames or a combination of these.

According to the model card, the same input gets a different embedding under a different instruction. An instruction might be, for example, ‘Represent the given image.’ or ‘Find a document image that matches the given query.’ Search queries and the documents being searched usually get different instructions.

The Python code supplied with the model has three main calls: encode produces embeddings, retrieve ranks a collection against a query and decide answers questions about a given situation. The code formats prompts itself in the same way as in training.

Decisions are chosen from options

Decide handles yes/no, multiple-choice and scale questions as retrieval. It embeds each option, and the score depends on how close the option’s embedding is to the question’s embedding. A multiple-choice question can have 2–26 options. Each question is decided independently of the others.

By default, answer probabilities are calculated with the temperature used in training, 0.02. Users with labelled examples can fit the temperature to their own data with the calibrate function. Options can be given descriptions, but the card says the training data for these question types contained only the option names.

Requirements and open questions

The model card gives no benchmark results as text, only an image named benchmarks.png. The card was edited several times within a few hours on the release day. According to the change history, the metadata line ‘license: apache-2.0’ was replaced with other information, and the card text does not mention a licence. Users should therefore check the terms of use in the repository before using the model.

The model processes images at native resolution up to 4,096 tokens. If a video has more than eight frames, the model samples eight at even intervals. It does not truncate inputs, but training sequences were at most 16,384 tokens long.

The model is downloaded from Hugging Face and requires version 5.15 or later of the transformers library. The model card says the model fits on a single 80 GB GPU in bf16 precision. The card also explains how to use the model directly with transformers library classes.