/v1/schemas/google-vertex/embeddings/v1/projects/{projectsId}/locations/{locationsId}/publishers/{publishersId}/models/{modelsId}:embedContent
| field | recorded source |
|---|---|
| schema | Source linked above. upstream-spec Source traced3236f93d1c1fdefdbca53d969da555041994e556e5c18bec8ee2ce3b415ba1f{
"fetchedAt": 1793251843,
"extractorVersion": "1",
"verifiedAt": null
} |
Request body
7 properties · 0 required · 18 $defs| property | type | description | constraints |
|---|---|---|---|
| taskType | enum | Optional. Deprecated: Please use EmbedContentConfig.task_type instead. The task type of the embedding. | UNSPECIFIED | RETRIEVAL_QUERY | RETRIEVAL_DOCUMENT | SEMANTIC_SIMILARITY | CLASSIFICATION | CLUSTERING · +3 more |
| content | GoogleCloudAiplatformV1Content | The structured data content of a message. A Content message contains a `role` field, which indicates the producer of the content, and a `parts` field, which contains the multi-part data of the message. | |
| embedContentConfig | GoogleCloudAiplatformV1EmbedContentRequestEmbedContentConfig | Configurations for the EmbedContent API. | |
| title | string | Optional. Deprecated: Please use EmbedContentConfig.title instead. The title for the text. | |
| outputDimensionality | integer | Optional. Deprecated: Please use EmbedContentConfig.output_dimensionality instead. Reduced dimension for the output embedding. If set, excessive values in the output embedding are truncated from the end. | format: int32 |
| model | string | Required. The name of the publisher model requested to serve the prediction. Format: `projects/{project}/locations/{location}/publishers/*/models/*` | |
| autoTruncate | boolean | Optional. Deprecated: Please use EmbedContentConfig.auto_truncate instead. Whether to silently truncate the input content if it's longer than the maximum sequence length. |
Types
18 $defsGoogleCloudAiplatformV1AudioTranscriptionobject
The transcription of an audio part. For multi-speaker audio, each speaker segment is a separate `Part` with its own `AudioTranscription` carrying the `speaker_label`.
| speakerLabel | string | Optional. A label identifying the speaker of this audio segment (e.g. `spk_1`, `spk_2`). Present when `diarization` is set. | |
| words | GoogleCloudAiplatformV1AudioTranscriptionWordInfo[] | Optional. Detailed word-level transcriptions and timing details. Present when `word_timestamp` is set. | |
| text | string | Required. The transcription text of this audio segment. |
GoogleCloudAiplatformV1AudioTranscriptionWordInfoobject
Information about a single recognized word.
| startOffset | string | Optional. Start offset in time of the word relative to the start of the audio. | format: google-duration |
| word | string | Required. Transcript of the word. | |
| endOffset | string | Optional. End offset in time of the word relative to the start of the audio. | format: google-duration |
GoogleCloudAiplatformV1Blobobject
A content blob. A Blob contains data of a specific media type. It is used to represent images, audio, and video.
| mimeType | string | Required. The IANA standard MIME type of the source data. | |
| data | string | Required. The raw bytes of the data. | format: byte |
| displayName | string | Optional. The display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in `PromptMessage` for prompt management. It is used in the Gemini calls only when server-side tools (`code_execution`, `google_search… |
GoogleCloudAiplatformV1CodeExecutionResultobject
Result of executing the ExecutableCode. Generated only when the `CodeExecution` tool is used.
| output | string | Optional. Contains stdout when code execution is successful, stderr or other description otherwise. | |
| outcome | enum | Required. Outcome of the code execution. | OUTCOME_UNSPECIFIED | OUTCOME_OK | OUTCOME_FAILED | OUTCOME_DEADLINE_EXCEEDED |
| id | string | Optional. The identifier of the `ExecutableCode` part this result is for. Only populated if the corresponding `ExecutableCode` has an id. |
GoogleCloudAiplatformV1Contentobject
The structured data content of a message. A Content message contains a `role` field, which indicates the producer of the content, and a `parts` field, which contains the multi-part data of the message.
| role | string | Optional. The producer of the content. Must be either 'user' or 'model'. If not set, the service will default to 'user'. | |
| parts | GoogleCloudAiplatformV1Part[] | Required. A list of Part objects that make up a single message. Parts of a message can have different MIME types. A Content message must have at least one Part. |
GoogleCloudAiplatformV1EmbedContentRequestEmbedContentConfigobject
Configurations for the EmbedContent API.
| title | string | Optional. The title for the text. Only applicable to text-only embedding models. | |
| audioTrackExtraction | boolean | Optional. Whether to extract audio from video content. | |
| documentOcr | boolean | Optional. Whether to enable OCR for document content. | |
| autoTruncate | boolean | Optional. Whether to silently truncate the input content if it's longer than the maximum sequence length. Only applicable to text-only embedding models. | |
| outputDimensionality | integer | Optional. Reduced dimension for the output embedding. If set, excessive values in the output embedding are truncated from the end. | format: int32 |
| taskType | enum | Optional. The task type of the embedding. Only applicable to text-only embedding models. | UNSPECIFIED | RETRIEVAL_QUERY | RETRIEVAL_DOCUMENT | SEMANTIC_SIMILARITY | CLASSIFICATION | CLUSTERING · +3 more |
GoogleCloudAiplatformV1ExecutableCodeobject
Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the `CodeExecution` tool, in which the code will be automatically executed, and a corresponding CodeExecutionResult will also be generated.
| id | string | Optional. Unique identifier of the `ExecutableCode` part. The server returns the `CodeExecutionResult` with the matching `id`. | |
| language | enum | Required. Programming language of the `code`. | LANGUAGE_UNSPECIFIED | PYTHON |
| code | string | Required. The code to be executed. |
GoogleCloudAiplatformV1FileDataobject
URI-based data. A FileData message contains a URI pointing to data of a specific media type. It is used to represent images, audio, and video stored in Google Cloud Storage.
| fileUri | string | Required. The URI of the file in Google Cloud Storage. | |
| mimeType | string | Required. The IANA standard MIME type of the source data. | |
| displayName | string | Optional. The display name of the file. Used to provide a label or filename to distinguish files. This field is only returned in `PromptMessage` for prompt management. It is used in the Gemini calls only when server side tools (`code_execution`, `google_search… |
GoogleCloudAiplatformV1FunctionCallobject
A predicted FunctionCall returned from the model that contains a string representing the FunctionDeclaration.name and a structured JSON object containing the parameters and their values.
| id | string | Optional. The unique id of the function call. If populated, the client to execute the `function_call` and return the response with the matching `id`. | |
| willContinue | boolean | Optional. Whether this is the last part of the FunctionCall. If true, another partial message for the current FunctionCall is expected to follow. | |
| partialArgs | GoogleCloudAiplatformV1PartialArg[] | Optional. The partial argument value of the function call. If provided, represents the arguments/fields that are streamed incrementally. | |
| name | string | Optional. The name of the function to call. Matches FunctionDeclaration.name. | |
| args | object | Optional. The function parameters and values in JSON object format. See FunctionDeclaration.parameters for parameter details. |
GoogleCloudAiplatformV1FunctionResponseobject
The result output from a FunctionCall that contains a string representing the FunctionDeclaration.name and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a `FunctionCall` made…
| id | string | Optional. The id of the function call this response is for. Populated by the client to match the corresponding function call `id`. | |
| parts | GoogleCloudAiplatformV1FunctionResponsePart[] | Optional. Ordered `Parts` that constitute a function response. Parts may have different IANA MIME types. | |
| response | object | Required. The function response in JSON object format. Use "output" key to specify function output and "error" key to specify error details (if any). If "output" and "error" keys are not specified, then whole "response" is treated as function output. | |
| name | string | Required. The name of the function to call. Matches FunctionDeclaration.name and FunctionCall.name. | |
| scheduling | enum | Optional. Specifies how the response should be scheduled in the conversation. Only applicable to NON_BLOCKING function calls, is ignored otherwise. Defaults to WHEN_IDLE. | SCHEDULING_UNSPECIFIED | SILENT | WHEN_IDLE | INTERRUPT |
GoogleCloudAiplatformV1FunctionResponseBlobobject
Raw media bytes for function response. Text should not be sent as raw bytes, use the 'text' field.
| mimeType | string | Required. The IANA standard MIME type of the source data. | |
| data | string | Required. Raw bytes. | format: byte |
| displayName | string | Optional. Display name of the blob. Used to provide a label or filename to distinguish blobs. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (code_executi… |
GoogleCloudAiplatformV1FunctionResponseFileDataobject
URI based data for function response.
| displayName | string | Optional. Display name of the file data. Used to provide a label or filename to distinguish file datas. This field is only returned in PromptMessage for prompt management. It is currently used in the Gemini GenerateContent calls only when server side tools (co… | |
| fileUri | string | Required. URI. | |
| mimeType | string | Required. The IANA standard MIME type of the source data. |
GoogleCloudAiplatformV1FunctionResponsePartobject
A datatype containing media that is part of a `FunctionResponse` message. A `FunctionResponsePart` consists of data which has an associated datatype. A `FunctionResponsePart` can only contain one of the accepted types in `FunctionResponsePart.data`. A `Functio…
| inlineData | GoogleCloudAiplatformV1FunctionResponseBlob | Raw media bytes for function response. Text should not be sent as raw bytes, use the 'text' field. | |
| fileData | GoogleCloudAiplatformV1FunctionResponseFileData | URI based data for function response. |
GoogleCloudAiplatformV1Partobject
A datatype containing media that is part of a multi-part Content message. A `Part` consists of data which has an associated datatype. A `Part` must contain one of the accepted types in `Part.data`. For media types that are not text, `Part` must have a fixed IA…
| executableCode | GoogleCloudAiplatformV1ExecutableCode | Code generated by the model that is meant to be executed, and the result returned to the model. Generated when using the `CodeExecution` tool, in which the code will be automatically executed, and a corresponding CodeExecutionResult will also be generated. | |
| speechMetadata | GoogleCloudAiplatformV1SpeechMetadata | Structured Metadata Sub-Message for Part | |
| functionCall | GoogleCloudAiplatformV1FunctionCall | A predicted FunctionCall returned from the model that contains a string representing the FunctionDeclaration.name and a structured JSON object containing the parameters and their values. | |
| thoughtSignature | string | Optional. An opaque signature for the thought so it can be reused in subsequent requests. | format: byte |
| audioTranscription | GoogleCloudAiplatformV1AudioTranscription | The transcription of an audio part. For multi-speaker audio, each speaker segment is a separate `Part` with its own `AudioTranscription` carrying the `speaker_label`. | |
| inlineData | GoogleCloudAiplatformV1Blob | A content blob. A Blob contains data of a specific media type. It is used to represent images, audio, and video. | |
| mediaResolution | GoogleCloudAiplatformV1PartMediaResolution | per part media resolution. Media resolution for the input media. | |
| codeExecutionResult | GoogleCloudAiplatformV1CodeExecutionResult | Result of executing the ExecutableCode. Generated only when the `CodeExecution` tool is used. | |
| text | string | Optional. The text content of the part. When sent from the VSCode Gemini Code Assist extension, references to @mentioned items will be converted to markdown boldface text. For example `@my-repo` will be converted to and sent as `**my-repo**` by the IDE agent. | |
| thought | boolean | Optional. Indicates whether the `part` represents the model's thought process or reasoning. | |
| fileData | GoogleCloudAiplatformV1FileData | URI-based data. A FileData message contains a URI pointing to data of a specific media type. It is used to represent images, audio, and video stored in Google Cloud Storage. | |
| videoMetadata | GoogleCloudAiplatformV1VideoMetadata | Provides metadata for a video, including the start and end offsets for clipping and the frame rate. | |
| functionResponse | GoogleCloudAiplatformV1FunctionResponse | The result output from a FunctionCall that contains a string representing the FunctionDeclaration.name and a structured JSON object containing any output from the function is used as context to the model. This should contain the result of a `FunctionCall` made… |
GoogleCloudAiplatformV1PartialArgobject
Partial argument value of the function call.
| nullValue | enum | Optional. Represents a null value. | NULL_VALUE |
| numberValue | number | Optional. Represents a double value. | format: double |
| stringValue | string | Optional. Represents a string value. | |
| jsonPath | string | Required. A JSON Path (RFC 9535) to the argument being streamed. https://datatracker.ietf.org/doc/html/rfc9535. e.g. "$.foo.bar[0].data". | |
| willContinue | boolean | Optional. Whether this is not the last part of the same json_path. If true, another PartialArg message for the current json_path is expected to follow. | |
| boolValue | boolean | Optional. Represents a boolean value. |
GoogleCloudAiplatformV1PartMediaResolutionobject
per part media resolution. Media resolution for the input media.
| level | enum | The tokenization quality used for given media. | MEDIA_RESOLUTION_UNSPECIFIED | MEDIA_RESOLUTION_LOW | MEDIA_RESOLUTION_MEDIUM | MEDIA_RESOLUTION_HIGH | MEDIA_RESOLUTION_ULTRA_HIGH |
GoogleCloudAiplatformV1SpeechMetadataobject
Structured Metadata Sub-Message for Part
| style | string | Optional. Natural language description of the vocal style (e.g., "cheerful"). | |
| speaker | string | Optional. Identifies which speaker is speaking this turn. |
GoogleCloudAiplatformV1VideoMetadataobject
Provides metadata for a video, including the start and end offsets for clipping and the frame rate.
| endOffset | string | Optional. The end offset of the video. | format: google-duration |
| startOffset | string | Optional. The start offset of the video. | format: google-duration |
| fps | number | Optional. The frame rate of the video sent to the model. If not specified, the default value is 1.0. The valid range is (0.0, 24.0]. | format: double |
Validate a payload
before you spend tokens$ curl -X POST https://modelschemas.com/v1/validate \
-d '{"provider":"google-vertex","endpointId":"v1/projects/{projectsId}/locations/{locationsId}/publishers/{publishersId}/models/{modelsId}:embedContent","payload":{…}}'
→ {"valid": false, "errors": [{"path": "#", "keyword": "required", …}]}