OpenAI

Abstract The present technology pertains to providing, configuring and operating of a virtual assistant that can be accessible via an application programming interface. The virtual assistant can be configured with a customized system message, and can have access to a conversation thread of undefined length, whereby the virtual assistant can both be specialized for a given task and can have an ability to recall past interactions, thus mimicking the skill and memory of a human assistant. The present technology provides for a first entity to configure the virtual assistant and to call the virtual assistant by referencing its assistant ID.

Abstract Disclosed herein are methods, systems, and computer-readable media for integrating an application programming interface (API) with a natural language model user interface. In one embodiment, a method includes receiving a registration of an external API via a user interface connected to a natural language model, the natural language model being configured to integrate a plurality of external APIs, accessing a manifest file hosted at a first online location by a publisher of the external API, the manifest file comprising parameters for interfacing with the external API and a second online location of a specification associated with the external API, the parameters and second online location being defined by the publisher of the external API, accessing the specification at the second online location, and integrating the external API with the natural language model based on data from at least one of the manifest file or the specification.

Abstract The present technology pertains to a system for generating automations based on a request/prompt from a user to a language model. The language model receives a prompt in a conversation thread with a user. When the prompt includes a request for a delayed action (i.e., an automation), the request is forwarded to an automations engine, which determines the delayed action from an action component of the request and a time at which to perform the delayed action from a temporal component of the request. A scheduling instruction based on the determined time is sent to a scheduler, which signals when the time occurs, triggering the automations engine to perform the delayed action, for example, by acting as the user's delegate and sending a delayed prompt to the language model to elicit a response and amending part or all of the resulting conversation to the conversation thread with the user.

Abstract Disclosed herein are methods, systems, and computer-readable media for regenerating a region of an image with a machine learning model based on a text input. Disclosed embodiments involve accessing a digital input image. Disclosed embodiments involve generating a masked image by removing a masked region from the input image. Disclosed embodiments involve accessing a text input corresponding to an image enhancement prompt. Disclosed embodiments include providing at least one of the input image, the masked region, or the text input to a machine learning model configured to generate an enhanced image. Disclosed embodiments involve generating, with the machine learning model, the enhanced image based on at least one of the input image, the masked region, or the text input.

Abstract Disclosed herein are methods, systems, and computer-readable media for generating an output transcript from an input audio segment using a multi-task transformer model. In some embodiments, the transformer model can be trained to transcribe or translate audio data in multiple languages using labeled audio data. The labeled audio data can include first audio segments associated with first same-language transcripts of the first audio segments and second audio segments associated with second different-language transcripts of the second audio segments. In some embodiments, a vocabulary of the model can include special purpose and time stamp tokens. The special purpose tokens can specify tasks for the model to perform.

Abstract Disclosed herein are methods, systems, servers, and computer-readable media for interpreting computer code with a multimodal machine learning model. In an embodiment, this comprises: receiving, an input comprising at least one of a text prompt, file prompt, or data object, determining, using a multimodal machine learning model, that the input requires implementing computer code, and in response to determining the input requires implementing computer code: generating computer code based on the input, executing the generated computer code using a code interpreter, and providing, through an interface, an output based on the generated computer code.

Abstract Disclosed herein are methods, systems, and computer-readable media for generating an image corresponding to a text input. In an embodiment, operations may include accessing a text description and inputting the text description into a text encoder. The operations may include receiving, from the text encoder, a text embedding, and inputting at least one of the text description or the text embedding into a first sub-model configured to generate, based on at least one of the text description or the text embedding, a corresponding image embedding. The operations may include inputting at least one of the text description or the corresponding image embedding, generated by the first sub-model, into a second sub-model configured to generate, based on at least one of the text description or the corresponding image embedding, an output image. The operations may include making the output image, generated by the first second sub-model, accessible to a device.

Abstract Disclosed herein are methods and systems for generating metadata from content using one or more machine learning models. In an embodiment, a method may include receiving the content through a graphical user interface associated with the large language model, generating a first file by tokenizing the content into an input format for the large language model and merging the tokenized content with a content instruction, inputting the first file into the large language model, generating, using the large language model, metadata from at least the first file, the metadata reflecting a context associated with the content, generating a second file, the second file comprising the metadata, and displaying the generated metadata on the graphical user interface.