OpenAI Introduces Next-Gen AI: GPT-4 Turbo, Assistants API, and More

On November 6, 2023, OpenAI took a significant leap forward, introducing a series of groundbreaking features and improvements to its platform. Among the key highlights are the unveiling of the powerful GPT-4 Turbo model, a more developer-friendly Assistants API, expanded multimodal capabilities, and a commitment to affordability with reduced pricing. 

GPTs are a new way for anyone to create a tailored version of ChatGPT to be more helpful in their daily life, at specific tasks, at work, or at home — and then share that creation with others. No code required. https://t.co/SPV4TcMiQw pic.twitter.com/PcmorZwtMF

— OpenAI (@OpenAI) November 6, 2023

In this blog post, we’ll delve into the details of these announcements and explore how they mark a transformative moment in the landscape of artificial intelligence.

GPT-4 Turbo: A Leap into the Future

OpenAI initially released GPT-4 in March, making it available to developers in July. Now, the anticipation builds as the company introduces a preview of the next generation: GPT-4 Turbo. This new model boasts enhanced capabilities, broader knowledge spanning up to April 2023, and a remarkable 128k context window. The latter allows GPT-4 Turbo to comprehend the equivalent of over 300 pages of text in a single prompt, significantly expanding its applicability.

We’re rolling out new features and improvements that developers have been asking for:

1. Our new model GPT-4 Turbo supports 128K context and has fresher knowledge than GPT-4. Its input and output tokens are respectively 3× and 2× less expensive than GPT-4. It’s available now to…

— OpenAI (@OpenAI) November 6, 2023

Notably, OpenAI has not only augmented the model’s performance but also optimized pricing. Developers can now harness the power of GPT-4 Turbo at a remarkable 3x lower cost for input tokens and 2x lower cost for output tokens compared to its predecessor, GPT-4. This strategic pricing move aims to empower a wider range of developers to leverage the advanced capabilities of GPT-4 Turbo, fostering innovation and exploration.

While GPT-4 Turbo is currently available for preview by passing the API identifier “gpt-4-1106-preview,” the stable production-ready model is expected to roll out in the coming weeks, opening up new possibilities for AI-driven applications and solutions.

Function Calling Updates: Streamlining Interactions

Function calling is a pivotal aspect of AI applications, allowing developers to describe functions to models and receive JSON objects containing relevant arguments. OpenAI has introduced several improvements in this domain. Users can now make multiple function calls in a single message, reducing the need for multiple roundtrips with the model. This enhancement enhances efficiency and responsiveness in interactions with GPT-4 Turbo.

Moreover, the accuracy of function calling has been improved, with GPT-4 Turbo exhibiting a higher likelihood of returning the correct function parameters. This improvement enhances the reliability of AI-driven applications, ensuring that they accurately interpret and execute user instructions.

Improved Instruction Following and JSON Mode

One of the notable strengths of GPT-4 Turbo lies in its enhanced ability to follow instructions meticulously. Tasks that demand precision, such as generating specific formats like XML, showcase the model’s proficiency. OpenAI has also introduced a new JSON mode, allowing developers to specify that the model’s output conforms to valid JSON. This feature, coupled with the response_format API parameter, empowers developers to seamlessly integrate JSON output in the Chat Completions API outside of function calling scenarios.

Reproducible Outputs and Log Probabilities

In a move towards providing developers with greater control and transparency, OpenAI has introduced the seed parameter, enabling reproducible outputs from GPT-4 Turbo. This beta feature ensures consistent completions in a majority of instances, facilitating tasks such as debugging and comprehensive unit testing. The ability to reproduce outputs is a valuable tool for developers, offering a higher degree of predictability in the behavior of the model.

Additionally, OpenAI plans to launch a feature that returns log probabilities for the most likely output tokens generated by both GPT-4 Turbo and GPT-3.5 Turbo in the coming weeks. This feature is anticipated to play a crucial role in building functionalities like autocomplete in search experiences, further enhancing user interactions with AI-powered applications.

Updated GPT-3.5 Turbo: Enhancing Contextual Understanding

In parallel with the advancements in GPT-4 Turbo, OpenAI introduces an updated version of GPT-3.5 Turbo. This model now supports a 16K context window by default, offering improved contextual understanding. Notable improvements include enhanced instruction following, JSON mode support, and parallel function calling. Internal evaluations demonstrate a 38% improvement in format following tasks, making GPT-3.5 Turbo a robust choice for developers seeking advanced contextual capabilities.

Developers can access the new model by calling “gpt-3.5-turbo-1106” in the API. Notably, applications using the older “gpt-3.5-turbo” identifier will automatically be upgraded to the new model on December 11, ensuring a seamless transition to the enhanced version.

Assistants API: Empowering Developers to Create Agent-Like Experiences

OpenAI’s commitment to empowering developers takes center stage with the introduction of the Assistants API. This API represents a significant step towards enabling developers to build agent-like experiences within their applications. An assistant, in this context, is a purpose-built AI with specific instructions, leveraging extra knowledge and the ability to call models and tools to perform tasks.

The Assistants API introduces several capabilities:

• Code Interpreter: This tool allows assistants to write and run Python code in a sandboxed execution environment. It can generate graphs, charts, and process files with diverse data and formatting. Code Interpreter enables assistants to iteratively run code, solving complex code and math problems.
• Retrieval: Augmenting assistants with knowledge from external sources, such as proprietary domain data or user-provided documents. This feature eliminates the need for developers to compute and store embeddings for documents, streamlining the integration of external knowledge into AI applications.
• Function Calling: Enabling assistants to invoke user-defined functions and incorporate the function response in their messages. This capability enhances the versatility of AI-driven applications by allowing seamless integration with custom functions defined by developers.

The Assistants API introduces a key change with persistent and infinitely long threads. This innovation allows developers to delegate thread state management to OpenAI, overcoming context window constraints and facilitating a more dynamic and continuous interaction with the assistant.

Developers can try the Assistants API beta without writing any code by accessing the Assistants playground, providing a user-friendly environment to experiment and create high-quality assistants.

New Modalities in the API: Vision, DALL·E 3, and Text-to-Speech (TTS)

Expanding the modalities supported by OpenAI’s API, GPT-4 Turbo now incorporates vision capabilities. Developers can input images in the Chat Completions API, enabling diverse use cases such as generating captions, detailed analysis of real-world images, and document reading with figures. For example, BeMyEyes utilizes this technology to assist individuals who are blind or have low vision with daily tasks.

The API identifier “gpt-4-vision-preview” unlocks access to this feature, with pricing dependent on the input image size. This addition positions GPT-4 Turbo as a versatile solution for applications that require a combination of text and image processing.

DALL·E 3, previously available to ChatGPT Plus and Enterprise users, is now integrable into third-party apps and products through the Images API. Developers can specify “dall-e-3” as the model, tapping into the creative potential of AI-generated images. Notable companies like Snap, Coca-Cola, and Shutterstock have leveraged DALL·E 3 for programmatically generating images and designs.

The introduction of Text-to-Speech (TTS) capabilities adds another layer of richness to the API. Developers can now generate human-quality speech from text using the TTS API. The model offers six preset voices, each optimized for different use cases. The pricing starts at $0.015 per input 1,000 characters, providing an accessible and effective solution for integrating voice capabilities into applications.

GPT-4 Fine Tuning and Custom Models: Tailoring AI to Specific Needs

OpenAI recognizes the importance of customization in AI applications and introduces initiatives to address varying levels of customization needs.

GPT-4 Fine Tuning Experimental Access

GPT-4 Fine Tuning, an experimental program, is introduced to provide developers with the ability to fine-tune the model according to their specific requirements. While preliminary results indicate that fine-tuning for GPT-4 may require more effort compared to GPT-3.5, OpenAI is actively working to improve quality and safety. Developers currently using GPT-3.5 Fine Tuning will have the option to apply for the GPT-4 program within their fine-tuning console as quality and safety measures progress.

Custom Models Program

For organizations with highly specific customization needs, OpenAI launches the Custom Models program. This initiative allows selected organizations to collaborate with a dedicated group of OpenAI researchers to train a custom GPT-4 model tailored to their specific domain.

The program encompasses every step of the model training process, from domain-specific pre-training to a custom RL post-training process. Organizations granted access to custom models will have exclusive rights, and the data provided for training will be handled with utmost privacy, aligning with OpenAI’s enterprise privacy policies.

While the Custom Models program is positioned as a limited and potentially expensive offering, it addresses the unique requirements of organizations dealing with extremely large proprietary datasets.

Lower Prices and Higher Rate Limits: Fostering Accessibility

OpenAI is committed to making its advanced AI capabilities more accessible to developers, and this commitment is reflected in the announcement of lower prices and higher rate limits.

Lower Prices

The reduction in prices spans across different models:

• GPT-4 Turbo: Input tokens are now 3x cheaper at $0.01, and output tokens are 2x cheaper at $0.03 compared to GPT-4.
• GPT-3.5 Turbo: Input tokens are 3x cheaper at $0.001, and output tokens are 2x cheaper at $0.002 compared to the previous 16K model. Developers previously using GPT-3.5 Turbo 4K benefit from a 33% reduction on input tokens at $0.001. These prices apply to the new GPT-3.5 Turbo introduced on this occasion.
• Fine-tuned GPT-3.5 Turbo 4K Model: Input tokens see a 4x reduction at $0.003, and output tokens are 2.7x cheaper at $0.006. Fine-tuning also supports 16K context at the same price as 4K with the new GPT-3.5 Turbo model.

These price reductions aim to democratize access to advanced AI models, encouraging developers to explore and integrate OpenAI’s capabilities into their applications.

Higher Rate Limits

Recognizing the need for scalability, OpenAI is doubling the tokens per minute limit for all paying GPT-4 customers. This enhancement provides developers with increased capacity, enabling them to scale their applications seamlessly. Developers can view their new rate limits in the rate limit page, and OpenAI has published usage tiers that determine automatic rate limit increases, providing transparency in how usage limits scale.

Developers can also request increases to usage limits directly from their account settings, ensuring flexibility and adaptability to varying application needs.

Copyright Shield: Defending Users Against Legal Claims

In a significant move to protect users, OpenAI introduces Copyright Shield. This feature signifies OpenAI’s commitment to defending customers facing legal claims related to copyright infringement. For users of ChatGPT Enterprise and the developer platform, OpenAI steps in and covers the costs incurred in legal claims. This proactive approach aligns with industry best practices and emphasizes OpenAI’s dedication to safeguarding its user community.

Whisper v3 and Consistency Decoder: Advancements in Speech Recognition

OpenAI isn’t solely focused on text-based AI advancements; it is also making strides in speech recognition. The release of Whisper large-v3, the latest version of the open-source automatic speech recognition model (ASR), brings improved performance across languages. OpenAI plans to integrate Whisper v3 into its API in the near future, expanding the applications of ASR in diverse scenarios.

Additionally, OpenAI is open-sourcing the Consistency Decoder, a drop-in replacement for the Stable Diffusion VAE decoder. This decoder enhances images compatible with the Stable Diffusion 1.0+ VAE, showcasing significant improvements in text, faces, and straight lines. These contributions to the open-source community underline OpenAI’s commitment to collaborative progress in the field of AI.

OpenAI’s Strategic Evolution: A Look at Industry Impact

The announcements made by OpenAI on November 6, 2023, reveal a strategic evolution in the company’s approach to artificial intelligence. By introducing powerful models like GPT-4 Turbo, refining APIs, and addressing customization needs with programs like GPT-4 Fine Tuning and Custom Models, OpenAI is not just advancing technology; it is reshaping how developers and organizations interact with AI.

The commitment to lower prices and higher rate limits underscores OpenAI’s dedication to fostering accessibility and inclusivity in the AI landscape. The introduction of the Assistants API and the GPT Store further opens avenues for developers to create and share AI-driven experiences, fostering a collaborative ecosystem.

As OpenAI continues to push the boundaries of AI capabilities, the industry watches closely. The democratization of advanced AI models, coupled with a proactive approach to legal challenges and contributions to open-source initiatives, positions OpenAI as a driving force in the ongoing AI revolution.

Conclusion: Shaping the Future of AI

OpenAI’s recent announcements mark a significant leap forward in the field of artificial intelligence. GPT-4 Turbo, with its extended context window, affordability, and improved capabilities, emerges as a powerhouse for developers seeking advanced natural language processing. The Assistants API introduces a new paradigm for building agent-like experiences, offering tools like Code Interpreter, Retrieval, and Function Calling to streamline the development of high-quality AI apps.

The integration of multimodal capabilities, including vision support in GPT-4 Turbo and the availability of DALL·E 3 and TTS, expands the horizons of AI applications, making it more versatile and adaptable across various domains. OpenAI’s commitment to model customization through fine-tuning and the Custom Models program empowers developers and organizations to tailor AI models to their specific needs.
Lower prices, higher rate limits, and the introduction of Copyright Shield demonstrate OpenAI’s dedication to accessibility, affordability, and customer protection. As OpenAI continues to push the boundaries of AI technology, these advancements pave the way for a future where AI becomes an integral and customizable part of diverse applications, from natural language understanding to image generation and beyond. The journey towards AI innovation is ongoing, and OpenAI’s contributions are shaping the landscape for developers, businesses, and users alike.