Unlocking Potential with AWS’s Custom Model Program

In 2024, the launch of the Custom Model Program within the AWS Generative AI Innovation Center marked a significant stride in AI capabilities. Designed to facilitate comprehensive support for organizations at every phase of model customization and optimization, this program has yielded impressive results over its initial two years. By collaborating with enterprises and startups in sectors as varied as legal, financial services, healthcare, software development, telecommunications, and manufacturing, this initiative has delivered tailored AI solutions that reflect each organization’s distinctive data expertise, brand voice, and business needs.

As enterprises move beyond basic proof-of-concept projects and foundational chatbots, there’s an increasing shift towards sophisticated personalization and optimization strategies. These advancements go well beyond simple prompt engineering or retrieval-augmented generation (RAG). The focus now extends to creating specialized models for precise tasks, refining larger models into more efficient versions, applying mid-training modifications, and optimizing hardware and accelerators to boost throughput while reducing costs.

Strategic Investment Yields Results

Partnerships established through this program have highlighted how strategic investment at the outset can deliver long-term benefits throughout a model’s life cycle. A prime example is Cosine AI, developers of an AI platform and software engineering agent integrated within user workflows. The company collaborated with the innovation center to customize Nova Pro, an Amazon Nova foundation model, leveraging Amazon SageMaker AI for their AI assistant, Genie. The results were groundbreaking: a five-fold increase in A/B testing capability, rapid ten-fold improvements in developer iterations, and a four-fold acceleration in overall project timelines.

Transitioning toward agentic systems requires very low latency with task specificity, emphasizing the importance of performance metrics and depth across complex processes.

Five Tips for Maximum Value in Customization

To maximize ROI from training and tuning generative AI models, the Innovation Center shares pivotal strategies that leaders can adopt:

1. Reverse Engineering from Business Goals

It’s essential to approach projects backward from tangible business goals instead of leading with a technical mindset. After engaging with over a thousand customers, the Innovation Center has identified that those who align their technical solutions with defined business outcomes are more successful, achieving a remarkable 65% production success rate, with some projects launching within just 45 days. Many organizations mistakenly dive into technical details, forgetting the critical step of defining downstream use cases and evaluation metrics. By framing discussions in terms of clear business objectives, organizations ensure that their technical decisions resonate with strategic goals.

2. Choosing the Right Customization Approach

Organizations should first assess what they’ve already tried as a baseline customization approach. This often includes leveraging lighter strategies, like prompt engineering and RAG, before delving into more deep-rooted techniques. While advanced model optimization can deliver enhanced performance, it’s prudent to exhaust simpler solutions before resorting to more complex adjustments.

Customization techniques include:

• Supervised Fine-Tuning: Shirks the focus on specific use cases. A notable instance comes from Volkswagen, who improved their brand-consistency check accuracy.
• Model Efficiency Tuning: Improves speed and performance, demonstrated by Robin AI in legal contract technologies.
• Reinforcement Learning: Aligns models with organizational preferences.
• Continued Pre-Training: Athena RC’s development of Greek-first models illustrates the power of local contextual understanding.
• Domain-Specific Model Development: TGS’s enhanced Seismic Foundation Models showcase targeted applications that address specialized industry challenges.

It’s crucial to prioritize high-quality data, as this becomes a cornerstone for successful model customization.

3. Defining Success Metrics

Establishing clear measures for success is vital. This includes not just technical performance but overall business outcomes derived from the initiative. Depending on the project, teams may prioritize different metrics such as relevance, clarity, and cost-efficiency. For instance, Volkswagen developed a model that improved brand compliance—illustrating how aligning technological evaluation with real-world business priorities aids ongoing improvements.

For qualitative metrics, automated evaluations should align closely with expert human assessments, offering the dual insight to refine models effectively.

4. Hardware-Level Optimizations

Organizations utilizing managed services like Amazon Bedrock benefit from optimized frameworks, yet more customized solutions require a keen focus on hardware efficiencies. TGS’s seismic data processing operation achieved tremendous efficiency by leveraging high-performance GPU infrastructures on AWS, which, in turn, allowed them to provide actionable insights much more rapidly.

Investments in infrastructure lead to reduced operational costs, especially concerning inference demands. Strategic optimization can unlock impressive gains in processing speed and efficiency.

5. Emphasizing Model Diversity

There’s no universal solution when it comes to model size and family. With so many models available that excel in specific tasks, it’s essential to select those that fit the current requirements, rather than simply opting for the largest or most complex models. Not all applications need extensive capabilities; some benefit more from simplified, specialized resources.

In complex agentic applications, a blend of lightweight models for specific tasks with robust models for oversight can yield superior results. Architecting a modular solution prepares organizations for the rapid evolution of model technologies.

Support from the Innovation Center

The Custom Model Program at the Innovation Center is committed to guiding businesses through the entire process—from selecting the right models to fine-tuning and integrating them into existing workflows. The initiative ensures that the training and tuning process aligns with unique customer needs, leading to significant performance boosts, faster time to market, and better value realization.

In closing, for organizations eager to explore how AWS’s Innovation Center can catalyze their AI initiatives, reaching out to an account manager or visiting the AWS Village at re:Invent is just the beginning.

About the Authors

• Sri Elaprolu: Director of the AWS Generative AI Innovation Center, with a wealth of experience in driving AI solutions for complex business needs.
• Hannah Marlowe: Leads the Model Customization and Optimization program, focused on delivering tailored generative AI solutions.
• Rohit Thekkanal: Manages ML Engineering efforts, emphasizing scalable generative AI applications.
• Alexandra Fedorova: Focuses on growth and strategy within the Model Customization program.

These leaders not only bring their technical expertise but also a clear vision of how to leverage AI for transformative business results.