Navigating the Wave: Generative AI in Financial Services

Author: Andy Zhong

In November 2022, the launch of ChatGPT marked a significant shift in the AI landscape, democratizing a domain primarily understood by specialists and making it accessible to the broader public through a conversational interface. This evolution in Generative AI (Gen AI) sparked unprecedented interest among consumers and across industries.

The tech sector, known for being at the forefront of innovation, eagerly embraced Gen AI. Major tech companies scrambled to develop their own versions of OpenAI’s groundbreaking GPT model. This enthusiasm also extended to the financial services industry. Despite the traditionally cautious approach many of the industry’s leading players have had with regard to bleeding edge new technologies, many institutions have begun to explore and implement solutions involving Gen AI to streamline processes, enhance customer engagement, and improve decision-making.

Now a year and a half after ChatGPT’s debut, an increasing number of financial institutions (FIs), from fintech startups to established banks, are unveiling their Gen AI applications. One of the most direct use cases is in customer support, and there are several notable examples illustrating this trend.

2OS reviewed and discussed current state AI in a March 2024 white paper, as well as hot topics including nuances in Large Language Models (LLMs), the intersection of automated decisioning and AI, and potential risks in transaction servicing. Our previous white paper deep dives into specific topics in AI — we focus in this blog post instead on measurable and observable impacts of Gen AI for FIs, and key issues that should be top-of-mind for FIs.

Gen AI Applications in Financial Institutions

Klarna’s Virtual Assistant

Klarna, a “buy now, pay later” financial service provider, partnered with OpenAI to release its 24/7, personalized, and multilingual Gen AI-powered virtual assistant. Klarna’s assistant is fully AI-powered, aimed at saving time and resources. According to published numbers, in its first month of release, the assistant handled 2/3 of its customer service chats with faster and more accurate resolution while maintaining a similar customer satisfaction score. Overall, it’s estimated to drive a $40 million in profit improvement for Klarna in 2024. This is one of the largest Gen AI application deployments so far, and initial results seem promising.

Wells Fargo’s Fargo

In 2023, Wells Fargo launched their virtual assistant, Fargo. Designed to simplify everyday banking tasks such as paying bills, transferring money, and providing transaction details, Fargo operates via voice or text commands on smartphones. Initially built on Google Cloud AI with PaLM 2 as its core LLM, Fargo has since evolved to incorporate multiple LLMs, each tailored to specific tasks. By early 2024, Fargo has managed over 20 million interactions, with the potential, as noted by CIO Chintan Mehtan, to handle upwards of 100 million interactions annually.

Behind these celebrated virtual assistants, the critical infrastructure and scaffolding remain less discussed. For example, to support Fargo and their future Gen AI applications, Wells Fargo developed a platform called Tachyon. Details about it remain scant, as FIs typically conceal such technical information for competitive and security reasons. What is known about Tachyon is that it is designed not to depend on a single cloud service provider or data model, making it robust and versatile. The platform supports model and tensor sharding [see also Xu et al., 2021], which optimize memory and computation demands, suggesting a high degree of technical sophistication aimed at scalability and resilience in handling AI tasks.

As these examples show, the focus often remains on the immediate functionalities of Gen AI, such as chatbots, so the broader implications and potential of Gen AI can be easily overlooked. While it is still a significant engineering feat to build a successful chatbot, it is crucial to think beyond simple conversation interfaces and develop a sustainable Gen AI strategy from the outset.

Getting Started with Gen AI

Implementing a successful Gen AI strategy within an organization hinges on three core components: model infrastructure, proprietary data, and business initiatives. Success stories in organizations often emerge from projects seamlessly integrating Gen AI models with internal data sources and into existing business processes.

Model Infrastructure

Today’s Gen AI models require significantly different architecture and support than the traditional ML models used by modern FIs. Due to their complexity and cost, these models are often prohibitive for individual developers or small businesses to train from scratch. Instead, they are typically accessed as pre-trained, off-the-shelf models. Here’s an overview of the three primary types of foundation models available, along with popular examples:

Proprietary models

• Examples: GPT-3.5/4 from OpenAI, Claude 3 Opus/Sonnet/Haiku from Anthropic
• Characteristics: These are the simplest to deploy, often just requiring a few lines of code to interact with through API endpoints. The ease of use and usually higher performance make them appealing for many applications, though they offer limited flexibility since customization options are restricted to what the provider allows. Cost-wise, these models typically have a straightforward pricing model that scales linearly with the volume of tokens processed.
• Cost: $

Open-weight models

• Examples: Llama 3 from Meta, Command R+ from Cohere
• Characteristics: Unlike proprietary models, open-weight models provide access to their underlying weights and architecture, allowing for customization through fine-tuning for specific tasks or knowledge domains. Outside of some popular base models, which have API endpoints through major cloud platforms, these models generally require hosting. The trade-off here is higher initial setup costs and ongoing expenses related to server uptime rather than per-token costs. While historically these models lagged behind proprietary models in performance, recent advancements indicate that they are rapidly improving, as shown in Figure 1 below.
• Cost: $$

Custom models

• Examples: OpenAI custom models, BloombergGPT (see Wu et al., 2023)
• Characteristics: Custom models offer the highest degree of control and customization, allowing organizations to tailor everything from the model’s architecture to its training data. However, this route requires substantial investments in terms of capital, time, and expertise. The risks are also higher, as evidenced by BloombergGPT’s performance issues despite significant resource allocation. Although this approach isn’t recommended for most, it may be suitable for organizations with specific needs that cannot be met by other model types.
• Cost: $$$

Figure 1: Language model system (LMSYS) rankings and key takeaways (Source: 2OS)

Proprietary models are ideal for those seeking a quick and easy setup with minimal technical overhead. Open-weight models are better suited for entities that need more control over their AI applications and can manage the associated complexities and costs. Custom models are reserved for well-resourced organizations that require the utmost customization and have the capacity to handle the inherent risks and challenges.

Proprietary Data

To understand the data imperative, a popular adage states that a model is only as good as the data it’s trained on. Currently, many companies developing the next generation models are running out of quality data. There are concerns that “the industry’s need for high-quality text data could outstrip supply within two years, potentially slowing AI’s development.” Strategic partnerships, such as the ones OpenAI set up with Associated Press, Reddit, and News Corp for data access, underscore this growing scarcity.

However, relying on general data pools alone may not meet the needs of organizations developing specialized Gen AI applications. It is becoming a consensus that pre-training data is not the sole factor in improving model performance. Therefore, the scarcity of and the diminishing returns on general data pools highlight the increasingly urgent need to leverage private data. In the end, without specific knowledge tailored to business needs, possessing a more sophisticated model isn’t inherently more useful.

Consider the online shopping companion Mastercard designed to assist retailers and their customers with product recommendations and personalized shopping experience. This AI-powered tool would not be effective in lifting revenue or increasing customer loyalty if it did not have access to the data that customers opted in. By strategically leveraging their proprietary data, while ensuring data privacy and security, organizations can create applications that deliver context-aware experiences, ultimately driving business value.

Enterprises that have prioritized their data strategy are already ahead of the curve. Private, enterprise-specific data will become a critical asset in the AI landscape. For organizations with targeted applications in mind, their proprietary data becomes exponentially valuable — from training custom models enriched with domain-specific knowledge to fine-tuning smaller models for particular workflows, or even utilizing this data as few-shot prompts when querying models. Having access to and effectively using this data will be a significant competitive advantage in the world of AI.

Business Initiatives

Lastly, business initiatives are the glue that unify the other elements. Notice that in the examples discussed earlier, success was not measured by the technical benchmarks of the underlying LLMs but rather on the tangible business outcomes. Building a robust and well-managed AI-driven service goes beyond engineering feats. With established data practices and model infrastructure, organizations are well-positioned to weave Gen AI tools into their business initiatives. Effective deployment of Gen AI applications often follows a common governance framework.

The journey typically begins with the translation of strategic priorities into a detailed, actionable roadmap. This roadmap outlines specific projects designed to bolster the broader organizational goals. During the product design phase, it is essential for teams to integrate Gen AI into the core of the design process, rather than inserting it as an afterthought. Incorporating insights from various disciplines during the review of prototypes can also help enrich the development process with diverse perspectives.

Following design, a rigorous assessment phase is critical. Organizations take this time to thoroughly evaluate and address potential risks associated with the product, focusing on legal, privacy, and security concerns. This phase includes reporting findings to stakeholders for approval and sign-off. Additionally, defining and agreeing on metrics to measure the product’s impact and the project’s overall success is crucial for aligning with business objectives.

As the product moves toward production, a soft, incremental launch allows for iterative refinement based on performance data and user feedback. For projects involving Generation AI, it’s crucial to steer clear of products that lack tangible business value, relying solely on novelty for appeal. This is a key distinction between fruitful Gen AI initiatives and those that falter.

What’s Next?

Reflecting on the journey of Gen AI within the financial sector, it’s clear that embracing this technology is not just about adopting new tools, but also about redefining how organizations operate and engage with their customers. The examples of Fargo and Klarna’s virtual assistants demonstrate more than technological prowess; they reveal a strategic shift toward more agile, efficient, and customer-focused business models. FIs that successfully integrate Gen AI are setting new standards in customer interaction, operational efficiency, and competitive advantage.

Moving forward, the challenge for businesses will be to balance innovation with responsibility. The integration of Gen AI into core business processes must be done with a keen eye on ethical considerations, data security, and the potential impacts on employment and customer relations. The ongoing development and refinement of Gen AI applications will likely continue to revolutionize industries, but its true success will be measured by how well it aligns with and supports the broader goals of the organizations that deploy it.

At 2OS, we are helping several FIs leverage these AI tools effectively and responsibly. Please reach out if you have any questions or would like to learn more about our AI capabilities.

Additional Reading

• Li, Xianzhi, et al. “Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks.” arXiv:2305.05862 (2023).
• McGuire, Aaron, and Syed Raza. “A quick overview of modern AI for lenders.” (2024).
• Wu, Shijie, et al. “BloomberGPT: A Large Language Model for Finance.” arXiv:2303.17564 (2023).
• Xu, Yuanzhong, et al. “GSPMD: general and scalable parallelization for ML computation graphs.” arXiv preprint arXiv:2105.04663 (2021).

Do you have questions? We’d love to answer them!

Reach out to us: https://2os.com/contact/. You can also contact the author by email at:

• Andy Zhong (Andy.Zhong@2os.com)

Interested in 2OS insights?

Check out the 2OS Insights page, where 2OS has shared industry insights through white papers, case studies, and more!