TLDR
Data is the fuel AI needs, but unchecked storage costs can quickly consume your AI budget. This isn’t about cutting corners, it’s about aligning storage choices with actual business needs by using data-driven decision making. Here’s how to keep AI workloads running efficiently without overspending on infrastructure.
The explosion of data sprawl and misaligned storage costs often derails AI efforts. Organisations risk overspending on storage, stifling innovation by diverting budgets from experimentation to infrastructure. Sustainable AI growth hinges on treating storage not as an afterthought, but as a core pillar of your data strategy.
At V2 AI, we recently helped an organisation match their storage modality to meet growing data needs. By optimising how and where data was stored, they realised a $500K reduction in their annual storage spend of $1.2M without adversely impacting operations.
The organisation’s innovation ability was tied to a finite budget, limiting AI experimentation capabilities. Working with V2 AI helped them achieve nearly 50% cost reduction in their original spend and redivert funds to AI innovation.
This article walks you through some guidelines we use at V2 AI to ensure your cloud storage is fit for purpose and can cost-effectively support your data and AI needs.
Role of Data Storage in AI Cost Optimisation
Over the past few years, at V2 AI, we have helped many organisations drive value with AI, from proof of concepts to productionised workloads. However, it is hard to justify spending if the cost of running AI workloads is going to be higher than the value they bring. But let's be honest: there is no straightforward path to AI cost optimisation.
Some AI projects can be implemented by feeding detailed prompts to existing foundation models like Gemini and ChatGPT via APIs. While the API cost for AI models has significantly reduced in the past year, our customers still find it expensive to customise existing models with internal data.
The costs of model customisation
Most complex enterprise use cases require customising an existing AI model with internal data. For example, if you want an intelligent chatbot for customer support, you need to feed the AI model with data about your troubleshooting workflows.
At V2 AI, we have avoided more expensive model customisation strategies like fine-tuning or training a new foundation model from scratch. Instead, we have set up Retrieval Augmentation Generation (RAG) workflows that pull data from the customer’s knowledge sources to augment the model prompt. RAG workflows are typically complemented by vector databases like Cockroach, Pinecone, or Elasticsearch.
A vector database is a specialised database engine designed to store, index, and query high-dimensional numerical representations (vectors) of data. One would typically calculate vectors on embeddings. These vectors represent data points in a multi-dimensional space, commonly used in machine learning and other analytical applications. They are handy for tasks like similarity search and finding relevant information based on semantic meaning.
For example, if you have a large image collection, you could use a vector database to find images visually similar to a given query image. This can be used in applications like image search on ecommerce sites. In AI, vector databases help to quickly retrieve information related to a particular user query from an internal knowledge source and feed that information to the AI model in a way that it can understand.
Vector databases require high input/output (I/O) operations and add to the costs of running AI workloads. However, we find that the most significant cost challenge in most large organisations is data sprawl. With so much data on hand (and growing), it is the cost of storage that is reducing the viability of AI use cases.
Organisations must match their data to the appropriate storage type and tiers to scale and embrace AI. Doing so increases the viability of AI prototyping and provides a cost-effective path to production. You can also run more experimental AI projects for the same dollar value.
Understanding Cloud Storage Options for Your Data
For your data and AI strategy to be viable, you first need to look beyond structured and unstructured data classification to your underlying cloud storage service configurations and types.
Broadly speaking, there are two main ways of storing data in the cloud.
Object Storage
Object storage sections any data type into units called objects and stores them in a structurally flat data environment. Each object includes the data, associated metadata, and a unique identifier that applications can use for easy access and retrieval. Typical access is over HTTP; examples of object storage are Amazon S3, Azure Blob Storage, and Google Cloud Storage.
Block Storage
Block storage controls data storage and storage devices. It takes any data, like a file or database entry, and divides it into blocks of equal size. It stores the data block on underlying physical storage in a manner optimised for fast access and retrieval. Access to block devices is as drive letters in Windows (C:, etc.) or /dev/sdx in Linux. Block storage examples include Amazon EBS, Azure Managed Disks, and Google Block Storage.
Different resiliency, latency, and performance levels impact costs within each storage type. Block storage is typically more expensive than object storage. Choosing incorrect storage configurations and pricing tiers can escalate costs quickly, especially as data scales.
Too often, organisations default to storing data on high-performance storage tiers like Azure Ultra Disks, AWS EBS IO1 volumes, or similar premium options without asking a simple question: Does this data need premium performance?
The Storage Optimisation Process
At V2 AI, we use a threefold approach to cloud storage optimisation.
Business alignment
Data-driven insights, and
Application-aware automation
Having these available will help you architect your storage cost-effectively.
Business Alignment
Does your business have disaster recovery and business continuity requirements? Do you need ongoing data duplication for backups? What are your data security and confidentiality requirements? How long do you plan to retain a particular data set?
Such business requirements often drive storage solutions. For example, if your downtime requirements are less stringent, you can use less expensive storage options for your AI projects. If you can recreate your data from Amazon S3 for your AI model consumption, you can use AWS EBS GP2, with a lower SLA instead of the more expensive AWS EBS IO1.
Data-Driven Insights
At V2 AI, everything we do is grounded in data, and telemetry helps you make data-driven decisions around storage optimisation. Telemetry data is information about your IT systems, applications, and how they operate. Telemetry services like Amazon CloudWatch, Azure Monitor, and GCP Cloud Monitoring allow you to analyse and align actual usage of your storage solutions against the capacity you are paying for.
The gap between services purchased and what is required is often large. Identifying this gap reduces wastage and presents opportunities for potential savings.
Application-Aware Automation
At V2 AI, we have developed scripts and processes that automatically tailor storage configurations and tiers in a way that is aligned with your current telemetry data. More importantly, storage configurations are realigned in ways that are often transparent to running applications. This means the underlying storage changes do not impact the application performance.
Where possible, our processes are always wrapped in policy to ensure we are not only making changes today but also ensuring they are enforced continuously. Our engineers use cloud native policy and rules enforcement, like Service control policies in AWS Organizations, Azure Policy and Organization Policy Service in Google Cloud.
Implementation examples
All public clouds provide means to change block storage from high to low Input/Output Operations Per Second (IOPS) on the fly.
Below is an AWS CLI command, shifting a volume to less expensive AWS EBS GP2 to drive cost savings.
However, on Azure, shifting from Premium SSD to Standard SSD requires the machine to be offline and the volume detached.
Choosing Storage for AI Workloads
Regarding storage, our approach is driven by policy that ensures the most cost-effective storage platform and modality is selected while meeting all application requirements. Here is a flow chart that allows us to quickly hone in and re-align on the most appropriate storage for AI workloads.
Final Words
At V2 AI, we’ve seen Retrieval Augmented Generation (RAG) and vector databases become game-changers, enabling enterprises to build AI applications with internal data. Such applications are a must-have to drive efficiencies and transform the customer experience.
However, data infrastructure costs can make it too expensive to run AI workloads, limiting AI innovation. At V2 AI, we implement a 3-pillar framework across business alignment, data-driven insights, and application-aware automation to transition storage from a cost centre into an innovation accelerator.
Ready to embrace AI in your organisation? Explore a technical deep dive where we recently re-aligned a customer's storage landscape and contact V2 AI to craft a storage strategy that scales with your AI needs.
