Object storage is an approach for storing unstructured data in the cloud that, in theory, makes it possible to store practically limitless amounts of data in its native format. In this storage architecture, data is managed as objects, compared to traditional systems that handle data as blocks or hierarchical files. Object storage is available on-premise, but is considered a good fit for the cloud because it is flexible and can more easily scale. Use cases include Backup & Recovery, Data Archiving & Compliance, Big Data Analytics, and Cloud-native Application Data.
For availability and scale, client data is often automatically distributed across three or more physical facilities that are geographically separated. This can be done regionally, across the country, or across the globe. For distributed storage environments like this, the performance of the network is critical to the end user experience.
The Challenge: Most organizations cannot fully assess how their system will scale and perform over wide area networks.
Because they can’t easily replicate distance, or the imperfections real networks experience, organizations don’t know how a specific object storage system will hold up under peak loads nor the impact adding additional workloads will have on performance. This lack of knowledge leads most organizations to either under or over-provision for bandwidth which can lead to costly fixes or wasted resources. The ability to pinpoint the exact bandwidth requirements saves organizations from overspending.
Before migrating to cloud object storage, organizations should start stress testing cloud applications and object storage workloads against different service providers. Providers use varying technology and offer different architectures which could impact performance and overall costs. Multi-tenant (public) architecture for example, may affect latency, throughput, or loss differently than single-tenant (private) architecture. Using network emulators to test the performance of different providers can help to decide which is a better fit for your organization.
It is important to test cloud object storage behavior over unreliable and challenging “real-world” networks during the entire development & deployment lifecycle.
For businesses considering cloud object storage, performance is the primary concern. According to industry analyst Stratecast “end-to-end application performance assurances” are the highest priority when choosing a storage provider. And fear of “poor or inconsistent application performance” was a common deterrent from moving to the cloud. This means it is paramount that object storage vendors verify system level scalability and performance under real-world conditions before customer deployments. It’s common to use a cloud architecture for object storage. Data is typically geo dispersed into multiple datacenters: East coast, Central, and West coast. Sometimes international locations are also part of the system level testing. Using 1G and 10G Ethernet links are typical, with 25/40/100G becoming more common.
Today there are several solutions that allow organizations to test application performance in the lab – as if it’s running over a wide area network. Network emulators and load testing platforms enable organizations to test the performance of a wide range of object types over a variety of network conditions. To do this they create real-world network conditions like bandwidth limits, excessive latency/jitter, packet loss, and other impairments. By intelligently mimicking network conditions you can ensure the solutions you are designing and deploying work as expected when rolled out to production.
Understanding how the object storage architecture is responding to application requests under loads from multiple applications is key to delivering predictable, consistent performance to users. A major element of the test must include emulating the network the distributed storage system will run on, considering distance, latency, loss, and other factors.
Let’s have a look at how several leading object storage vendors are approaching these challenges in their system integration test labs.
Network emulation appliances are an integral part of proving a solution before deploying. These appliances can control multiple site and network characteristics seamlessly and accurately. Combining client workload emulation tools like Virtual Instruments (virtualinstruments.com) and network emulation appliances gives storage system vendors and enterprises a fast path to object storage proof of concept deployments – at real world scale.
When your application depends on quick, reliable access to stored data, you need to be certain your storage system performs as expected.
