At a Glance:
Project Metrics
| Period: | September 2014 - present |
| Budget: | six-figure |
| Team: | 2 developers, 1 project manager, 1 DevOps |
| Area of Application: | Ad-Marketing & High-Availability Systems |
| Industry: | Online Advertising and Marketing |
Technologies
In today's data-driven business world, the ability to process large amounts of data in real-time is a decisive competitive advantage. SlimSpots, a globally operating provider of ad-marketing solutions, brings traffic suppliers and advertisers together and requires an infrastructure that not only can handle enormous data volumes but also meets the highest requirements for scalability, availability, and speed.
Our task was to develop and maintain a robust, scalable infrastructure that can make decisions about advertising delivery in milliseconds while simultaneously providing trillions of data sets for detailed analysis. SlimSpots decides for every click and banner view which advertising is displayed – based on criteria such as browser, device, country, internet provider, time, previous interactions, and many other factors. At the same time, traffic suppliers and advertisers must be able to track the performance of their campaigns in real-time to react quickly to changes.
Our Services
Database and Server Administration
For SlimSpots, we implemented a comprehensive database and server administration system designed for maximum availability and performance. At the center of our solution is the use of MariaDB ColumnStore, a column-based database specifically designed for processing enormous data volumes. This technology makes it possible to distribute queries across multiple servers and all available CPU cores, ensuring exceptional scalability.
The implementation of a column-based database brings specific challenges that we mastered through careful optimization of the database structure, queries, and software. We designed the data models to be optimally suited for analytical queries while ensuring real-time processing. This balance is crucial for SlimSpots, as the system must both make lightning-fast decisions and enable comprehensive historical data analysis.
In addition to database optimization, we also took over the management of the entire server infrastructure, which at its peak consisted of 40 servers. This included the configuration, monitoring, and maintenance of the servers as well as the implementation of high-availability solutions to minimize downtime. Through continuous performance analysis and proactive optimizations, we were able to continuously adapt the infrastructure to growing requirements.
Optimization of Delivery Software
An essential part of our work for SlimSpots was improving the delivery software responsible for real-time decisions in advertising placement. This software must decide in milliseconds which advertising is shown to a specific user, based on numerous criteria such as device, location, browser, and previous interactions.
We fundamentally revised and optimized the existing software to increase processing speed while improving the precision of decision-making. Through the use of efficient caching mechanisms, optimization of database queries, and implementation of intelligent algorithms, we were able to significantly reduce response times and increase system scalability.
We placed special emphasis on real-time analysis and visualization of campaign performance. Advertisers and traffic suppliers need immediate feedback on the effectiveness of their measures. Our optimized software enables real-time data processing and generation of meaningful dashboards that allow immediate evaluation of campaign performance.
Special Challenges
Extreme Scalability with Constant Performance
One of the biggest challenges in this project was ensuring extreme scalability while maintaining constant performance. The data volumes that SlimSpots processes are continuously growing, and the infrastructure must be able to keep pace with this growth without performance degradation.
We mastered this challenge through the use of MariaDB ColumnStore, a column-based database specifically developed for big data analytics. We carefully coordinated the database structure, queries, and application logic to enable both lightning-fast decisions and comprehensive historical analysis.
Scaling to up to 40 servers also required a well-thought-out architectural concept that includes efficient load distribution and fault tolerance mechanisms. Through the use of intelligent caching mechanisms and optimization of index structures, we were able to ensure that the system functions reliably even with sudden spikes in query volumes.
Global Availability and Data Analysis
As a globally operating company, SlimSpots needs an infrastructure that is globally available yet provides low latency times. At the same time, data must be stored and analyzable for years to identify long-term trends and evaluate campaign effectiveness.
We implemented a distributed system that combines local processing with global data consistency. The challenge was to create a consistent data foundation for analysis despite global distribution.
Particularly demanding was the requirement to keep historical data available and efficiently queryable over years. Our system architecture enables SlimSpots and its customers to perform detailed evaluations over long periods, gaining valuable insights for optimizing their campaigns.