SolarWinds has tremendously improved our ability to proactively monitor our infrastructure, an infrastructure whose stability has a direct and critical impact on the development of Africa’s financial ecosystem. Baseline monitoring no longer becomes enough; intelligent and instant root cause analysis become imperative to quick fault resolution, and with SolarWinds, that is what we have achieved—an incident resolution turnaround time that is magnificent.
—Oche Eluma, Network Infrastructure Lead
About Appzone
Based in Lagos, Nigeria, the Appzone Group develops financial transaction software for democratizing and decentralizing the movement of funds across Africa. After launching in 2008, Appzone began by providing proprietary core banking solutions leveraging blockchain technology to digitize payments across all currencies and African nations, significantly modernizing and improving the efficiencies of the sub-Saharan financial industry.
Today, approximately 95% of microfinance institutions in Nigeria—and a cross- section of banks in a majority of the 54 nation-states in Africa—rely on Appzone for their core transactional operations.
Challenge
In building out one of the first fintech startups in Africa, co-founders Emeka Emetarom, Obi Emetarom, and Wale Onawunmi decided to mitigate the cost of purchasing on-premises hardware by standardizing on a hybrid cloud infrastructure. Over time, Appzone’s public cloud footprint has expanded to account for roughly 80% of their environment, with the remainder on-premises and in a private cloud.
When network infrastructure lead Oche Eluma joined Appzone in 2021, the organization struggled with the minute-by-minute monitoring of the distributed applications, databases, and microservices capturing and recording the vast number of transactions running on Appzone’s blockchain and core banking solutions. Complicating the job of maintaining continuous oversight was the need to access logs of Appzone solutions running in the security-hardened in-house data centers of their banking and processing clients.
As a fintech operating a network hosting hundreds of millions of instances of payment and settlement events on the blockchain and our proprietary core banking solutions, we needed instantaneous visibility of the performance and health of our operations with zero tolerance for service interruptions.
—Oche Eluma, Network Infrastructure Lead
Having previously managed network and server infrastructure with several monitoring solutions at a publicly traded commercial bank, Eluma and his infrastructure engineering colleagues suggested Appzone leadership begin a pilot deployment of SolarWinds©Network Performance Monitor (NPM).
Looking back at his exposure to his previous employer’s culture of innovation and technical best practices, Eluma recalls it was the ideal proving ground for the infrastructure team to hone their technical skills for the always-on, fault-tolerant banking culture.
Upon joining Appzone, he was familiar with and impressed by Appzone’s strict adherence to the ISO/IEC 27001 information security regulations and PCI DSS requirements. Each of the growing fintech organization’s solutions had been engineered to meet and exceed the guidelines required by international banking and the Central Bank of Nigeria, the nation’s monetary regulator.
To keep up with the brisk pace of adoption of their solutions, the Appzone developers and customer support teams broadened their professional services to include advising clients on a variety of decentralized and blockchain-based processing strategies. Onboarding and deployment discussions routinely centered on fraud prevention and cybersecurity precautions. However, new challenges emerged with an increasing percentage of the assets of commercial banks, microfinance banks, and processors transitioning to the cloud.
“Practically all the businesses in Nigeria, especially the big banks, were installing our software as a service (SaaS) solutions on in-house data centers,” Eluma adds. “However, we weren’t really sure how this would play out in the cloud—how we would integrate the installations here in Africa with our solutions hosted in distributed cloud platforms around the world.”
As a software vendor bringing innovations in a tightly regulated industry with powerful incumbent entities (many with entrenched alliances in Nigeria’s financial establishment), Appzone’s management and investors are compelled to maintain staunch vigilance of failure avoidance and failover discipline. These low tolerances for performance interruptions are particularly vital in their partnerships with other solutions providers.“
We have a tremendous volume of integrations with third-party solutions and services, some of which are the source of downtimes we were determined to reduce,” he says. “The solution we were using for monitoring wasn’t efficient enough, resulting in outages we were sometimes unaware of until a customer contacted us. We needed to be proactive and anticipate problems before issues came up.”
Appzone’s infrastructure team of eight engineers is constantly on guard for performance degradation and outages. In an industry where mere minutes of service interruption can prevent hundreds of incomplete transactions from processing and cause millions in lost revenue, it’s critical to isolate the precise source of problems and expedite their resolution.
Within weeks of taking on the responsibilities of network infrastructure lead, Eluma compiled a list of operational objectives:
- Isolate and analyze issues originating from APIs and customized integrations
- Review real-time and historical incidence metrics and associated downtime
- Reduce alert noise and simplify log metrics and outage reporting
- Achieve intelligent flow logging and application recognition
- Unify the monitoring of cloud-hosted assets on other continents with customer installations and Appzone’s hybrid cloud infrastructure
- Test cyberattack vulnerabilities and find a monitoring solution that wasn’t exposed and could counter these risks
Further complicating the monitoring of mission-critical services was the distributed nature of the software production and hosting environment.
“Much of our application and processing is hosted in Europe and the U.S., yet we’re serving a local market distributed across Africa,” he affirms. “When an anomaly affects a customer which originates in the cloud overseas, there’s an immediate need to know if it’s in a data center or something that went down in a client site in Nigeria. I don’t want an outage anywhere, and I’m finding out five minutes after the fact.”
In defining the landscape before selecting SolarWinds, Eluma describes a situation where monitoring was simply not optimal and resolution time was frustratingly slow.
“When an alert was triggered, or a call came in, we had to start searching for issues all over the place, and it wasn’t unusual to spend over an hour before we found out what it was and it was resolved,” he comments.
Solution
Once SolarWinds was chosen as a candidate for solving Appzone’s monitoring and network management requirements, testing parameters were established to determine the effectiveness of SolarWinds Network Performance Monitor in meeting Appzone’s stringent security, detection, and diagnostic capabilities.
“There were a lot of questions, a lot of asks, a lot of theoretical work,” Eluma recalls. “We started the process by trying to imitate or deploy servers that could mimic the requirements of what we have on-prem. It was a process, step by step, getting those requirements, installing the software, and proving to management that solution was secure.”
Key to the review protocols was the deployment of VMs with specifications designed to test the ability of NPM to detect potential service degradation and determine the degree to which they could ensure the integrity and availability of the infrastructure. As the tests unfolded, multiple forms of integrations consisting of different networking protocols, devices, and original equipment manufacturers (OEMs) were onboarded and immediately visible through the NPM dashboard. Log data and alert reports were presented to managers to confirm each test case passed.
The backbone of your technology is SNMP, ICMP, and those protocols are what we constantly monitor, so we use SNMP polling and traps, which are essentially the same as every other solution collects. What makes SolarWinds special is your presentation of the information. I want to be able to automatically discover the range of my network in AWS or Azure and move away from onboarding traditional servers or On-Prem VMs to VMs in the Cloud. I wanted something that could pick up SNMP traps without a hassle or interoperability issues.
—Oche Eluma, Network Infrastructure Lead
“I think the testing duration was maybe a month or six weeks or almost two months because we had to integrate the SolarWinds solutions with our cloud as one phase and show we knew how to optimize it whilst still trying to securely integrate our hybrid infrastructure and varying third-party assets,” Eluma says.
Equally persuasive was the flexibility the SolarWinds monitoring solutions offered to adapt to a broad spectrum of heterogeneous environments and to meet Appzone’s customers’ evolving needs.
“Every implementation and customer situation was going to be different, as each had its own complexities,” Eluma states. “We were confident that with the large install base of SolarWinds network management and monitoring solutions, it was doubtful a situation would arise which we couldn’t detect and react to.”
Before deciding to go live with NPM, Eluma said management asked about other users of the SolarWinds solutions.
“Luckily, there are a good number of big installations here in Nigeria,” he remembers.
Benefits
With over a year of NPM implementation behind them, Eluma says Appzone is tallying up mean time to resolution (MTTR) improvements and cost savings stemming from the now-pervasive use of the SolarWinds solution.
Currently, we’re relying on NPM for every aspect of our monitoring, for on-prem, cloud, and our client deployments,” he states. “And it’s made a difference for our customers and us.”
An example of the dramatic shift in the response posture of his team can be seen in an incident at a major colocation data center hosting some of Appzone’s resources.
“We are very proactive now, and here’s a situation where that made a big difference,” he says. “We have a colocation data center in Nigeria, and some months ago, they had a major downtime at about 1:00 a.m. Nigerian time. They were totally down, and that data center is supposed to have NOC monitoring.”
Alerts from NPM instantly notified the Appzone’s infrastructure engineers on call, and an escalation was immediately raised to inform the NOC’s response team of the facility-wide outage. Systems were brought back up quickly, averting a prolonged downtime that could have widely impacted a cross-section of critical services, as the organization provided both collocation and internet service to a couplemajor businesses in the country.
Looking back, Eluma regards his team’s role in preventing a different outcome as yet another benefit of having best-of-breed monitoring in play.
“It’s funny to know that we were the ones that informed the organization that their connection was down, that the whole data center was out, and yet for a couple of minutes on the clock, we were the only ones aware,” he says.
He credits the highly detailed email alert for the quick drill-down into the root cause, which enabled the team to act quickly to resolve the problem.
“When my team is focused on working on our Zone platform or they’re doing other tasks unrelated to network maintenance, once there’s downtime, we receive an email instantly,” he says. “In this case, the other tenants in the data center might not have discovered the issue until they came to work the next day around 8:00.”
Apart from cost savings, Eluma emphasized in the payment processing industry, faster issue resolution and better uptime directly translate to revenue from processing fee volume.
“Our old monitoring solution probably had about the same cost,” he says. “However, we reduced the downtime by over 30%, and when you convert that to money per transaction, SolarWinds has saved Appzone an incalculable amount in reduction of downtime and driving more income.”
As the company grows its customer base, Eluma expects his IT budget to allow for a push into a more extensive SolarWinds portfolio, including SolarWinds NetFlow Traffic Analyzer (NTA). Pilot usage of NTA has already helped shape the planned expansion of Appzone’s global footprint.
“Due to the nature of running millions of financial transactions on blockchain, we have special needs,” Eluma says. “Because we’ve decreased the turnaround time of incidents by over 50%, it only makes sense to continue moving toward NTA and other modules we can turn on as needed.
