Monitoring your performance in the vastness of highly scalable databases. SolarWinds DPM introduces performance monitoring for Vitess.
Understanding the need to scale is certainly a topic that is not new. Scaling databases, not new. Scaling efficiently is and always has been a very challenging subject. For many "legacy" environments throwing hardware at the problem and building bigger and bigger monolithic databases has been the “go-to” solution for decades at this point. However, with the ever greater shift to using open source databases came the notion of distributed systems, sharding and building effective database clusters that behave like a single unit.
At the bleeding edge of this has been Vitess, a database clustering system (originating at YouTube) for horizontal scaling of MySQL. Vitess also is a CNCF graduated project. Read more about Vitess (www.vitess.io) or the Cloud Native Computing Foundation (www.cncf.io).
But scaling the DBMS alone isn’t all that scaling is. The need to be able to effectively monitor performance on a query level is still important and maybe even more so than in a more traditional database instance based environment.
Vitess consists of three main components:
- A stateless proxy where applications connect, called VTgate .
- A manager for MySQL called VTtablet (note: Vitess still stores data in MySQL )
- A metadata store (also known as topology).
For the sake of an example let’s use VTgate. SolarWinds DPM does however also provides support for VTtablet.
Assuming we want to ensure VTgate is performing and want insight into queries executed by VTgate to the individual tablets, SolarWinds DPM allows for Vitess specific charts and metrics
Now that we have an overview of VTgate connections, the next step is to get query visibility directly at VTgate as distributed datastores are more complex with an expected increase in latency.
Being able to measure exactly what that latency increase is, will help evaluate whether the benefits of scaling outweigh the costs of slower performance.
So how to setup Vitess support in SolarWinds DPM?
SolarWinds DPM is able to collect VTgate queries in the same way that we collect queries from MySQL: either through tcp packet sniffing (on-host) or collecting queries from Vitess' HTTP port (off-host).
Note: SolarWinds DPM does not need VTgate credentials for the information we gather, since the metrics collected come from VTgate’s HTTP port. This also means that vc-vitess-metrics agent must be able to reach VTgate’s HTTP port.
You should choose off-host install method in the following situations:
- You install the agent on a separate host than the VTgate process, in which tcp sniffing can't be used.
- Your applications connect to VTgate with encryption.
- Your workload utilizes the gRPC protocol.
On-host:
- Installation is similar to on-host MySQL. Follow our installation documentation and choose “On-Host”.
- For automated installation, an example global.conf might look like:
{
"use-drv-vitess":"true",
"use-drv-mysql":"false",
# other typical options
}
Off-host:
- For off-host VTgate , follow our installation documentation for off-host deployments of MySQL.
- During installation, you must supply the http connection string, such as http://localhost:15001 which is
VTgate’s default HTTP port
○ For automated installation, an example global.conf might look like:
{
"use-drv-vitess":"false",
"use-drv-mysql":"false",
"drv-manual-host-uri": "vitess://127.0.0.1:15306?binding-http=127.0.0.1:15001",
"drv-manual-query-capture":"query-log"
# other typical options
}
Note: ports in the above example assume default VTgate configuration. Please update according to your environment.