笔记 - Twitter 分布式架构的可观察性

笔记 - Twitter 分布式架构的可观察性

Andy [email protected]
2013/09/20-2013/09/20

原文：Observability at Twitter Monday, September 9, 2013 | By Cory G Watson (@gphat) [19:11 UTC]

As Twitter has moved from a monolithic to a distributed architecture, our scalability has increased dramatically.

• 目录

  • Architecture

  • Collection

    • these metrics are exported by in-JVM stats libraries open-source Twitter-Server framework which provide convenient functions for adding instrumentation.
    • Depending on the level of instrumentation and which internal libraries are used (such as Finagle, which exports rich datasets)
    • For batch-processing and non-numeric values, the data is routed to HDFS using Scribe.
    • Scalding and Pig jobs can be run to produce reports for situations that are not time-sensitive.
    • Determining the network location of applications running in a multi-tenant scheduled environment such as Mesos adds additional complexity to metric collection when compared to ones deployed on statically allocated hosts.

  • Storage

    • Twitter 开发了自己的时间序列数据库，从架构图来看是基于 Cassandra的。另外一个可以考虑的是基于HBase之上的 OpenTSDB，对于已经使用该集群的公司来说更加合适。

  • Query Language

    • 支持query与alert

  • Visualization

    • 支持多种视角的切换：They can toggle between chart types (stacked and/or filled), chart scales (linear or logarithmic), and intervals (per-minute, per-hour, per-day).

  • Monitoring

  • Related Systems

    • Zipkin

  • Future Work

    • 提到机器的开销 About 1.5% of the machines in a data center are used for collection, storage, query processing, visualization, and monitoring (0.3% if storage is excluded).