Datadog

Abstract The technology provides a consistent hashing approach that can be used with many types of hash functions. This approach, called flip hashing, enables dynamic adjustment of a hash table while satisfying balance and monotonicity requirements. Flip hashing is particularly applicable to database and load rebalancing applications due to its low computational cost and ease of implementation. As computing resources are added to a system, keys are remapped evenly across the newly added resources, e.g., by one or more load-balancing or routing servers. This enables upscaling of the system to minimize hotspot issues. The computational cost for a flip hash approach is effectively constant and regardless of the number of resources. This can provide fast response times to queries and avoid overloading of routing servers.

Abstract The technology involves sampling space-saving set sketches, which relates to tracking and evaluating information associated with large data sets. This can include a number of distributed data streams or other data sets that may come from different sources such as Internet traffic. The sets each have corresponding labels and distinct items. One approach includes performing a query for each distributed data set to determine whether a given label associated with that distributed data set is in a corresponding sketch. When that label is in the corresponding sketch, the distinct item associated with that label is inserted into the corresponding sketch. When the label is not in the corresponding sketch, then depending on whether a number of labels in the sketch is less than a specified size, the approach may add the label to a particular sketch and/or insert the distinct item into the particular sketch according to certain defined criteria.

Abstract A monitoring system is configured for de-duplicating data for storing in a database. The monitoring system is configured for receiving a message including a sequence of fields and corresponding field values; generating an entry in a first mapping table that associates each unique field and each unique field value of the message to a corresponding index value that is unique; generating a first vector including index values; ordering the index values of the vector, the ordered index values indicating a structure of the message; generating an entry in a second mapping table that associates the structure of the message with a structure index value; generating a second vector including the structure index value and an ordered set of index values representing the field values; and storing, in a database, the second vector.