The ML test score: A rubric for ML production readiness and technical debt reduction

E Breck, S Cai, E Nielsen, M Salib… - 2017 IEEE international …, 2017‏ - ieeexplore.ieee.org
Creating reliable, production-level machine learning systems brings on a host of concerns
not found in small toy examples or even large offline research experiments. Testing and …

[PDF][PDF] A reliable effective terascale linear learning system

A Agarwal, O Chapelle, M Dudík, J Langford - The Journal of Machine …, 2014‏ - jmlr.org
We present a system and a set of techniques for learning linear predictors with convex
losses on terascale data sets, with trillions of features, 1 billions of training examples and …

In defense of minhash over simhash

A Shrivastava, P Li - Artificial Intelligence and Statistics, 2014‏ - proceedings.mlr.press
MinHash and SimHash are the two widely adopted Locality Sensitive Hashing (LSH)
algorithms for large-scale data processing applications. Deciding which LSH to use for a …

Combining predictive models in predictive analytical modeling

WH Lin, TH Green, R Kaplow, G Fu… - US Patent 8,370,280, 2013‏ - Google Patents
(57) ABSTRACT A method can include the actions of receiving a feature vector, the feature
vector including one or more elements; identifying an element type for each of the one or …

Dynamic predictive modeling platform

JM Breckenridge, T Green, R Kaplow, WH Lin… - US Patent …, 2013‏ - Google Patents
5,271,088 A 12, 1993 Bahler 5,583,968 A 12/1996 Trompf 5,586.221 A 12/1996 Isik et al.
5,727,128 A 3, 1998 Morrison 5,752,007 A 5, 1998 Morrison 5,862,513 A 1/1999 Mezzatesta …

Optimal densification for fast and accurate minwise hashing

A Shrivastava - International Conference on Machine …, 2017‏ - proceedings.mlr.press
Minwise hashing is a fundamental and one of the most successful hashing algorithm in the
literature. Recent advances based on the idea of densification (Shrivastava\& Li, 2014) have …

Predictive analytical model matching

WH Lin, THK Green, R Kaplow, G Fu… - US Patent …, 2012‏ - Google Patents
(*) Notice: Subject to any disclaimer, the term of this OTHER PUBLICATIONS patent is
extended or adjusted under 35 Duchi, John, et al.,“Boosting with Structural Sparsity', 2009 …

Updateable predictive analytical modeling

JM Breckenridge, T Green, R Kaplow, WH Lin… - US Patent …, 2013‏ - Google Patents
Methods, systems, and apparatus, including computer pro grams encoded on one or more
computer storage devices, for training and retraining predictive models. A series of training …

[PDF][PDF] What's your ML test score? A rubric for ML production systems

E Breck, S Cai, E Nielsen, M Salib, D Sculley - 2016‏ - research.google.com
Using machine learning in real-world production systems is complicated by a host of issues
not found in small toy examples or even large offline research experiments. Testing and …

Hosting predictive models

WH Lin, TH Green, R Kaplow, G Fu… - US Patent 8,364,613, 2013‏ - Google Patents
US PATENT DOCUMENTS 5,271,088 A 12, 1993 Bahler 6,243,696 B1 6, 2001 Keeler et al.
6,778,959 B1 8, 2004 Wu et al. 6,879,971 B1 4/2005 Keeler et al. 6,920,458 B1 7/2005 Chu …