A Brief History of Machine Learning Platforms
Data and machine learning technologies for the big data era have been developed in the last twenty years. Let’s review these two decades in terms of data processing and machine learning frameworks as well as machine learning platforms that have been spotted in the wild.
While ML platforms such as IBM’s SPSS and SAS have been around for decades, open-source data and machine learning technologies are a recent phenomenon. Modern machine learning (ML) and deep learning frameworks (for Python) have only been around for 10–15 years.
The associated end-to-end machine learning platforms have only popped up at tech companies in the last five years or so. No wonder frameworks and platforms have not yet converged towards a dominant design, especially since we have been able to witness advances in public clouds, containers and orchestration, DevOps in years since 2000. In other words, the entire cloud-native suite has not yet been around that long.
Bold entries in the timeline below are end-to-end machine learning platforms announced by the respective companies. As you can see, the first tech company to come forward with details of their platform was Facebook in 2016, a mere four years ago. Since then, many more have published details of their platforms.
-
OpenCV released (June)
-
IPython released
NLTK released
-
Torch released (October)
-
Hadoop released (April)
-
Amazon Web Services (AWS) launched (March)
-
Theano released
Nvidia CUDA released (June)
scikit-learn released (June)
-
HBase released (March)
Google Cloud Platform (GCP) launched (April)
Cassandra released (July)
-
Mahout released (April)
Alibaba Cloud launched (September)
Avro released (November)
-
Microsoft Azure launched (February)
GCP BigQuery launched (May)
Hive released (October)
-
Kafka released (January)
Mesos released (April)
Flink released (May)
Storm released (September)
-
Luigi released
AWS Redshift launched (October)
-
Caffe released
Docker released (March)
Parquet released (March)
Impala released (April)
Presto released (November)
-
Deeplearning4j released (January)
spaCy released (February)
XGBoost released (March)
Spark released (May)
IBM Cloud ('Bluemix') launched (June)
Kubernetes released (June)
Terraform released (July)
Toree released (July)
-
Jupyter released (February)
Keras released (March)
Airflow released (June)
Singa released (October)
OpenNN released (November)
SystemML released (November)
TensorBoard released (November)
TensorFlow released (November)
-
CNTK released (January)
Arrow released (February)
TensorFlow Serving released (February)
OpenAI Gym released (April)
Facebook's FBLearner Flow announced (May)
MXNet released (May)
Beam released (June)
Twitter's DeepBird announced (June)
LightGBM released (August)
Hudi released (December)
Kudu released (September)
MLeap released (September)
PyTorch released (September)
-
GCP Spanner launched (February)
PySyft released (July)
Google's TFX announced (August)
Horovod released (August)
Featuretools released (September)
ONNX released (September)
Uber's Michelangelo announced (September)
AWS SageMaker launched (November)
Groupon's Flux announced (December)
-
Hopsworks released (February)
Kubeflow released (March)
TensorFlow Lite released (March)
Airbnb's Bighead announced (June)
Comcast's ML platform announced (June)
MLflow released (June)
LinkedIn's Pro-ML announced (October)
Netflix's Metaflow announced (October)
Dask released (October)
RAPIDS released (October)
Flipkart's Hunch announced (December)
-
Hopsworks Feature Store released (January)
Alibi released (February)
FATE released (February)
Feast released (February)
LINE's Rekcurd announced (February)
TensorFlow Federated released (February)
OVHcloud Prescience launched (April)
Condé Nast's feature store announced (May)
Stripe's Railyard announced (May)
Pinterest's ML platform announced (June)
Apple's Overton announced (September)
PyTorch Mobile released (October)
SHAP released (November)
eBay's Krylov announced (December)
Gojek's ML platform announced (December)
Spotify's ML platform announced (December)
-
Lyft's Flyte announced (January)
Submarine released (January)
Wix's ML platform announced (February)
DoorDash's ML platform announced (April)
TorchServe released (April)
Yelp's ML platform announced (June)
Intuit's ML platform announced (August)
AWS SageMaker Feature Store launched (December)
-
Databricks Feature Store launched (May)
GCP Vertex AI Feature Store launched (August)
Etsy's redesigned cloud-based ML platform announced (December)
-
LinkedIn's DARWIN announced (January)
LinkedIn's Feathr feature store released (April)
Zalando's ML platform announced (April)
Stitch Fix's ML platform announced (July)
Ubisoft's Merlin announced (December)
Portions of this article are also available as a recorded conference talk (28 min) at Cloud Native Summit.