Machine Learning Icon

Machine Learning

Machine Learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
127 Stories
All Topics

Craig Kerstiens info.crunchydata.com

Building a recommendation engine inside Postgres with Python and Pandas

Craig Kerstiens told me about this on our recent Postgres episode of The Changelog and my jaw about dropped out of my mouth.

… earlier today I was starting to wonder why couldn’t I do more machine learning directly inside [Postgres]. Yeah, there is madlib, but what if I wanted to write my own recommendation engine? So I set out on a total detour of a few hours and lo and behold, I can probably do a lot more of this in Postgres than I realized before. What follows is a quick walkthrough of getting a recommendation engine setup directly inside Postgres.

Craig doesn’t necessarily suggest you put this kind of solution in production, but he doesn’t come out and say don’t do it either. 😉

Machine Learning blog.acolyer.org

The case for a learned sorting algorithm

Adrian Colyer walks us through a paper from SageDB that’s taking machine learning and applying it to old Computer Science problems such as sorting. Here’s the big idea:

Suppose you had a model that given a data item from a list, could predict its position in a sorted version of that list. 0.239806? That’s going to be at position 287! If the model had 100% accuracy, it would give us a completed sort just by running over the dataset and putting each item in its predicted position. There’s a problem though. A model with 100% accuracy would essentially have to see every item in the full dataset and memorise its position – there’s no way training and then using such a model can be faster than just sorting, as sorting is a part of its training! But maybe we can sample a subset of the data and get a model that is a useful approximation, by learning an approximation to the CDF (cumulative distribution function).

InfoQ Icon InfoQ

AI training method exceeds GPT-3 performance with 99.9% fewer parameters

A team of scientists at LMU Munich have developed Pattern-Exploiting Training (PET), a deep-learning training technique for natural language processing (NLP) models. Using PET, the team trained a Transformer NLP model with 223M parameters that out-performed the 175B-parameter GPT-3 by over 3 percentage points on the SuperGLUE benchmark.

AI (Artificial Intelligence) github.com

Unsplash makes available 2M+ images for research and machine learning

They’ve split the dataset up into two bundles:

  1. Lite, which you can download w/ a click, but is limited to 25K image
  2. Full, which you have to request access to and is limited to non-commercial use

This is interesting for a couple of reasons. First, it’s a great resource for anyone training models for image classification, etc. Second, it’s a nice business model for Unsplash as a startup.

OpenAI Icon OpenAI

OpenAI now has an API

For years now I’ve been asking AI/ML experts when these powerful-yet-complicated tools will become available to average developers like you and me. It’s happening! Just look at how high-level this text generation code sample is:

import openai

prompt = """snipped for brevity's sake"""

response = openai.Completion.create(model="davinci",
  prompt=prompt, 
  stop="\n",
  temperature=0.9,
  max_tokens=100)

They’re oftening all kinds of language tasks: semantic search, summarization, sentiment analysis, content generation, translation, and more. The API is still in beta and there’s a waitlist, but this is exciting news, nonetheless.

Uber Engineering Icon Uber Engineering

A uniform interface to run deep learning models from multiple frameworks

Neuropod is a library that provides a uniform interface to run deep learning models from multiple frameworks in C++ and Python. Neuropod makes it easy for researchers to build models in a framework of their choosing while also simplifying productionization of these models.

This looks nice because you can make your inference code framework agnostic and easily switch between frameworks if necessary. Currently supports TensorFlow, PyTorch, TorchScript, and Keras.

Python github.com

A research framework for reinforcement learning

Acme is a library of reinforcement learning (RL) agents and agent building blocks. Acme strives to expose simple, efficient, and readable agents, that serve both as reference implementations of popular algorithms and as strong baselines, while still providing enough flexibility to do novel research. The design of Acme also attempts to provide multiple points of entry to the RL problem at differing levels of complexity.

Python github.com

A modular toolbox for accelerating meta-learning research

Meta-Blocks is a modular toolbox for research, experimentation, and reproducible benchmarking of learning-to-learn algorithms. The toolbox provides flexible APIs for working with MetaDatasets, TaskDistributions, and MetaLearners (see the figure below). The APIs make it easy to implement a variety of meta-learning algorithms, run them on well-established and emerging benchmarks, and add your own meta-learning problems to the suite and benchmark algorithms on them.

This repo is still under “heavy construction” (a.k.a. unstable) so downloader beware, but it’s worth a star/bookmark for later use.

A modular toolbox for accelerating meta-learning research

Python github.com

Efficient, reusable components for 3D computer vision research with PyTorch

PyTorch3d is designed to integrate smoothly with deep learning methods for predicting and manipulating 3D data. For this reason, all operators in PyTorch3d:

  • Are implemented using PyTorch tensors
  • Can handle minibatches of hetereogenous data
  • Can be differentiated
  • Can utilize GPUs for acceleration

Get started with tutorials on deforming a sphere mesh into a dolphin, rendering textured meshes, camera position optimization, and more.

Uber Engineering Icon Uber Engineering

Uber's new GTN algorithm speeds up deep learning by 9x

Here’s a new acronym for you: Generative Teaching Networks (GTN)

GTNs are deep neural networks that generate data and/or training environments on which a learner (e.g., a freshly initialized neural network) trains before being tested on a target task (e.g., recognizing objects in images). One advantage of this approach is that GTNs can produce synthetic data that enables other neural networks to learn faster than when training on real data. That allowed us to search for new neural network architectures nine times faster than when using real data.

Fake data, real results? Sounds pretty slick.

Victor Zhou victorzhou.com

A gentle introduction to Visual Question Answering using neural networks

Show us humans a picture of someone in uniform on a mound of dirt throwing a ball and we will quickly tell you we’re looking at baseball. But how do you make a computer come to the same conclusion?

Visual Question Answering

In this post, we’ll explore basic methods for performing VQA and build our own simple implementation in Python

JavaScript github.com

7 simple functions to give you a feel for how machines can actually "learn"

NanoNeuron is an over-simplified version of the Neuron concept from Neural Networks. NanoNeuron is trained to convert temperature values from Celsius to Fahrenheit.

The NanoNeuron.js code example contains 7 simple JavaScript functions (which touches on model prediction, cost calculation, forward/backwards propagation, and training) that will give you a feeling of how machines can actually “learn”. No 3rd-party libraries, no external data-sets or dependencies, only pure and simple JavaScript functions.

This is not a complete guide to machine learning. Just a primer.

Learn github.com

A booklet on machine learning systems design with exercises

This booklet covers four main steps of designing a machine learning system:

  1. Project setup
  2. Data pipeline
  3. Modeling: selecting, training, and debugging
  4. Serving: testing, deploying, and maintaining

It comes with links to practical resources that explain each aspect in more details. It also suggests case studies written by machine learning engineers at major tech companies who have deployed machine learning systems to solve real-world problems.

TensorFlow github.com

TensorFlow 2.0 focuses on simplicity and ease of use

Folks have been talking about TensorFlow 2 for some time now (See Practical AI #42 for one excellent example), but now it’s finally here. The bulleted list:

  • Easy model building with Keras and eager execution.
  • Robust model deployment in production on any platform.
  • Powerful experimentation for research.
  • API simplification by reducing duplication and removing deprecated endpoints.

This is a huge release. Check out the highlights list in the changelog to see for yourself.

0:00 / 0:00