🥇Top ML Papers of the Week

The top ML Papers of the Week (Jan 23-29)

In this edition of the NLP Newsletter, we cover the top ML Papers of the Week (Jan 23-29).

---

1) MusicLM - a generative model for generating high-fidelity music from text descriptions. (Paper | Tweet)

AK @_akhaliq

MusicLM: Generating Music From Text abs: arxiv.org/abs/2301.11325 project page: google-research.github.io/seanet/musiclm…

---

2) H3 - an approach to reduce the gap, in terms of performance and hardware utilization, between state space models and attention for language modeling. (Paper | Tweet)

Dan Fu @realDanFu

Attention is all you need... but how much of it do you need? Announcing H3 - a new generative language models that outperforms GPT-Neo-2.7B with only *2* attention layers! Accepted as a *spotlight* at #ICLR2023! 📣 w/ @tri_dao 📜 arxiv.org/abs/2212.14052 1/n

---

3) A Watermark for LLMs - a watermarking framework for proprietary language models. (Paper | Tweet)

Tom Goldstein @tomgoldsteincs

#OpenAI is planning to stop #ChatGPT users from making social media bots and cheating on homework by "watermarking" outputs. How well could this really work? Here's just 23 words from a 1.3B parameter watermarked LLM. We detected it with 99.999999999994% confidence. Here's how 🧵

---

4) Make-A-Video3D - a new text-to-4D model for dynamic scene generation from input text. (Paper | Tweet | Project)

Devi Parikh @deviparikh

Introducing Make-A-Video3D! Generating 3D dynamic (mini) scenes from input text. That is, text --> 4D! Needs no 4D data (i.e., no dynamic 3D data), no static 3D data, no paired text-video data. Paper: arxiv.org/abs/2301.11280 Website: make-a-video3d.github.io

---

5). ClimaX - a foundation model for weather and climate, including many capabilities for atmospheric science tasks. (Paper | Tweet | Blog)

Tung Nguyen @tungnd_13

Introducing ClimaX, the first foundation model for weather and climate. A fast and accurate one-stop AI solution for a range of atmospheric science tasks. Paper: arxiv.org/abs/2301.10343 Blog: microsoft.com/en-us/research… Thread🧵 #ML #Climate #Weather #FoundationModel

---

6) Open Problems in Applied Deep Learning - a new reference to learn about interesting open problems in deep learning. (Paper | Tweet)

elvis @omarsar0

Open Problems in Applied Deep Learning If you're looking for interesting open problems in DL, this is a good reference. Not sure if intentional but it also looks useful to get a general picture of current trends in deep learning with ~300 references. arxiv.org/abs/2301.11316 https://t.co/XGqIo9Hjnk

---

7) DetectGPT - an approach for zero-shot machine-generated text detection. Uses raw log probabilities from the LLM to determine if the passage was sampled from it. (Paper | Tweet)

Chelsea Finn @chelseabfinn

LLMs like ChatGPT are becoming more fluent – how can we detect if something was written by a language model or a human? We developed DetectGPT: a method for detecting if a passage was written by a particular language model.

---

8) StyleGAN-T - a new model that aims to regain competitiveness of GANs for fast large scale text-to-image synthesis. (Paper | Tweet)

AK @_akhaliq

StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis significantly improves over previous GANs and outperforms distilled diffusion models in terms of sample quality and speed abs: arxiv.org/abs/2301.09515 project page: sites.google.com/view/stylegan-…

---

9) ProGen - an LLM that can generate protein sequences with a predictable function across large protein families. (Paper | Tweet)

Nikhil Naik @nikhil_ai

Excited to have our paper on using large language models like ChatGPT for protein design come out in @NatureBiotech! You can tell a language model which type of protein to design, and it can generate one from scratch!

nature.comLarge language models generate functional protein sequences across diverse families - Nature BiotechnologyA generative deep-learning model designs artificial proteins with desired enzymatic activities.

---

10) The Impossibility of Parallelizing Boosting - investigates the possibility of parallelizing boosting. (Paper | Tweet)

Amin Karbasi @aminkarbasi

Well, it turned out we cannot parallelize boosting!!! arxiv.org/abs/2301.09627