> I don’t really agree that training massive causal LMs is a “product focus”.

I'd hope that Google DeepMind's mission statement is a bit broader than that, but certainly Gemini seems to be the focus. It seems to me there's a world of difference between DeepMind's original goal of developing AGI (via whatever means - as a research objective), and now being told they have to build LLMs.

If we compare Google to Meta, it seems it used to be that DeepMind was equivalent to FAIR as a pure research organization, and Brain equivalent to Meta's product focused ML group(s), but now the joint Google DeepMind is more akin to Meta's GenAI group, and there is no unfettered research group at Google left free to pursue AGI in any way other than hoping it can be developed out of LLMs. However, FAIR also seems constrained now that they have been moved into a product-focused part of the organization (under CPO Chris Cox).

Did you work in this field? I just don’t find that a fair characterization. They are all still doing blue-sky research.

There is a ton of further research to be done in deep models. A lot of it will incorporate LLMs because that is currently the most powerful primitive you have. Research is a lot more closed now but I would not take that to mean research is no longer happening.

No - I don't work in the ML field, or for any of these companies (but I have built a Torch-like NN framework in C++ from scratch, and followed the early transformer development closely, so do understand the tech).

I'm just writing about the changes to these organizations, and their corporate governance, as reported in the press. I don't think you need to be an insider to appreciate the difference between, say, DeepMind as an independent entity pursuing AGI anyway they saw fit (RL), and now as part of Google DeepMind apparently tasked with developing SOTA LLMs. No doubt this is still a research vs pure engineering endeavor, but hard to call it blue sky when the research direction and goal is so proscribed. I personally don't believe that LLMs (or RL for that matter) are the path to AGI, but at least DeepMind used to have the flexibility to pivot and pursue whatever lines of research they felt were most promising. Do they still have that flexibility today?

> I have built a Torch-like NN framework in C++ from scratch

Mm, care to share? I am skeptical when people make claims like this, even though it is very achievable. Many people are simply "playing house" when it comes to ML tech.

I would not believe everything you read even in the tech aligned press, it is very often false. Google Deepmind is not exclusively researching LLMs.

I never released my framework, and don't intend to (abandoned this a good few years ago), but it was more than "playing house" ... It was complete enough to build/train a convnet that worked with CIFAR-10, supporting both GPU via cuDNN and CPU via my own Tensor class with MKL/IPP BLAS/etc acceleration. The API was Torch-like where you build the graph (create nodes, then connect them), then run it. I was in process of writing a version 2 with support for RNNs and auto migration of tensors from CPU to/from GPU, but gave up since PyTorch had since appeared (obviously a better approach) and it became increasingly obvious how ridiculous it was for a one-man project to attempt to catch up to SOTA!