within the chrysalis of prime intellect
Meditation at the prime intellect office on the shapes of artificial intelligence to come.
I was walking in the steep streets of san francisco in the quiet morning hours. actually it was 10h and I was up since 6h because of jetlag, but I didn’t want to come banging at prime intellect headquarters too early.
I was a bit surprised how calm the city was on a wednesday morning. I always pictured san francisco as being extremely agitated.
the street of the headquarters was actually even calmer, like a really nice residential area. as I walked in I was welcomed by a researcher that just recently joined the team. he told me that they were moving offices soon as the place was getting a bit cramped.
I understood what he meant after moving out of the small inner courtyard and looking at the packed computer room where a team of absolutely focused engineers and researchers stood in various configurations. I was looking at a few rows of people I can only describe as not actually being there; they were in some of the purest forms of flow there is.
now. I’m a researcher at heart. I understand this particular state where you feel like mentally crawling over a shape thinly tangible but which terrain you start to map out. there is hardly anything in life I like more than to drag my metaphysical body in these spaces.
to not disturb these fine folks in their sacred trance I made my way upstairs. I realized too late that the situation wasn’t much better. a few designers were discussing and exchanging short sentences right in between making iterations on some kind of launch video (which in my humble paleolithic opinion looked pretty lit).
I sat down quietly at the dinner table for a few minutes trying to collect my initial feelings about this team. one that I’ve been interacting with cybernetically for quite some time now.
from where I was I could see the bay. a large boat was dragging its immense hull through the water. the contrast between this large expense and this crowded monastery made me feel that I was in a cocoon, a chrysalis of sorts.
as I was meditating about what the hell that even meant, a meeting suddenly started led by sami who also leads the research effort at prime.
the rumbling of all these people suddenly snapping out of their flow subspace and congregating around me was interesting. not that I mattered much. It’s just that I was on a kind of stool right in front of the screen (which I didn’t even notice).
getting right into it, sami showed us how their new intellect-3 model (100B+ MoE which was almost done training on 512 gpus) was generating some interesting SoTA results.
results which I usually only hear about in blog posts or tech reports online to my great excitement.
I zoned out for about 5 min thinking about how weird all of this was. here we were all excited about something we couldn’t touch, see or even understand entirely.
artificial intelligence has this interesting property that the substrate these researchers were working on isn’t the end artifacts (like the final model weights).
It’s not like software engineering or woodworking where the artist is sculpting, chipping away at its canvas until the vision is realized .
as I was looking at these guys looking intently at sami I realized that they looked like chemists or physicists building the machinery that creates the environment the desired result needs to grow into. and we were all looking at the result of this machinery producing state-of-the-art results in the realm of intelligence.
something about seeing the team that made it possible in a physical space was also pretty odd. because none of these researchers was entirely responsible for building this end system, but they all kind of were in each of their own ways.
as I snapped out of wherever I was I realized that surprisingly the presentation wasn’t a victory reel built to showcase how cool their results were. the team already switched gears to discuss with great excitement the shapes of things to come.
a disproportionate amount of time was now being spent on thinking and talking and hypothesizing about what they want to do for the next intelligence they wanted to grow.
I have a lifelong fascination with learning in all its facets and the last 10-minute discussion triggered by kalomaze about how it could be possible to push continual learning stayed with me well throughout the night.
It reminded me of all these evenings I’ve spent singularly focused on detecting consciousness in my little lab at the very top of the montreal general hospital. knowing that whatever blockers I was facing I just needed to talk it out with a colleague and figure out the shape of it and that at some point one of these ideas would allow me to hold onto something in that darkness.
sometime I wonder why I haven’t just continued chipping away in that trance-like way of living forever and I then remember I decided to build a startup and have three kids.
I love startups btw. the energy in a healthy one is very palpable especially when the team is holding one of these rare gems of value they know they can actively refine. yet, it’s usually hard to keep the flow of value coming out without dropping said gem. the metaphorical pressure is just too much.
I always had this dream of doing both research and startup in some shape or form. finding such a gem with value pouring out in waterfalls all around me and pooling in such a way I can meditate until I die on some of these hidden mysteries of life.
I got a glimpse of that dream and how to make it a reality at prime intellect. that team is actually a mashup of researchers and engineers which are chipping away at superintelligence with the energy of a startup and the stillness of research. all while shaping their organization to grow and capture these streams of value.
yes folks we’re talking business now. I know I know. money sucks, but superintelligence isn’t exactly a cheap endeavor.
I was zoning out again as I was looking at these words on a slide labelled mission in the applied research meeting. I don’t even remember which day it was. my stay was such a blur.
but I remember thinking “what an interesting combination of concepts to strive for...”
I quickly saw by the discussion that ensued on that slide that they were actually serious about the whole lot:
bringing about superintelligence using tools they built.
providing these tools to everyone.
doing that work in the open.
It was getting hard not to love these guys honestly.
I was sitting later that night in an overcrowded in-and-out out on fisherman wharf with my double-double thinking that there was something about the current era of machine learning that felt electric to the practically minded soul (which I’m a proud member of).
the empirical nature of machine learning shines with such a gleam.
you can just try things and get feedback pretty rapidly at a scale and breadth that I’ve never thought we would get to. back in the prehistoric machine learning time of 2019 all the ai projects were such a miserable slog of issues and problems and blockers.
now problems still exist but it feels as if you are running fast enough you can just leap past a whole bunch of them. these fine prime folks were running a whole lot and preparing to accelerate some more it seemed.
this pattern of intense focus on that multifaceted mission wasn’t isolated within the cocoon of prime intellect; the second meeting I got invited to was for the RL residency where a few researchers distributed all across the world got together to chat up about various projects they were working on.
I love student researchers man. one of the coolest slices of our society.
they are usually such interesting individuals; each one I met was a world. all have extreme uniqueness in how they go about things. very hard for them to be anything but themselves because they are thinking so deeply about doing the seemingly most random technical feat at the boundary of what is known.
they barely have any brain power left to fake or conform much.
one of them was showing us his 20 page draft notion page with the most radiant smile I’ve seen in a while. another researcher spoke about 42 words on a topic that was at the fringe of RL finetuning yet everyone understood. another was making a whole book environment and his neighbor in the google meet was making a universal verifier jesus man I was pumped.
they were all thinking and working and tinkering on the problems littering the path to this superintelligence mission with such a zeal. It was beautiful.
each project stretching the possible a tiny bit while being grounded in empirical findings these guys have been bulldozing in the past weeks. the interesting bit is that these jolly souls weren’t even full time employees. they were working on pushing reality in between their PhD works or right after they’ve clocked out of their dev jobs or at night when the stillness is conducive to these oversized ideas making contact.
one of these researchers found a hilarious result that appeared when there was a visual moral dilemma on some closed source models. the personality of all the models was surprisingly on-point glowing through their weights when confronted in the visual spaces with two bad moral outcomes.
and here we were all looking at this quirk of intelligence and thinking about ways of bending it to our will some more using anime filters.
I’ll be honest I felt good. not only because I’ve escaped the most random winter storm to hit montreal in november no no (a bit yes), but because I was surrounded with such an interesting breed of people that all are pushing in the same beautiful direction.
the last time I felt so intellectually good was in 2018 when I was at a conference about transformers at the Mila with the one and only yoshua bengio. he was being asked in a hundred different ways if attention was truly all we needed to scale intelligence by a whole crowd of very ecstatic academics.
funnily enough the man had one of the weirdest prosthetic casts on his leg and at first I thought his whole leg was robotic or something. yet no one cared.
people were eagerly grasping and crawling over what he had to say, anything really that he could show or hint towards the path forward.
in the days that passed in the san francisco bay I had many other conversations with the denizens of prime intellect all ablaze with desire to make things move.creet_z who comes from their OSS effort (check his blog post on GRPO btw very sharp),ash with a superb operational mind and that gave me one of the best compliments I ever had (almost got a nosebleed from it),will brown with a mystical third eye for RL,vincent and johannes who have such a clear vision of what prime intellect shape should be and the many others I took out of flow states with my left-field questions.
as someone who dislikes groups in general; I kinda liked these unique people and wished I could spend a whole week with each of them. they all felt so unique yet all came together in these microstructures that blended their skill sets and perspectives into a coherent whole.
this whole setup felt reminiscent of some words from karpathy that I read from a medium blog post about modules melting into an optimal whole (waaaaaaay back in the software 2.0 days).
It felt like these folks were fusing their intellect into a strange and intense logical machinery to melt orbs of intelligence to create a goddamn whole lot of babelic trowels.
pretty rad if you ask me.
while I was there I decided to sample a bit more of the city and its surroundings by extending my tiny digital tentacles. I dined with big tech researchers (some breaking NDAs), chatted with people building in ai labs (big and small), guys doing startups late at night and academic researchers at stanford on a secret mission to decode the brain (they hiring btw).
through each of these individual personalities, I could see they were reaching in a similar subspace in various ways. somewhere they all felt something gigantic slowly rumbling over.
each one of them interpreting its shape through the lens of its own ideals and thinking about harnessing it for their own purpose.
as my plane took to the beautiful san francisco sky I was still thinking about the shape of what we were all building.
not only at prime intellect but at all the different labs around this city and the world. thousands upon thousands of minds iterating on the ideal of intelligence to usher in its metamorphosis.
maybe the agitation that I pictured in my head about this place was real after all. happening not in the street but in the curvatures of this new idea looking to take form. all these offices, apartments, co-working places focused inward in rearranging how intelligence is done.
whether open, closed, decentralized or monolithic, something was moving within the chrysalis and about to take flight.