Idea Machines

About Idea Machines

Idea Machines is a deep dive into the systems and people that bring innovations from glimmers in someone's eye all the way to tools, processes, and ideas that can shift paradigms. We see the outputs of innovation systems everywhere but rarely dig into how they work. Idea Machines digs below the surface into crucial but often unspoken questions to explore themes of how we enable innovations today and how we could do it better tomorrow. Idea Machines is hosted by Benjamin Reinhardt.

Publisher

Benjamin Reinhardt

Category

business · technology

Language

en

Explicit

No

First episode

7 Dec 2018

Latest episode

27 May 2024

Latest episodes

50 episodes in the feed.

1. 

   27 May 2024

   Speculative Technologies with Ben Reinhardt [Macroscience cross-post]

   Tim Hwang turns the tables and interviews me (Ben) about Speculative Technologies and research management.

2. 

   10 Feb 2024

   Industrial Research with Peter van Hardenberg [Idea Machines #50]

   Peter van Hardenberg talks about Industrialists vs. Academics, Ink&Switch's evolution over time, the Hollywood Model, internal lab infrastructure, and more! Peter is the lab director and CEO of Ink&Switch, a private, creator oriented, computing research lab. References Ink&Switch (https://www.inkandswitch.com/) (and their many publications) The Hollywood Model in R&D Idea Machines Episode with Adam Wiggins (https://ideamachinespodcast.com/adam-wiggins) Paul Erdós (https://en.wikipedia.org/wiki/Paul_Erd%C5%91s) Transcript Peter Van Hardenberg [00:01:21] Ben: Today I have the pleasure of speaking with Peter van Hardenbergh. Peter is the lab director and CEO of Inkin switch. Private creator oriented, competing research lab. I talked to Adam Wiggins, one of inkind switches founders, [00:01:35] way back in episode number four. It's amazing to see the progress they've made as an organization. They've built up an incredible community of fellow travelers and consistently released research reports that gesture at possibilities for competing that are orthogonal to the current hype cycles. Peter frequently destroys my complacency with his ability to step outside the way that research has normally done and ask, how should we be operating, given our constraints and goals. I hope you enjoy my conversation with Peter. Would you break down your distinction between academics and industrialists [00:02:08] Peter: Okay. Academics are people whose incentive structure is connected to the institutional rewards of the publishing industry, right? You, you publish papers. And you get tenure and like, it's a, it's, it's not so cynical or reductive, but like fundamentally the time cycles are long, right? Like you have to finish work according to when, you know, submission deadlines for a conference are, you know, you're [00:02:35] working on something now. You might come back to it next quarter or next year or in five years, right? Whereas when you're in industry, you're connected to users, you're connected to people at the end of the day who need to touch and hold and use the thing. And you know, you have to get money from them to keep going. And so you have a very different perspective on like time and money and space and what's possible. And the real challenge in terms of connecting these two, you know, I didn't invent the idea of pace layers, right? They, they operate at different pace layers. Academia is often intergenerational, right? Whereas industry is like, you have to make enough money every quarter. To keep the bank account from going below zero or everybody goes home, [00:03:17] Ben: Right. Did. Was it Stuart Brand who invented pace [00:03:22] Peter: believe it was Stewart Brand. Pace layers. Yeah. [00:03:25] Ben: That actually I, I'd never put these two them together, but the, the idea I, I, I think about impedance mismatches between [00:03:35] organizations a lot. And that really sort of like clicks with pace layers Exactly. Right. Where it's like [00:03:39] Peter: Yeah, absolutely. And, and I think in a big way what we're doing at, Ink& Switch on some level is trying to provide like synchro mesh between academia and industry, right? Because they, the academics are moving on a time scale and with an ambition that's hard for industry to match, right? But also, Academics. Often I think in computer science are like, have a shortage of good understanding about what the real problems people are facing in the world today are. They're not disinterested. [00:04:07] Ben: just computer [00:04:08] Peter: Those communication channels don't exist cuz they don't speak the same language, they don't use the same terminology, they don't go to the same conferences, they don't read the same publications. Right. [00:04:18] Ben: Yeah. [00:04:18] Peter: so vice versa, you know, we find things in industry that are problems and then it's like you go read the papers and talk to some scientists. I was like, oh dang. Like. We know how to solve this. It's just nobody's built it. [00:04:31] Ben: Yeah. [00:04:32] Peter: Or more accurately it would be to say [00:04:35] there's a pretty good hunch here about something that might work, and maybe we can connect the two ends of this together. [00:04:42] Ben: Yeah. Often, I, I think of it as someone, someone has, it is a quote unquote solved problem, but there are a lot of quote unquote, implementation details and those implementation details require a year of work. [00:04:56] Peter: yeah, a year or many years? Or an entire startup, or a whole career or two? Yeah. And, and speaking of, Ink&Switch, I don't know if we've ever talked about, so a switch has been around for more than half a decade, right? [00:05:14] Peter: Yeah, seven or eight years now, I think I could probably get the exact number, but yeah, about that. [00:05:19] Ben: And. I think I don't have a good idea in my head over that time. What, what has changed about in, can switches, conception of itself and like how you do things. Like what is, what are some of the biggest things that have have changed over that time?[00:05:35] [00:05:35] Peter: So I think a lot of it could be summarized as professionalization. But I, I'll give a little brief history and can switch began because the. You know, original members of the lab wanted to do a startup that was Adam James and Orion, but they recognized that they didn't, they weren't happy with computing and where computers were, and they knew that they wanted to make something that would be a tool that would help people who were solving the world's problems work better. That's kinda a vague one, but You know, they were like, well, we're not physicists, we're not social scientists. You know, we can't solve climate change or radicalization directly, or you know, the journalism crisis or whatever, but maybe we can build tools, right? We know how to make software tools. Let's build tools for the people who are solving the problems. Because right now a lot of those systems they rely on are getting like steadily worse every day. And I think they still are like the move to the cloud disempowerment of the individual, like, you [00:06:35] know, surveillance technology, distraction technology. And Tristan Harris is out there now. Like hammering on some of these points. But there's just a lot of things that are like slow and fragile and bad and not fun to work with and lose your, you know, lose your work product. You know, [00:06:51] Ben: Yeah, software as a service more generally. [00:06:54] Peter: Yeah. And like, there's definitely advantages. It's not like, you know, people are rational actors, but something was lost. And so the idea was well go do a bit of research, figure out what the shape of the company is, and then just start a company and, you know, get it all solved and move on. And I think the biggest difference, at least, you know, aside from scale and like actual knowledge is just kind of the dawning realization at some point that like there won't really be an end state to this problem. Like this isn't a thing that's transitional where you kind of come in and you do some research for a bit, and then we figure out the answer and like fold up the card table and move on to the next thing. It's like, oh no, this, this thing's gotta stick around because these problems aren't gonna [00:07:35] go away. And when we get through this round of problems, we already see what the next round are. And that's probably gonna go on for longer than any of us will be working. And so the vision now, at least from my perspective as the current lab director, is much more like, how can I get this thing to a place where it can sustain for 10 years, for 50 years, however long it takes, and you know, to become a place that. Has a culture that can sustain, you know, grow and change as new people come in. But that can sustain operations indefinitely. [00:08:07] Ben: Yeah. And, and so to circle back to the. The, the jumping off point for this, which is sort of since, since it began, what have been some of the biggest changes of how you operate? How you, or just like the, the model more generally or, or things that you were [00:08:30] Peter: Yeah, so the beginning was very informal, but, so maybe I'll skip over the first like [00:08:35] little period where it was just sort of like, Finding our footing. But around the time when I joined, we were just four or five people. And we did one project, all of us together at a time, and we just sort of like, someone would write a proposal for what we should do next, and then we would argue about like whether it was the right next thing. And, you know, eventually we would pick a thing and then we would go and do that project and we would bring in some contractors and we called it the Hollywood model. We still call it the Hollywood model. Because it was sort of structured like a movie production. We would bring in, you know, to our little core team, we'd bring in a couple specialists, you know, the equivalent of a director of photography or like a, you know, a casting director or whatever, and you bring in the people that you need to accomplish the task. Oh, we don't know how to do Bluetooth on the web. Okay. Find a Bluetooth person. Oh, there's a bunch of crypto stuff, cryptography stuff. Just be clear on this upcoming project, we better find somebody who knows, you know, the ins and outs of like, which cryptography algorithms to use or [00:09:35] what, how to build stuff in C Sharp for Windows platform or Surface, whatever the, the project was over time. You know, we got pretty good at that and I think one of the biggest changes, sort of after we kind of figured out how to actually do work was the realization that. Writing about the work not only gave us a lot of leverage in terms of our sort of visibility in the community and our ability to attract talent, but also the more we put into the writing, the more we learned about the research and that the process of, you know, we would do something and then write a little internal report and then move on. But the process of taking the work that we do, And making it legible to the outside world and explaining why we did it and what it means and how it fits into the bigger picture. That actually like being very diligent and thorough in documenting all of that greatly increases our own understanding of what we did.[00:10:35] And that was like a really pleasant and interesting surprise. I think one of my sort of concerns as lab director is that we got really good at that and we write all these like, Obscenely long essays that people claim to read. You know, hacker News comments on extensively without reading. But I think a lot about, you know, I always worry about the orthodoxy of doing the same thing too much and whether we're sort of falling into patterns, so we're always tinkering with new kind of project systems or new ways of working or new kinds of collaborations. And so yeah, that's ongoing. But this, this. The key elements of our system are we bring together a team that has both longer term people with domain contexts about the research, any required specialists who understand like interesting or important technical aspects of the work. And then we have a specific set of goals to accomplish [00:11:35] with a very strict time box. And then when it's done, we write and we put it down. And I think this avoids number of the real pitfalls in more open-ended research. It has its own shortcomings, right? But one of the big pitfalls that avoids is the kind of like meandering off and losing sight of what you're doing. And you can get great results from that in kind of a general research context. But we're very much an industrial research context. We're trying to connect real problems to specific directions to solve them. And so the time box kind of creates the fear of death. You're like, well, I don't wanna run outta time and not have anything to show for it. So you really get focused on trying to deliver things. Now sometimes that's at the cost, like the breadth or ambition of a solution to a particular thing, but I think it helps us really keep moving forward. [00:12:21] Ben: Yeah, and, and you no longer have everybody in the lab working on the same projects, right. [00:12:28] Peter: Yeah. So today, at any given time, The sort of population of the lab fluctuates between sort of [00:12:35] like eight and 15 people, depending on, you know, whether we have a bunch of projects in full swing or you know, how you count contractors. But we usually, at the moment we have sort of three tracks of research that we're doing. And those are local first software Programmable Inc. And Malleable software. [00:12:54] Ben: Nice. And so I, I actually have questions both about the, the write-ups that you do and the Hollywood model and so on, on the Hollywood model. Do you think that I, I, and this is like, do you think that the, the Hollywood model working in, in a. Industrial Research lab is particular to software in the sense that I feel like the software industry, people change jobs fairly frequently. Contracting is really common. Contractors are fairly fluid and. [00:13:32] Peter: You mean in terms of being able to staff and source people?[00:13:35] [00:13:35] Ben: Yeah, and people take, like, take these long sabbaticals, right? Where it's like, it's not uncommon in the software industry for someone to, to take six months between jobs. [00:13:45] Peter: I think it's very hard for me to generalize about the properties of other fields, so I want to try and be cautious in my evaluation here. What I would say is that, I think the general principle of having a smaller core of longer term people who think and gain a lot of context about a problem and pairing them up with people who have fresh ideas and relevant expertise, does not require you to have any particular industry structure. Right. There are lots of ways of solving this problem. Go to a research, another research organization and write a paper with someone from [00:14:35] an adjacent field. If you're in academia, right? If you're in a company, you can do a partnership you know, hire, you know, I think a lot of fields of science have much longer cycles, right? If you're doing material science, you know, takes a long time to build test apparatus and to formulate chemistries. Like [00:14:52] Ben: Yeah. [00:14:52] Peter: someone for several years, right? Like, That's fine. Get a detach detachment from another part of the company and bring someone as a secondment. Like I think that the general principle though, of putting together a mixture of longer and shorter term people with the right set of skills, yes, we solve it a particular way in our domain. But I don't think that that's software u unique to software. [00:15:17] Ben: Would, would it be overreaching to map that onto professors and postdocs and grad students where you have the professor who is the, the person who's been working on the, the program for a long time has all the context and then you have postdocs and grad students [00:15:35] coming through the lab. [00:15:38] Peter: Again, I need to be thoughtful about. How I evaluate fields that I'm less experienced with, but both my parents went through grad school and I've certainly gotten to know a number of academics. My sense of the relationship between professors and or sort of PhD, yeah, I guess professors and their PhD students, is that it's much more likely that the PhD students are given sort of a piece of the professor's vision to execute. [00:16:08] Ben: Yeah. [00:16:09] Peter: And that that is more about scaling the research interests of the professor. And I don't mean this in like a negative way but I think it's quite different [00:16:21] Ben: different. [00:16:22] Peter: than like how DARPA works or how I can switch works with our research tracks in that it's, I it's a bit more prescriptive and it's a bit more of like a mentor-mentee kind of relationship as [00:16:33] Ben: Yeah. More training.[00:16:35] [00:16:35] Peter: Yeah. And you know, that's, that's great. I mean, postdocs are a little different again, but I think, I think that's different than say how DARPA works or like other institutional research groups. [00:16:49] Ben: Yeah. Okay. I, I wanted to see how, how far I could stretch the, stretch [00:16:55] Peter: in academia there's famous stories about Adosh who would. Turn up on your doorstep you know, with a suitcase and a bottle of amphetamines and say, my, my brain is open, or something to that effect. And then you'd co-author a paper and pay his room and board until you found someone else to send him to. I think that's closer in the sense that, right, like, here's this like, great problem solver with a lot of like domain skills and he would parachute into a place where someone was working on something interesting and help them make a breakthrough with it. [00:17:25] Ben: Yeah. I think the, the thing that I want to figure out, just, you know, long, longer term is how to. Make those [00:17:35] short term collaborations happen when with, with like, I, I I think it's like, like there's some, there's some coy intention like in, in the sense of like Robert Kos around like organizational boundaries when you have people coming in and doing things in a temporary sense. [00:17:55] Peter: Yeah, academia is actually pretty good at this, right? With like paper co-authors. I mean, again, this is like the, the pace layers thing. When you have a whole bunch of people organized in an industry and a company around a particular outcome, You tend to have like very specific goals and commitments and you're, you're trying to execute against those and it's much harder to get that kind of like more fluid movement between domains. [00:18:18] Ben: Yeah, and [00:18:21] Peter: That's why I left working in companies, right? Cause like I have run engineering processes and built products and teams and it's like someone comes to me with a really good idea and I'm like, oh, it's potentially very interesting, but like, [00:18:33] Ben: but We [00:18:34] Peter: We got [00:18:35] customers who have outages who are gonna leave if we don't fix the thing, we've got users falling out of our funnel. Cause we don't do basic stuff like you just, you really have a lot of work to do to make the thing go [00:18:49] Ben: Yeah. [00:18:49] Peter: business. And you know, my experience of research labs within businesses is that they're almost universally unsuccessful. There are exceptions, but I think they're more coincidental than, than designed. [00:19:03] Ben: Yeah. And I, I think less and less successful over time is, is my observation that. [00:19:11] Peter: Interesting. [00:19:12] Ben: Yeah, there's a, there's a great paper that I will send you called like, what is the name? Oh, the the Changing Structure of American Innovation by She Aurora. I actually did a podcast with him because I like the paper so much. that that I, I think, yeah, exactly. And so going back to your, your amazing [00:19:35] write-ups, you all have clearly invested quite a chunk of, of time and resources into some amount of like internal infrastructure for making those really good. And I wanted to get a sense of like, how do you decide when it's worth investing in internal infrastructure for a lab? [00:19:58] Peter: Ooh. Ah, that's a fun question. Least at In and Switch. It's always been like sort of demand driven. I wish I could claim to be more strategic about it, but like we had all these essays, they were actually all hand coded HTML at one point. You know, real, real indie cred there. But it was a real pain when you needed to fix something or change something. Cause you had to go and, you know, edit all this H T M L. So at some point we were doing a smaller project and I built like a Hugo Templating thing [00:20:35] just to do some lab notes and I faked it. And I guess this is actually a, maybe a somewhat common thing, which is you do one in a one-off way. And then if it's promising, you invest more in it. [00:20:46] Ben: Yeah. [00:20:46] Peter: And it ended up being a bigger project to build a full-on. I mean, it's not really a cms, it's sort of a cms, it's a, it's a templating system that produces static HT m l. It's what all our essays come out of. But there's also a lot of work in a big investment in just like design and styling. And frankly, I think that one of the things that in can switch apart from other. People who do similar work in the space is that we really put a lot of work into the presentation of our work. You know, going beyond, like we write very carefully, but we also care a lot about like, picking good colors, making sure that text hyphenates well, that it, you know, that the the screencast has the right dimensions and, you know, all that little detail work and. It's expensive [00:21:35] in time and money to do, but I think it's, I think the results speak for themselves. I think it's worth it. [00:21:47] Ben: Yeah. I, and I mean, if, if the ultimate goal is to influence what people do and what they think, which I suspect is, is at least some amount of the goal then communicating it. [00:22:00] Peter: It's much easier to change somebody's mind than to build an entire company. [00:22:05] Ben: Yes. Well, [00:22:06] Peter: you wanna, if you wanna max, it depends. Well, you don't have to change everybody's mind, right? Like changing an individual person's mind might be impossible. But if you can put the right ideas out there in the right way to make them legible, then you'll change the right. Hopefully you'll change somebody's mind and it will be the right somebody. [00:22:23] Ben: yeah. No, that is, that is definitely true. And another thing that I am. Always obscenely obsessed, exceedingly impressed by that. In Switch. [00:22:35] Does is your sort of thoughtfulness around how you structure your community and sort of tap into it. Would you be willing to sort of like, walk me through how you think about that and like how you have sort of the, the different layers of, of kind of involvement? [00:22:53] Peter: Okay. I mean, sort of the, maybe I'll work from, from the inside out cuz that's sort of the history of it. So in the beginning there was just sort of the people who started the lab. And over time they recruited me and, and Mark Mcg again and you know, some of our other folk to come and, and sign on for this crazy thing. And we started working with these wonderful, like contractors off and on and and so the initial sort of group was quite small and quite insular and we didn't publish anything. And what we found was that. Once we started, you know, just that alone, the act of bringing people in and working with them started to create the beginning of a [00:23:35] community because people would come into a project with us, they'd infect us with some of their ideas, we'd infect them with some of ours. And so you started to have this little bit of shared context with your past collaborators. And because we have this mix of like longer term people who stick with the lab and other people who come and go, You start to start to build up this, this pool of people who you share ideas and language with. And over time we started publishing our work and we began having what we call workshops where we just invite people to come and talk about their work at Ink and Switch. And by at, I mean like now it's on a discord. Back in the day it was a Skype or a Zoom call or whatever. And the rule back then in the early days was like, if you want to come to the talk. You have to have given a talk or have worked at the lab. And so it was like very good signal to noise ratio in attendance cuz the only people who would be on the zoom call would be [00:24:35] people who you knew were grappling with those problems. For real, no looky lose, no, no audience, right? And over time it just, there were too many really good, interesting people who are doing the work. To fit in all those workshops and actually scheduling workshops is quite tiring and takes a lot of energy. And so over time we sort of started to expand this community a little further. And sort of now our principle is you know, if you're doing the work, you're welcome to come to the workshops. And we invite some people to do workshops sometimes, but that's now we have this sort of like small private chat group of like really interesting folk. And it's not open to the public generally because again, we, I don't want to have an audience, right? I want it to practitioner's space. And so over time, those people have been really influential on us as well. And having that little inner [00:25:35] circle, and it's a few hundred people now of people who, you know, like if you have a question to ask about something tricky. There's probably somebody in there who has tried it, but more significantly, like the answer will come from somebody who has tried it, not from somebody who will call you an idiot for trying or who will, right, like you, you avoid all the, don't read the comments problems because the sort of like, if anybody was like that, I would probably ask them to leave, but we've been fortunate that we haven't had any of that kind of stuff in the community. I will say though, I think I struggle a lot because I think. It's hard to be both exclusive and inclusive. Right, but exclusive community deliberately in the sense that I want it to be a practitioner's space and one where people can be wrong and it's not too performative, like there's not investors watching or your, your user base or whatever. [00:26:32] Ben: Yeah. [00:26:32] Peter: at the same time, [00:26:33] Ben: strangers. [00:26:34] Peter: [00:26:35] inclusive space where we have people who are earlier in their career or. From non-traditional backgrounds, you know, either academically or culturally or so on and so forth. And it takes constant work to be like networking out and meeting new people and like inviting them into this space. So it's always an area to, to keep working on. At some point, I think we will want to open the aperture further, but yeah, it's, it's, it's a delicate thing to build a community. [00:27:07] Ben: Yeah, I mean the, the, frankly, the reason I'm asking is because I'm trying to figure out the same things and you have done it better than basically anybody else that I've seen. This is, this is maybe getting too down into the weeds. But why did you decide that discourse or discord was the right tool for it? And the, the reason that I ask is that I personally hate sort of [00:27:35] streaming walls of texts, and I find it very hard to, to seriously discuss ideas in, in that format. [00:27:43] Peter: Yeah, I think async, I mean, I'm an old school like mailing list guy. On some level I think it's just a pragmatic thing. We use Discord for our internal like day-to-day operations like. Hey, did you see the pr? You know, oh, we gotta call in an hour with so-and-so, whatever. And then we had a bunch of people in that community and then, you know, we started having the workshops and inviting more people. So we created a space in that same discord where. You know, people didn't have to get pinged when we had a lab call and we didn't want 'em turning up on the zoom anyway. And so it wasn't so much like a deliberate decision to be that space. I think there's a huge opportunity to do better and you know, frankly, what's there is [00:28:35] not as designed or as deliberate as I would like. It's more consequence of Organic growth over time and just like continuing to do a little bit here and there than like sort of an optimum outcome. And it could, there, there's a lot of opportunity to do better. Like we should have newsletters, there should be more, you know, artifacts of past conversations with better organizations. But like all of that stuff takes time and energy. And we are about a small little research lab. So many people you know, [00:29:06] Ben: I, I absolutely hear you on that. I think the, the, the tension that I, I see is that people, I think like texting, like sort of stream of texts. Slack and, and discord type things. And, and so there's, there's the question of like, what can you get people to do versus like, what creates the, the right conversation environment?[00:29:35] And, and maybe that's just like a matter of curation and like standard setting. [00:29:42] Peter: Yeah, I don't know. We've had our, our rabbit trails and like derailed conversations over the years, but I think, you know, if you had a forum, nobody would go there. [00:29:51] Ben: Yeah. [00:29:52] Peter: like, and you could do a mailing list, but I don't know, maybe we could do a mailing list. That would be a nice a nice form, I think. But people have to get something out of a community to put things into it and you know, you have to make, if you want to have a forum or, or an asynchronous posting place, you know, the thing is people are already in Discord or slack. [00:30:12] Ben: exactly. [00:30:13] Peter: something else, you have to push against the stream. Now, actually, maybe one interesting anecdote is I did experiment for a while with, like, discord has sort of a forum post feature. They added a while back [00:30:25] Ben: Oh [00:30:25] Peter: added it. Nobody used it. So eventually I, I turned it off again. Maybe, maybe it just needs revisiting, but it surprised me that it wasn't adopted, I guess is what [00:30:35] I would say. [00:30:36] Ben: Yeah. I mean, I think it, I think the problem is it takes more work. It's very easy to just dash off a thought. [00:30:45] Peter: Yeah, but I think if you have the right community, then. Those thoughts are likely to have been considered and the people who reply will speak from knowledge [00:30:55] Ben: Yeah. [00:30:56] Peter: and then it's not so bad, right? [00:30:59] Ben: it's [00:30:59] Peter: The problem is with Hacker News or whatever where like, or Reddit or any of these open communities like you, you know, the person who's most likely to reply is not the person who's most helpful to apply. [00:31:11] Ben: Yeah, exactly. Yeah, that makes, that makes a lot of sense. And sort of switching tracks yet again, how so one, remind me how long your, your projects are, like how long, how big are the, is the time box. [00:31:28] Peter: the implementation phase for a standard income switch Hollywood project, which I can now call them standard, I think, cuz we've done like, [00:31:35] Ooh, let me look. 25 or so over the years. Let's see, what's my project count number at? I have a little. Tracker. Yeah, I think it's 25 today. So we've done about 20 some non-trivial number of these 10 to 12 weeks of implementation is sort of the core of the project, and the idea is that when you hit that start date, at the beginning of that, you should have the team assembled. You should know what you're building, you should know why you're building it, and you should know what done looks like. Now it's research, so inevitably. You know, you get two weeks in and then you take a hard left and like, you know, but that, that we write what's called the brief upfront, which is like, what is the research question we are trying to answer by funding this work and how do we think this project will answer it? Now, your actual implementation might change, or you might discover targets of opportunity along the way. But the idea is that by like having a, a narrow time box, like a, a team [00:32:35] that has a clear understanding of what you're trying to accomplish. And like the right set of people on board who already have all the like necessary skills. You can execute really hard for like that 10 to 12 weeks and get quite far in that time. Now, that's not the whole project though. There's usually a month or two upfront of what we call pre-infusion, kind of coming from the espresso idea that like you make better espresso if you take a little time at low pressure first to get ready with the shot, and so we'll do. You know, and duration varies here, but there's a period before that where we're making technical choices. Are we building this for the web or is this going on iPad? Are we gonna do this with rust and web assembly, or is this type script is this, are we buying Microsoft Surface tablets for this as we're like the ink behavior, right? So all those decisions we try and make up front. So when you hit the execution phase, you're ready to go. Do we need, what kind of designer do we want to include in this project? And who's available, you know? All of that stuff. We [00:33:35] try and square away before we get to the execution phase. [00:33:38] Ben: right. [00:33:38] Peter: when the end of the execution phase, it's like we try to be very strict with like last day pencils down and try to also reserve like the last week or two for like polish and cleanup and sort of getting things. So it's really two to two and a half, sometimes three months is like actually the time you have to do the work. And then after that, essays can take between like two months and a year or two. To produce finally. But we try to have a dr. We try to have a good first draft within a month after the end of the project. And again, this isn't a process that's like probably not optimal, but basically someone on the team winds up being the lead writer and we should be more deliberate about that. But usually the project lead for a given project ends up being the essay writer. And they write a first draft with input and collaboration from the rest of the group. And then people around [00:34:35] the lab read it and go, this doesn't make any sense at all. Like, what? What do you do? And you know, to, to varying degrees. And then it's sort of okay, right? Once you've got that kind of feedback, then you go back and you restructured and go, oh, I need to explain this part more. You know, oh, these findings don't actually cover the stuff that other people at the lab thought was interesting from the work or whatever. And then that goes through, you know, an increasing sort of, you know, standard of writing stuff, right? You send it out to some more people and then you send it to a bigger group. And you know, we send it to people who are in the field that whose input we respect. And then we take their edits and we debate which ones to take. And then eventually it goes in the HTML template. And then there's a long process of like hiring an external copy editor and building nice quality figures and re-recording all your crappy screencasts to be like, Really crisp with nice lighting and good, you know, pacing and, you know, then finally at the end of all of that, we publish. [00:35:33] Ben: Nice. And [00:35:35] how did you settle on the, the 10 to 12 weeks as the right size, time box? [00:35:42] Peter: Oh, it's it's it's, it's clearly rationally optimal. [00:35:46] Ben: Ah, of course, [00:35:47] Peter: No, I'm kidding. It's totally just, it became a habit. I mean, I think. Like I, I can give an intuitive argument and we've, we've experimented a bit. You know, two weeks is not long enough to really get into anything, [00:36:02] Ben: right. [00:36:02] Peter: and the year is too long. There's too much, too much opportunity to get lost along the way. There's no, you go too long with no real deadline pressure. It's very easy to kind of wander off into the woods. And bear in mind that like the total project duration is really more like six months, right? And so where we kind of landed is also that we often have like grad students or you know, people who are between other contracts or things. It's much easier to get people for three months than for eight months. And if I feel like [00:36:35] just intuitively, if I, if someone came to you with an eight month project, I'd be, I'm almost positive that I would be able to split it into two, three month projects and we'd be able to like find a good break point somewhere in the middle. And then write about that and do another one. And it's like, this is sort of a like bigger or smaller than a bread box argument, but like, you know, a month is too little and six months feels too long. So two to four months feels about right. In terms of letting you really get into, yeah, you can really get into the meat of a problem. You can try a few different approaches. You can pick your favorite and then spend a bit of time like analyzing it and like working out the kinks. And then you can like write it up. [00:37:17] Ben: Thanks. [00:37:18] Peter: But you know, there have been things that are not, that haven't fit in that, and we're doing some stuff right now that has, you know, we've had a, like six month long pre-infusion going this year already on some ink stuff. So it's not a universal rule, but like that's the, that's the [00:37:33] Ben: Yeah. No, I [00:37:35] appreciate that intuition [00:37:36] Peter: and I think it also, it ties into being software again, right? Like again, if you have to go and weld things and like [00:37:43] Ben: yeah, exactly. [00:37:44] Peter: You know, [00:37:44] Ben: let let some bacteria grow. [00:37:46] Peter: or like, you know, the, it's very much a domain specific answer. [00:37:51] Ben: Yeah. Something that I wish people talked about more was like, like characteristic time scales of different domains. And I, I think that's software, I mean, software is obviously shorter, but it'd be interesting to, to sort of dig down and be like, okay, like what, what actually is it? So the, the, the last question I'd love to ask is, To what extent does everybody in the lab know what's, what everybody else is working on? Like. [00:38:23] Peter: So we use two tools for that. We could do a better job of this. Every Monday the whole lab gets together for half an hour only. [00:38:35] And basically says what they're doing. Like, what are you up to this week? Oh, we're trying to like, you know, figure out what's going on with that you know, stylist shaped problem we were talking about at the last demo, or, oh, we're, you know, we're in essay writing mode. We've got a, we're hoping to get the first draft done this week, or, you know, just whatever high level kind of objectives the team has. And then I was asked the question like, well, Do you expect to have anything for show and tell on Friday and every week on Friday we have show and tell or every other week. Talk a bit more about that and at show and tell. It's like whatever you've got that you want input on or just a deadline for you can share. Made some benchmark showing that this code is now a hundred times faster. Great. Like bring it to show and tell. Got that like tricky you know, user interaction, running real smooth. Bring it to show and tell, built a whole new prototype of a new kind of [00:39:35] like notetaking app. Awesome. Like come and see. And different folks and different projects have taken different approaches to this. What has been most effective, I'm told by a bunch of people in their opinion now is like, kind of approaching it. Like a little mini conference talk. I personally actually air more on the side of like a more casual and informal thing. And, and those can be good too. Just from like a personal alignment like getting things done. Perspective. What I've heard from people doing research who want to get useful feedback is that when they go in having sort of like rehearsed how to explain what they're doing, then how to show what they've done and then what kind of feedback they want. That not only do they get really good feedback, but also that process of making sure that the demo you're gonna do will actually run smoothly and be legible to the rest of the group [00:40:35] forces you. Again, just like the writing, it forces you to think about what you're doing and why you made certain choices and think about which ones people are gonna find dubious and tell them to either ignore that cuz it was a stand-in or let's talk about that cuz it's interesting. And like that, that that little cycle is really good. And that tends to be, people often come every two weeks for that [00:40:59] Ben: Yeah. [00:41:01] Peter: within when they're in active sort of mode. And so not always, but like two weeks feels about like the right cadence to, to have something. And sometimes people will come and say like, I got nothing this week. Like, let's do it next week. It's fine. And the other thing we do with that time is we alternate what we call zoom outs because they're on Zoom and I have no, no sense of humor I guess. But they're based on, they're based on the old you and your research hamming paper with where the idea is that like, at least for a little while, every week [00:41:35] we all get together and talk about something. Bigger picture that's not tied to any of our individual projects. Sometimes we read a paper together, sometimes we talk about like an interesting project somebody saw, you know, in the world. Sometimes it's skills sharing. Sometimes it's you know, just like, here's how I make coffee or something, right? Like, You know, just anything that is bigger picture or out of the day-to-day philosophical stuff. We've read Illich and, and Ursula Franklin. People love. [00:42:10] Ben: I like that a lot. And I, I think one thing that, that didn't, that, that I'm still wondering about is like, On, on sort of a technical level are, are there things that some peop some parts of the lab that are working on that other parts of the lab don't get, like they, they know, oh, like this person's working on [00:42:35] inks, but they kind of have no idea how inks actually work? Or is it something where like everybody in the lab can have a fairly detailed technical discussion with, with anybody else [00:42:45] Peter: Oh no. I mean, okay, so there are interesting interdependencies. So some projects will consume the output of past projects or build on past projects. And that's interesting cuz it can create almost like a. Industry style production dependencies where like one team wants to go be doing some research. The local first people are trying to work on a project. Somebody else is using auto merge and they have bugs and it's like, oh but again, this is why we have those Monday sort of like conversations. Right? But I think the teams are all quite independent. Like they have their own GitHub repositories. They make their own technology decisions. They use different programming languages. They, they build on different stacks, right? Like the Ink team is often building for iPad because that's the only place we can compile like [00:43:35] ink rendering code to get low enough latency to get the experiences we want. We've given up on the browser, we can't do it, but like, The local first group for various reasons has abandoned electron and all of these like run times and mostly just build stuff for the web now because it actually works and you spend all, spend way less calories trying to make the damn thing go if you don't have to fight xcode and all that kind of stuff. And again, so it really varies, but, and people choose different things at different times, but no, it's not like we are doing code review for each other or like. Getting into the guts. It's much more high level. Like, you know, why did you make that, you know, what is your programming model for this canvas you're working on? How does you know, how does this thing relate to that thing? Why is, you know, why does that layout horizontally? It feels hard to, to parse the way you've shown that to, you know, whatever. [00:44:30] Ben: Okay, cool. That, that makes sense. I just, I, the, the, the reason I ask [00:44:35] is I am just always thinking about how how related do projects inside of a single organization need to be for, like, is, is there sort of like an optimum amount of relatedness? [00:44:50] Peter: I view them all as the aspects of the same thing, and I think that that's, that's an important. Thing we didn't talk about. The goal of income switch is to give rise to a new kind of computing that is more user-centric, that's more productive, that's more creative in like a very raw sense that we want people to be able to think better thoughts, to produce better ideas, to make better art, and that computers can help them with that in ways that they aren't and in fact are [00:45:21] Ben: Yeah. [00:45:25] Peter: whether you're working on ink, Or local first software or malleable software media canvases or whatever domain you are working in. It [00:45:35] is the same thing. It is an ingredient. It is an aspect, it is a dimension of one problem. And so some, in some sense, all of this adds together to make something, whether it's one thing or a hundred things, whether it takes five years or 50 years, you know, that's, we're all going to the same place together. But on many different paths and at different speeds and with different confidence, right? And so in the small, the these things can be totally unrelated, but in the large, they all are part of one mission. And so when you say, how do you bring these things under one roof, when should they be under different roofs? It's like, well, when someone comes to me with a project idea, I ask, do we need this to get to where we're going? [00:46:23] Ben: Yeah, [00:46:24] Peter: And if we don't need it, then we probably don't have time to work on it because there's so much to do. And you know, there's a certain openness to experimentation and, [00:46:35] and uncertainty there. But that, that's the rubric that I use as the lab director is this, is this on the critical path of the revolution?

3. 

   27 Nov 2023

   MACROSCIENCE with Tim Hwang [Idea Machines #49]

   A conversation with Tim Hwang about historical simulations, the interaction of policy and science, analogies between research ecosystems and the economy, and so much more. Topics Historical Simulations Macroscience Macro-metrics for science Long science The interaction between science and policy Creative destruction in research “Regulation” for scientific markets Indicators for the health of a field or science as a whole “Metabolism of Science” Science rotation programs Clock speeds of Regulation vs Clock Speeds of Technology References Macroscience Substack (https://www.macroscience.org/) Ada Palmer’s Papal Simulation Think Tank Tycoon Universal Paperclips (Paperclip maximizer html game) Pitt Rivers Museum (https://www.prm.ox.ac.uk/) Transcript [00:02:02] Ben: Wait, so tell me more about the historical LARP that you're doing. Oh, [00:02:07] Tim: yeah. So this comes from like something I've been thinking about for a really long time, which is You know in high school, I did model UN and model Congress, and you know, I really I actually, this is still on my to do list is to like look into the back history of like what it was in American history, where we're like, this is going to become an extracurricular, we're going to model the UN, like it has all the vibe of like, after World War II, the UN is a new thing, we got to teach kids about international institutions. Anyways, like, it started as a joke where I was telling my [00:02:35] friend, like, we should have, like, model administrative agency. You know, you should, like, kids should do, like, model EPA. Like, we're gonna do a rulemaking. Kids need to submit. And, like, you know, there'll be Chevron deference and you can challenge the rule. And, like, to do that whole thing. Anyways, it kind of led me down this idea that, like, our, our notion of simulation, particularly for institutions, is, like, Interestingly narrow, right? And particularly when it comes to historical simulation, where like, well we have civil war reenactors, they're kind of like a weird dying breed, but they're there, right? But we don't have like other types of historical reenactments, but like, it might be really valuable and interesting to create communities around that. And so like I was saying before we started recording, is I really want to do one that's a simulation of the Cuban Missile Crisis. But like a serious, like you would like a historical reenactment, right? Yeah. Yeah. It's like everybody would really know their characters. You know, if you're McNamara, you really know what your motivations are and your background. And literally a dream would be a weekend simulation where you have three teams. One would be the Kennedy administration. The other would be, you know, Khrushchev [00:03:35] and the Presidium. And the final one would be the, the Cuban government. Yeah. And to really just blow by blow, simulate that entire thing. You know, the players would attempt to not blow up the world, would be the idea. [00:03:46] Ben: I guess that's actually the thing to poke, in contrast to Civil War reenactment. Sure, like you know how [00:03:51] Tim: that's gonna end. Right, [00:03:52] Ben: and it, I think it, that's the difference maybe between, in my head, a simulation and a reenactment, where I could imagine a simulation going [00:04:01] Tim: differently. Sure, right. [00:04:03] Ben: Right, and, and maybe like, is the goal to make sure the same thing happened that did happen, or is the goal to like, act? faithfully to [00:04:14] Tim: the character as possible. Yeah, I think that's right, and I think both are interesting and valuable, right? But I think one of the things I'm really interested in is, you know, I want to simulate all the characters, but like, I think one of the most interesting things reading, like, the historical record is just, like, operating under deep uncertainty about what's even going on, right? Like, for a period of time, the American [00:04:35] government is not even sure what's going on in Cuba, and, like, you know, this whole question of, like, well, do we preemptively bomb Cuba? Do we, we don't even know if the, like, the warheads on the island are active. And I think I would want to create, like, similar uncertainty, because I think that's where, like, that's where the strategic vision comes in, right? That, like, you have the full pressure of, like, Maybe there's bombs on the island. Maybe there's not even bombs on the island, right? And kind of like creating that dynamic. And so I think simulation is where there's a lot, but I think Even reenactment for some of these things is sort of interesting. Like, that we talk a lot about, like, oh, the Cuban Missile Crisis. Or like, the other joke I had was like, we should do the Manhattan Project, but the Manhattan Project as, like, historical reenactment, right? And it's kind of like, you know, we have these, like, very, like off the cuff or kind of, like, stereotype visions of how these historical events occur. And they're very stylized. Yeah, exactly, right. And so the benefit of a reenactment that is really in detail Yeah. is like, oh yeah, there's this one weird moment. You know, like that, that ends up being really revealing historical examples. And so even if [00:05:35] you can't change the outcome, I think there's also a lot of value in just doing the exercise. Yeah. Yeah. The, the thought of [00:05:40] Ben: in order to drive towards this outcome that I know. Actually happened I wouldn't as the character have needed to do X. That's right That's like weird nuanced unintuitive thing, [00:05:50] Tim: right? Right and there's something I think about even building into the game Right, which is at the very beginning the Russians team can make the decision on whether or not they've even actually deployed weapons into the cube at all, yeah, right and so like I love that kind of outcome right which is basically like And I think that's great because like, a lot of this happens on the background of like, we know the history. Yeah. Right? And so I think like, having the team, the US team put under some pressure of uncertainty. Yeah. About like, oh yeah, they could have made the decision at the very beginning of this game that this is all a bluff. Doesn't mean anything. Like it's potentially really interesting and powerful, so. [00:06:22] Ben: One precedent I know for this completely different historical era, but there's a historian, Ada Palmer, who runs [00:06:30] Tim: a simulation of a people election in her class every year. That's so good. [00:06:35] And [00:06:36] Ben: it's, there, you know, like, it is not a simulation. [00:06:40] Tim: Or, [00:06:41] Ben: sorry, excuse me, it is not a reenactment. In the sense that the outcome is indeterminate. [00:06:47] Tim: Like, the students [00:06:48] Ben: can determine the outcome. But... What tends to happen is like structural factors emerge in the sense that there's always a war. Huh. The question is who's on which sides of the war? Right, right. And what do the outcomes of the war actually entail? That's right. Who [00:07:05] Tim: dies? Yeah, yeah. And I [00:07:07] Ben: find that that's it's sort of Gets at the heart of the, the great [00:07:12] Tim: man theory versus the structural forces theory. That's right. Yeah. Like how much can these like structural forces actually be changed? Yeah. And I think that's one of the most interesting parts of the design that I'm thinking about right now is kind of like, what are the things that you want to randomize to impose different types of like structural factors that could have been in that event? Right? Yeah. So like one of the really big parts of the debate at XCOM in the [00:07:35] early phases of the Cuban Missile Crisis is You know, McNamara, who's like, right, he runs the Department of Defense at the time. His point is basically like, look, whether or not you have bombs in Cuba or you have bombs like in Russia, the situation has not changed from a military standpoint. Like you can fire an ICBM. It has exactly the same implications for the U. S. And so his, his basically his argument in the opening phases of the Cuban Missile Crisis is. Yeah. Which is actually pretty interesting, right? Because that's true. But like, Kennedy can't just go to the American people and say, well, we've already had missiles pointed at us. Some more missiles off, you know, the coast of Florida is not going to make a difference. Yeah. And so like that deep politics, and particularly the politics of the Kennedy administration being seen as like weak on communism. Yeah. Is like a huge pressure on all the activity that's going on. And so it's almost kind of interesting thinking about the Cuban Missile Crisis, not as like You know us about to blow up the world because of a truly strategic situation but more because of like the local politics make it so difficult to create like You know situations where both sides can back down [00:08:35] successfully. Basically. Yeah [00:08:36] Ben: The the one other thing that my mind goes to actually to your point about it model UN in schools. Huh, right is Okay, what if? You use this as a pilot, and then you get people to do these [00:08:49] Tim: simulations at [00:08:50] Ben: scale. Huh. And that's actually how we start doing historical counterfactuals. Huh. Where you look at, okay, you know, a thousand schools all did a simulation of the Cuban Missile Crisis. In those, you know, 700 of them blew [00:09:05] Tim: up the world. Right, right. [00:09:07] Ben: And it's, it actually, I think it's, That's the closest [00:09:10] Tim: thing you can get to like running the tape again. Yeah. I think that's right. And yeah, so I think it's, I think it's a really underused medium in a lot of ways. And I think particularly as like you know, we just talk, talk like pedagogically, like it's interesting that like, it seems to me that there was a moment in American pedagogical history where like, this is a good way of teaching kids. Like, different types of institutions. And like, but it [00:09:35] hasn't really matured since that point, right? Of course, we live in all sorts of interesting institutions now. And, and under all sorts of different systems that we might really want to simulate. Yeah. And so, yeah, this kind of, at least a whole idea that there's lots of things you could teach if you, we like kind of opened up this way of kind of like, Thinking about kind of like educating for about institutions. Right? So [00:09:54] Ben: that is so cool. Yeah, I'm going to completely, [00:09:59] Tim: Change. Sure. Of course. [00:10:01] Ben: So I guess. And the answer could be no, but is, is there connections between this and your sort of newly launched macroscience [00:10:10] Tim: project? There is and there isn't. Yeah, you know, I think like the whole bid of macroscience which is this project that I'm doing as part of my IFP fellowship. Yeah. Is really the notion that like, okay, we have all these sort of like interesting results that have come out of metascience. That kind of give us like, kind of like the beginnings of a shape of like, okay, this is how science might work and how we might like get progress to happen. And you know, we've got [00:10:35] like a bunch of really compelling hypotheses. Yeah. And I guess my bit has been like, I kind of look at that and I squint and I'm like, we're, we're actually like kind of in the early days of like macro econ, but for science, right? Which is like, okay, well now we have some sense of like the dynamics of how the science thing works. What are the levers that we can start, like, pushing and pulling, and like, what are the dials we could be turning up and turning down? And, and, you know, I think there is this kind of transition that happens in macro econ, which is like, we have these interesting results and hypotheses, but there's almost another... Generation of work that needs to happen into being like, oh, you know, we're gonna have this thing called the interest rate Yeah, and then we have all these ways of manipulating the money supply and like this is a good way of managing like this economy Yeah, right and and I think that's what I'm chasing after with this kind of like sub stack but hopefully the idea is to build it up into like a more coherent kind of framework of ideas about like How do we make science policy work in a way that's better than just like more science now quicker, please? Yeah, right, which is I think we're like [00:11:35] we're very much at at the moment. Yeah, and in particular I'm really interested in the idea of chasing after science almost as like a Dynamic system, right? Which is that like the policy levers that you have You would want to, you know, tune up and tune down, strategically, at certain times, right? And just like the way we think about managing the economy, right? Where you're like, you don't want the economy to overheat. You don't want it to be moving too slow either, right? Like, I am interested in kind of like, those types of dynamics that need to be managed in science writ large. And so that's, that's kind of the intuition of the project. [00:12:04] Ben: Cool. I guess, like, looking at macro, how did we even decide, macro econ, [00:12:14] Tim: how did we even decide that the things that we're measuring are the right things to measure? Right? Like, [00:12:21] Ben: isn't it, it's like kind of a historical contingency that, you know, it's like we care about GDP [00:12:27] Tim: and the interest rate. Yeah. I think that's right. I mean in, in some ways there's a triumph of like. It's a normative triumph, [00:12:35] right, I think is the argument. And you know, I think a lot of people, you hear this argument, and it'll be like, And all econ is made up. But like, I don't actually think that like, that's the direction I'm moving in. It's like, it's true. Like, a lot of the things that we selected are arguably arbitrary. Yeah. Right, like we said, okay, we really value GDP because it's like a very imperfect but rough measure of like the economy, right? Yeah. Or like, oh, we focus on, you know, the money supply, right? And I think there's kind of two interesting things that come out of that. One of them is like, There's this normative question of like, okay, what are the building blocks that we think can really shift the financial economy writ large, right, of which money supply makes sense, right? But then the other one I think which is so interesting is like, there's a need to actually build all these institutions. that actually give you the lever to pull in the first place, right? Like, without a federal reserve, it becomes really hard to do monetary policy. Right. Right? Like, without a notion of, like, fiscal policy, it's really hard to do, like, Keynesian as, like, demand side stuff. Right. Right? And so, like, I think there's another project, which is a [00:13:35] political project, to say... Okay, can we do better than just grants? Like, can we think about this in a more, like, holistic way than simply we give money to the researchers to work on certain types of problems. And so this kind of leads to some of the stuff that I think we've talked about in the past, which is like, you know, so I'm obsessed right now with like, can we influence the time horizon of scientific institutions? Like, imagine for a moment we had a dial where we're like, On average, scientists are going to be thinking about a research agenda which is 10 years from now versus next quarter. Right. Like, and I think like there's, there's benefits and deficits to both of those settings. Yeah. But man, if I don't hope that we have a, a, a government system that allows us to kind of dial that up and dial that down as we need it. Right. Yeah. The, the, [00:14:16] Ben: perhaps, quite like, I guess a question of like where the analogy like holds and breaks down. That I, that I wonder about is, When you're talking about the interest rate for the economy, it kind of makes sense to say [00:14:35] what is the time horizon that we want financial institutions to be thinking on. That's like roughly what the interest rate is for, but it, and maybe this is, this is like, I'm too, [00:14:49] Tim: my note, like I'm too close to the macro, [00:14:51] Ben: but thinking about. The fact that you really want people doing science on like a whole spectrum of timescales. And, and like, this is a ill phrased question, [00:15:06] Tim: but like, I'm just trying to wrap my mind around it. Are you saying basically like, do uniform metrics make sense? Yeah, exactly. For [00:15:12] Ben: like timescale, I guess maybe it's just. is an aggregate thing. [00:15:16] Tim: Is that? That's right. Yeah, I think that's, that's, that's a good critique. And I think, like, again, I think there's definitely ways of taking the metaphor too far. Yeah. But I think one of the things I would say back to that is It's fine to imagine that we might not necessarily have an interest rate for all of science, right? So, like, you could imagine saying, [00:15:35] okay, for grants above a certain size, like, we want to incentivize certain types of activity. For grants below a certain size, we want different types of activity. Right, another way of slicing it is for this class of institutions, we want them to be thinking on these timescales versus those timescales. Yeah. The final one I've been thinking about is another way of slicing it is, let's abstract away institutions and just think about what is the flow of all the experiments that are occurring in a society? Yeah. And are there ways of manipulating, like, the relative timescales there, right? And that's almost like, kind of like a supply based way of looking at it, which is... All science is doing is producing experiments, which is like true macro, right? Like, I'm just like, it's almost offensively simplistic. And then I'm just saying like, okay, well then like, yeah, what are the tools that we have to actually influence that? Yeah, and I think there's lots of things you could think of. Yeah, in my mind. Yeah, absolutely. What are some, what are some that are your thinking of? Yeah, so I think like the two that I've been playing around with right now, one of them is like the idea of like, changing the flow of grants into the system. So, one of the things I wrote about in Microscience just the past week was to think [00:16:35] about, like sort of what I call long science, right? And so the notion here is that, like, if you look across the scientific economy, there's kind of this rough, like, correlation between size of grant and length of grant. Right, where so basically what it means is that like long science is synonymous with big science, right? You're gonna do a big ambitious project. Cool. You need lots and lots and lots of money Yeah and so my kind of like piece just briefly kind of argues like but we have these sort of interesting examples like the You know Like framing a heart study which are basically like low expense taking place over a long period of time and you're like We don't really have a whole lot of grants that have that Yeah. Right? And so the idea is like, could we encourage that? Like imagine if we could just increase the flow of those types of grants, that means we could incentivize more experiments that take place like at low cost over long term. Yeah. Right? Like, you know, and this kind of gets this sort of interesting question is like, okay, so what's the GDP here? Right? Like, or is that a good way of cracking some of the critical problems that we need to crack right now? Right? Yeah. And it's kind of where the normative part gets into [00:17:35] it is like, okay. So. You know, one way of looking at this is the national interest, right? We say, okay, well, we really want to win on AI. We really want to win on, like, bioengineering, right? Are there problems in that space where, like, really long term, really low cost is actually the kind of activity we want to be encouraging? The answer might be no, but I think, like, it's useful for us to have, like, that. Color in our palette of things that we could be doing Yeah. In like shaping the, the dynamics of science. Yeah. Yeah. [00:18:01] Ben: I, I mean, one of the things that I feel like is missing from the the meta science discussion Mm-Hmm. is, is even just, what are those colors? Mm-Hmm. like what, what are the, the different and almost parameters of [00:18:16] Tim: of research. Yeah. Right, right, right. And I think, I don't know, one of the things I've been thinking about, which I'm thinking about writing about at some point, right, is like this, this view is, this view is gonna piss people off in some ways, because where it ultimately goes is this idea that, like, like, the scientist or [00:18:35] science Is like a system that's subject to the government, or subject to a policy maker, or a strategist. Which like, it obviously is, right? But like, I think we have worked very hard to believe that like, The scientific market is its own independent thing, And like, that touching or messing with it is like, a not, not a thing you should do, right? But we already are. True, that's kind of my point of view, yeah exactly. I think we're in some ways like, yeah I know I've been reading a lot about Keynes, I mean it is sort of interesting that it does mirror... Like this kind of like Great Depression era economic thinking, where you're basically like the market takes care of itself, like don't intervene. In fact, intervening is like the worst possible thing you could do because you're only going to make this worse. And look, I think there's like definitely examples of like kind of like command economy science that like don't work. Yes. But like, you know, like I think most mature people who work in economics would say there's some room for like at least like Guiding the system. Right. And like keeping it like in balance is like [00:19:35] a thing that should be attempted and I think it's kind of like the, the, the argument that I'm making here. Yeah. Yeah. I [00:19:41] Ben: mean, I think that's, [00:19:42] Tim: that's like the meta meta thing. Right. Right. Is even [00:19:46] Ben: what, what level of intervention, like, like what are the ways in which you can like usefully intervene and which, and what are the things that are, that are foolish and kind of. crEate the, the, [00:20:01] Tim: Command economy. That's right. Yeah, exactly. Right. Right. And I think like, I think the way through is, is maybe in the way that I'm talking about, right? Which is like, you can imagine lots of bad things happen when you attempt to pick winners, right? Like maybe the policymaker whoever we want to think of that as like, is it the NSF or NIH or whatever? Like, you know, sitting, sitting in their government bureaucracy, right? Like, are they well positioned to make a choice about who's going to be the right solution to a problem? Maybe yes, maybe no. I think we can have a debate about that, right? But I think there's a totally reasonable position, which is they're not in it, so they're not well positioned to make that call. Yeah. [00:20:35] Right? But, are they well positioned to maybe say, like, if we gave them a dial that was like, we want researchers to be thinking about this time horizon versus that time horizon? Like, that's a control that they actually may be well positioned to inform on. Yeah. As an outsider, right? Yeah. Yeah. And some of this I think, like, I don't know, like, the piece I'm working on right now, which will be coming out probably Tuesday or Wednesday, is you know, some of this is also like encouraging creative destruction, right? Which is like, I'm really intrigued by the idea that like academic fields can get so big that they become they impede progress. Yes. Right? And so this is actually a form of like, I like, it's effectively an intellectual antitrust. Yeah. Where you're basically like, Basically, like the, the role of the scientific regulator is to basically say these fields have gotten so big that they are actively reducing our ability to have good dynamism in the marketplace of ideas. And in this case, we will, we will announce new grant policies that attempt to break this up. And I actually think that like, that is pretty spicy for a funder to do. But like actually maybe part of their role and maybe we should normalize that [00:21:35] being part of their role. Yeah. Yeah, absolutely. [00:21:37] Ben: I I'm imagining a world where There are, where this, like, sort of the macro science is as divisive as [00:21:47] Tim: macroeconomics. [00:21:48] Ben: Right? Because you have, you have your like, your, your like, hardcore free market people. Yeah. Zero government intervention. Yeah, that's right. No antitrust. No like, you know, like abolish the Fed. Right, right. All of that. Yeah, yeah. And I look forward to the day. When there's there's people who are doing the same thing for research. [00:22:06] Tim: Yeah, that's right. Yeah. Yeah when I think that's actually I mean I thought part of a lot of meta science stuff I think is this kind of like interesting tension, which is that like look politically a lot of those people in the space are Pro free market, you know, like they're they're they're liberals in the little L sense. Yeah, like at the same time Like it is true that kind of like laissez faire science Has failed because we have all these examples of like progress slowing down Right? Like, I don't know. Like, I think [00:22:35] that there is actually this interesting tension, which is like, to what degree are we okay with intervening in science to get better outcomes? Yeah. Right? Yeah. Well, as, [00:22:43] Ben: as I, I might put on my hat and say, Yeah, yeah. Maybe, maybe this is, this is me saying true as a fair science has never been tried. Huh, right. Right? Like, that, that, that may be kind of my position. Huh. But anyways, I... And I would argue that, you know, since 1945, we have been, we haven't had laissez faire [00:23:03] Tim: science. Oh, interesting. [00:23:04] Ben: Huh. Right. And so I'm, yeah, I mean, it's like, this is in [00:23:09] Tim: the same way that I think [00:23:11] Ben: a very hard job for macroeconomics is to say, well, like, do we need [00:23:15] Tim: more or less intervention? Yeah. Yeah. [00:23:17] Ben: What is the case there? I think it's the same thing where. You know, a large amount of science funding does come from the government, and the government is opinionated about what sorts of things [00:23:30] Tim: it funds. Yeah, right. Right. And you [00:23:33] Ben: can go really deep into that. [00:23:35] So, so I [00:23:35] Tim: would. Yeah, that's actually interesting. That flips it. It's basically like the current state of science. is right now over regulated, is what you'd say, right? Or, or [00:23:44] Ben: badly regulated. Huh, sure. That is the argument I would say, very concretely, is that it's badly regulated. And, you know, I might almost argue that it is... It's both over and underregulated in the sense that, well, this is, this is my, my whole theory, but like, I think that there, we need like some pockets where it's like much less regulated. Yeah. Right. Where you're, and then some pockets where you're really sort of going to be like, no. You don't get to sort of tune this to whatever your, your project, your program is. Yeah, right, right. You're gonna be working with like [00:24:19] Tim: these people to do this thing. Yeah, yeah. Yeah, and I think there actually is interesting analogies in like the, the kind of like economic regulation, economic governance world. Yeah. Where like the notion is markets generally work well, like it's a great tool. Yeah. Like let it run. [00:24:35] Right. But basically that there are certain failure states that actually require outside intervention. And I think what's kind of interesting in thinking about in like a macro scientific, if you will, context is like, what are those failure states for science? Like, and you could imagine a policy rule, which is the policymaker says, we don't intervene until we see the following signals emerging in a field or in a region. Right. And like, okay, that's, that's the trigger, right? Like we're now in recession mode, you know, like there's enough quarters of this problem of like more papers, but less results. You know, now we have to take action, right? Oh, that's cool. Yeah, yeah. That would be, that would be very interesting. And I think that's like, that's good, because I think like, we end up having to think about like, you know, and again, this is I think why this is a really exciting time, is like MetaScience has produced these really interesting results. Now we're in the mode of like, okay, well, you know, on that policymaker dashboard, Yeah. Right, like what's the meter that we're checking out to basically be like, Are we doing well? Are we doing poorly? Is this going well? Or is this going poorly? Right, like, I think that becomes the next question to like, make this something practicable Yeah. For, for [00:25:35] actual like, Right. Yeah. Yeah. One of my frustrations [00:25:38] Ben: with meta science [00:25:39] Tim: is that it, I [00:25:41] Ben: think is under theorized in the sense that people generally are doing these studies where they look at whatever data they can get. Huh. Right. As opposed to what data should we be looking at? What, what should we be looking for? Yeah. Right. Right. And so, so I would really like to have it sort of be flipped and say, okay, like this At least ideally what we would want to measure maybe there's like imperfect maybe then we find proxies for that Yeah, as opposed to just saying well, like here's what we can measure. It's a proxy for [00:26:17] Tim: okay. That's right, right Yeah, exactly. And I think a part of this is also like I mean, I think it is like Widening the Overton window, which I think like the meta science community has done a good job of is like trying to widen The Overton window of what funders are willing to do. Yeah. Or like what various existing incumbent actors are willing to [00:26:35] do. Because I think one way of getting that data is to run like interesting experiments in this space. Right? Like I think one of the things I'm really obsessed with right now is like, okay, imagine if you could change the overhead rate that universities charge on a national basis. Yeah. Right? Like, what's that do to the flow of money through science? And is that like one dial that's actually like On the shelf, right? Like, we actually have the ability to influence that if we wanted to. Like, is that something we should be running experiments against and seeing what the results are? Yeah, yeah. [00:27:00] Ben: Another would be earmarking. Like, how much money is actually earmarked [00:27:05] Tim: for different things. That's right, yeah, yeah. Like, how easy it is to move money around. That's right, yeah. I heard actually a wild story yesterday about, do you know this whole thing, what's his name? It's apparently a very wealthy donor. That has convinced the state of Washington's legislature to the UW CS department. it's like, it's written into law that there's a flow of money that goes directly to the CS department. I don't think CS departments need more money. I [00:27:35] know, I know, but it's like, this is a really, really kind of interesting, like, outcome. Yeah. Which is like a very clear case of basically just like... Direct subsidy to like, not, not just like a particular topic, but like a particular department, which I think is like interesting experiment. I don't like, I don't know what's been happening there, but yeah. Yeah. Yeah. Natural, natural experiment. [00:27:50] Ben: Totally. Has anybody written down, I assume the answer is no, but it would be very interesting if someone actually wrote down a list of sort of just all the things you [00:28:00] Tim: could possibly [00:28:00] Ben: want to pay attention to, right? Like, I mean, like. Speaking of CS, it'd be very interesting to see, like, okay, like, what fraction of the people who, like, get PhDs in an area, stay in this area, right? Like, going back to the, the [00:28:15] Tim: health of a field or something, right? Yeah, yeah. I think that's right. I, yeah. And I think that those, those types of indicators are interesting. And then I think also, I mean, in the spirit of like it being a dynamic system. Like, so a few years back I read this great bio by Sebastian Malaby called The Man Who Knew, which is, it's a bio of Alan Greenspan. So if you want to ever read, like, 800 pages about [00:28:35] Alan Greenspan, book for you. It's very good. But one of the most interesting parts about it is that, like, there's a battle when Alan Greenspan becomes head of the Fed, where basically he's, like, extremely old school. Like, what he wants to do is he literally wants to look at, like, Reams of data from like the steel industry. Yeah, because that's kind of got his start And he basically is at war with a bunch of kind of like career People at the Fed who much more rely on like statistical models for predicting the economy And I think what's really interesting is that like for a period of time actually Alan Greenspan has the edge Because he's able to realize really early on that like there's It's just changes actually in like the metabolism of the economy that mean that what it means to raise the interest rate or lower the interest rate has like very different effects than it did like 20 years ago before it got started. Yeah. And I think that's actually something that I'm also really quite interested in science is basically like When we say science, people often imagine, like, this kind of, like, amorphous blob. But, like, I think the metabolism is changing all the [00:29:35] time. And so, like, what we mean by science now means very different from, like, what we mean by science, like, even, like, 10 to 20 years ago. Yes. And, like, it also means that all of our tactics need to keep up with that change, right? And so, one of the things I'm interested in to your question about, like, has anyone compiled this list of, like, science health? Or the health of science, right? It's maybe the right way of thinking about it. is that, like, those indicators may mean very different things at different points in time, right? And so part of it is trying to understand, like, yeah, what is the state of the, what is the state of this economy of science that we're talking about? Yeah. You're kind of preaching [00:30:07] Ben: to the, to the choir. In the sense that I'm, I'm always, I'm frustrated with the level of nuance that I feel like many people who are discussing, like, science, quote, making air quotes, science and research, are, are talking about in the sense that. They very often have not actually like gone in and been part of the system. Huh, right. And I'm, I'm open to the fact that [00:30:35] you [00:30:35] Tim: don't need to have got like [00:30:36] Ben: done, been like a professional researcher to have an opinion [00:30:41] Tim: or, or come up with ideas about it. [00:30:43] Ben: Yeah. But at the same time, I feel like [00:30:46] Tim: there's, yeah, like, like, do you, do you think about that tension at all? Yeah. I think it's actually incredibly valuable. Like, I think So I think of like Death and Life of Great American Cities, right? Which is like, the, the, the really, one of the really, there's a lot of interesting things about that book. But like, one of the most interesting things is sort of the notion that like, you had a whole cabal of urban planners that had this like very specific vision about how to get cities to work right and it just turns out that like if you like are living in soho at a particular time and you like walk along the street and you like take a look at what's going on like there's always really actually super valuable things to know about yeah that like are only available because you're like at that like ultra ultra ultra ultra micro level and i do think that there's actually some potential value in there like one of the things i would love to be able to set up, like, in the community of MetaScience or whatever you want to call it, right, [00:31:35] is the idea that, like, yeah, you, you could afford to do, like, very short tours of duty, where it's, like, literally, you're just, like, spending a day in a lab, right, and, like, to have a bunch of people go through that, I think, is, like, really, really helpful and so I think, like, thinking about, like, what the rotation program for that looks like, I think would be cool, like, you, you should, you should do, like, a six month stint at the NSF just to see what it looks like. Cause I think that kind of stuff is just like, you know, well, A, I'm selfish, like I would want that, but I also think that like, it would also allow the community to like, I think be, be thinking about this in a much more applied way. Yeah. Yeah. Yeah. [00:32:08] Ben: I think it's the, the meta question there for, for everything, right? Is how much in the weeds, like, like what am I trying to say? The. It is possible both to be like two in the weeds. Yeah, right and then also like too high level Yeah, that's right. And in almost like what what is the the right amount or like? Who, who should [00:32:31] Tim: be talking to whom in that? That's right. Yeah, I mean, it's like what you were saying earlier that like the [00:32:35] success of macro science will be whether or not it's as controversial as macroeconomics. It's like, I actually hope that that's the case. It's like people being like, this is all wrong. You're approaching it like from a too high level, too abstract of a level. Yeah. I mean, I think the other benefit of doing this outside of like the level of insight is I think one of the projects that I think I have is like We need to, we need to be like defeating meta science, like a love of meta science aesthetics versus like actual like meta science, right? Like then I think like a lot of people in meta science love science. That's why they're excited to not talk about the specific science, but like science in general. But like, I think that intuition also leads us to like have very romantic ideas of like what science is and how science should look and what kinds of science that we want. Yeah. Right. The mission is progress. The mission isn't science. And so I think, like, we have to be a lot more functional. And again, I think, like, the benefit of these types of, like, rotations, like, Oh, you just are in a lab for a month. Yeah. It's like, I mean, you get a lot more of a sense of, like, Oh, okay, this is, this is what it [00:33:35] looks like. Yeah. Yeah. I'd like to do the same thing for manufacturing. Huh. Right. [00:33:39] Ben: Right. It's like, like, and I want, I want everybody to be rotating, right? Huh. Like, in the sense of, like, okay, like, have the scientists go and be, like, in a manufacturing lab. That's right. [00:33:47] Tim: Yeah. [00:33:48] Ben: And be like, okay, like, look. Like, you need to be thinking about getting this thing to work in, like, this giant, like, flow pipe instead of a [00:33:54] Tim: test tube. That's right, right. Yeah, yeah, yeah. Yeah, [00:33:57] Ben: unfortunately, the problem is that we can't all spend our time, like, if everybody was rotating through all the [00:34:03] Tim: things they need to rotate, we'd never get anything done. Yeah, exactly. [00:34:06] Ben: ANd that's, that's, that's kind of [00:34:08] Tim: the problem. Well, and to bring it all the way back, I mean, I think you started this question on macroscience in the context of transitioning away from all of this like weird Cuban Missile Crisis simulation stuff. Like, I do think one way of thinking about this is like, okay, well, if we can't literally send you into a lab, right? Like the question is like, what are good simulations to give people good intuitions about the dynamics in the space? Yeah. And I think that's, that's potentially quite interesting. Yeah. Normalized weekend long simulation. That's right. Like I love the idea of basically [00:34:35] like like you, you get to reenact the publication of a prominent scientific paper. It's like kind of a funny idea. It's just like, you know, yeah. Or, or, or even trying to [00:34:44] Ben: get research funded, right? Like, it's like, okay, like you have this idea, you want yeah. [00:34:55] Tim: I mean, yeah, this is actually a project, I mean, I've been talking to Zach Graves about this, it's like, I really want to do one which is a game that we're calling Think Tank Tycoon, which is basically like, it's a, it's a, the idea would be for it to be a strategy board game that simulates what it's like to run a research center. But I think like to broaden that idea somewhat like it's kind of interesting to think about the idea of like model NSF Yeah, where you're like you you're in you're in the hot seat you get to decide how to do granting Yeah, you know give a grant [00:35:22] Ben: a stupid thing. Yeah, some some some congressperson's gonna come banging [00:35:26] Tim: on your door Yeah, like simulating those dynamics actually might be really really helpful Yeah I mean in the very least even if it's not like a one for one simulation of the real world just to get like some [00:35:35] common intuitions about like The pressures that are operating here. I [00:35:38] Ben: think you're, the bigger point is that simulations are maybe underrated [00:35:42] Tim: as a teaching tool. I think so, yeah. Do you remember the the paperclip maximizer? Huh. The HTML game? Yeah, yeah. [00:35:48] Ben: I'm, I'm kind of obsessed with it. Huh. Because, it, you've, like, somehow the human brain, like, really quickly, with just, like, you know, some numbers on the screen. Huh. Like, just like numbers that you can change. Right, right. And some, like, back end. Dynamic system, where it's like, okay, like based on these numbers, like here are the dynamics of the [00:36:07] Tim: system, and it'll give you an update. [00:36:09] Ben: Like, you start to really get an intuition for, for system dynamics. Yeah. And so, I, I, I want to see more just like plain HTML, like basically like spreadsheet [00:36:20] Tim: backend games. Right, right, like the most lo fi possible. Yeah, I think so. Yeah. Yeah, I think it's helpful. I mean, I think, again, particularly in a world where you're thinking about, like, let's simulate these types of, like, weird new grant structures that we might try out, right? Like, you know, we've got a bunch [00:36:35] of hypotheses. It's kind of really expensive and difficult to try to get experiments done, right? Like, does a simulation with a couple people who are well informed give us some, at least, inclinations of, like, where it might go or, like, what are the unintentional consequences thereof? Yeah. [00:36:51] Ben: Disciplines besides the military that uses simulations [00:36:56] Tim: successfully. Not really. And I think what's kind of interesting is that like, I think it had a vogue that like has kind of dissipated. Yeah, I think like the notion of like a a game being the way you kind of do like understanding of a strategic situation, I think like. Has kind of disappeared, right? But like, I think a lot of it was driven, like, RAND actually had a huge influence, not just on the military. But like, there's a bunch of corporate games, right? That were like, kind of invented in the same period. Yeah. That are like, you determine how much your steel production is, right? And was like, used to teach MBAs. But yeah, I think it's, it's been like, relatively limited. Hm. [00:37:35] Yeah. It, yeah. Hm. [00:37:38] Ben: So. Other things. Huh. Like, just to, [00:37:41] Tim: to shift together. Sure, sure, go ahead. Yeah, yeah, yeah, yeah. I guess another [00:37:44] Ben: thing that we haven't really talked about, but actually sort of plays into all of this, is thinking about better [00:37:50] Tim: ways of regulating technology. [00:37:52] Ben: I know that you've done a lot of thinking about that, and maybe this is another thing to simulate. [00:38:00] Tim: Yeah, it's a model OSTP. But [00:38:04] Ben: it's maybe a thing where, this is actually like a prime example where the particulars really matter, right? Where you can't just regulate. quote unquote technology. Yeah. Right. And it's like, there's, there's some technologies that you want to regulate very, very closely and very tightly and others that you want to regulate very [00:38:21] Tim: loosely. Yeah, I think that's right. And I think that's actually, you know, I think it is tied to the kind of like macro scientific project, if you will. Right. Which is that I think we have often a notion of like science regulation being like. [00:38:35] literally the government comes in and is like, here are the kind of constraints that we want to put on the system. Right. And there's obviously like lots of different ways of doing that. And I think there's lots of contexts in which that's like appropriate. But I think for a lot of technologies that we confront right now, the change is so rapid that the obvious question always becomes, no matter what emerging technology talking about is like, how does your clock speed of regulation actually keep up with like the clock speed of technology? And the answer is frequently like. It doesn't, right? And like you run into these kind of like absurd situations where you're like, well, we have this thing, it's already out of date by the time it goes into force, everybody kind of creates some like notional compliance with that rule. Yeah. And like, in terms of improving, I don't know, safety outcomes, for instance, it like has not actually improved safety outcomes. And I think in that case, right, and I think I could actually make an argument that like, the problem is becoming more difficult with time. Right? Like, if you really believe that the pace of technological change is faster than it used to be, then it is possible that, like, there was a point at which, like, government was operating, and it could actually keep [00:39:35] pace effectively, or, like, a body like Congress could actually keep pace with society, or with technology successfully, to, like, make sure that it was conformant with, sort of, like, societal interests. Do you think that was [00:39:46] Ben: actually ever the case, or was it that we didn't, we just didn't [00:39:50] Tim: have as many regulations? I would say it was sort of twofold, right? Like, I think one of them was you had, at least, let's just talk about Congress, right? It's really hard to talk about, like, government as a whole, right? Like, I think, like, Congress was both better advised and was a more efficient institution, right? Which means it moved faster than it does today. Simultaneously, I also feel like for a couple reasons we can speculate on, right? Like, science, or in the very least, technology. Right, like move slower than it does today. Right, right. And so like actually what has happened is that both both dynamics have caused problems, right? Which is that like the organs of government are moving slower at the same time as science is moving faster And like I think we've passed some inflection [00:40:35] point now where like it seems really hard to craft You know, let's take the AI case like a sensible framework that would apply You know, in, in LLMs where like, I don't know, like I was doing a little recap of like recent interoperability research and I like took a step back and I was like, Oh, all these papers are from May, 2023. And I was like, these are all big results. This is all a big deal. Right. It's like very, very fast. Yeah. So that's kind of what I would say to that. Yeah. I don't know. Do you feel differently? You feel like Congress has never been able to keep up? Yeah. [00:41:04] Ben: Well, I. I wonder, I guess I'm almost, I'm, I'm perhaps an outlier in that I am skeptical of the claim that technology overall has sped up significantly, or the pace of technological change, the pace of software change, certainly. Sure. Right. And it's like maybe software as a, as a fraction of technology has spread up, sped up. And maybe like, this is, this is a thing where like to the point of, of regulations needing to, to. Go into particulars, [00:41:35] right? Mm-Hmm. . Right, right. Like tuning the regulation to the characteristic timescale of whatever talk [00:41:40] Tim: technology we're talking about. Mm-Hmm. , right? [00:41:42] Ben: But I don't know, but like, I feel like outside of software, if anything, technology, the pace of technological change [00:41:52] Tim: has slowed down. Mm hmm. Right. Right. Yeah. [00:41:55] Ben: This is me putting on my [00:41:57] Tim: stagnationist bias. And would, given the argument that I just made, would you say that that means that it should actually be easier than ever to regulate technology? Yeah, I get targets moving slower, right? Like, yeah, [00:42:12] Ben: yeah. Or it's the technology moving slowly because of the forms of [00:42:14] Tim: the regulator. I guess, yeah, there's like compounding variables. [00:42:16] Ben: Yeah, the easiest base case of regulating technology is saying, like, no, you can't have [00:42:20] Tim: any. Huh, right, right, right. Like, it can't change. Right, that's easy to regulate. Yeah, right, right. That's very easy to regulate. I buy that, I buy that. It's very easy to regulate well. Huh, right, right. I think that's [00:42:27] Ben: That's the question. It's like, what do we want to lock in and what don't we [00:42:31] Tim: want to lock in? Yeah, I think that's right and I think, you [00:42:35] know I guess what that moves me towards is like, I think some people, you know, will conclude the argument I'm making by saying, and so regulations are obsolete, right? Or like, oh, so we shouldn't regulate or like, let the companies take care of it. And I'm like, I think so, like, I think that that's, that's not the conclusion that I go to, right? Like part of it is like. Well, no, that just means we need, we need better ways of like regulating these systems, right? And I think they, they basically require government to kind of think about sort of like moving to different parts of the chain that they might've touched in the past. Yeah. So like, I don't know, we, Caleb and I over at IFP, we just submitted this RFI to DARPA. In part they, they were thinking about like how does DARPA play a role in dealing with like ethical considerations around emerging technologies. Yep. But the deeper point that we were making in our submission. was simply that like maybe actually science has changed in a way where like DARPA can't be the or it's harder for DARPA to be the originator of all these technologies. Yeah. So they're, they're almost, they're, they're placing the, the, the ecosystem, the [00:43:35] metabolism of technology has changed, which requires them to rethink like how they want to influence the system. Yeah. Right. And it may be more influence at the point of like. Things getting out to market, then it is things like, you know, basic research in the lab or something like that. Right. At least for some classes of technology where like a lot of it's happening in private industry, like AI. Yeah, exactly. Yeah. [00:43:55] Ben: No, I, I, I think the, the concept of, of like the metabolism of, of science and technology is like really powerful. I think in some sense it is, I'm not sure if you would, how would you map that to the idea of there being a [00:44:11] Tim: research ecosystem, right? Right. Is it, is it that there's like [00:44:17] Ben: the metabolic, this is, this is incredibly abstract. Okay. Like, is it like, I guess if you're looking at the metabolism, does, does the metabolism sort of say, we're going to ignore institutions for now and the metabolism is literally just the flow [00:44:34] Tim: of [00:44:35] like ideas and, and, and outcomes and then maybe like the ecosystem is [00:44:41] Ben: like, okay, then we like. Sort of add another layer and say there are institutions [00:44:46] Tim: that are sure interacting with this sort of like, yeah, I think like the metabolism view or, you know, you might even think about it as like a supply chain view, right? To move it away from, like, just kind of gesturing at bio for no reason, right? Is I think what's powerful about it is that, you know, particularly in foundation land, which I'm most familiar with. There's a notion of like we're going to field build and what that means is we're going to name a field and then researchers Are going to be under this tent that we call this field and then the field will exist Yeah, and then the proper critique of a lot of that stuff is like researchers are smart They just like go where the money is and they're like you want to call up like I can pretend to be nanotech for a Few years to get your money Like, that's no problem. I can do that. And so there's kind of a notion that, like, if you take the economy of science as, like, institutions at the very beginning, you actually miss the bigger [00:45:35] picture. Yes. Right? And so the metabolism view is more powerful because you literally think about, like, the movement of, like, an idea to an experiment to a practical technology to, like, something that's out in the world. Yeah. And then we basically say, how do we influence those incentives before we start talking about, like, oh, we announced some new policy that people just, like... Cosmetically align their agendas to yeah, and like if you really want to shape science It's actually maybe arguably less about like the institution and more about like Yeah, the individual. Yeah, exactly. Like I run a lab. What are my motivations? Right? And I think this is like, again, it's like micro macro, right? It's basically if we can understand that, then are there things that we could do to influence at that micro level? Yeah, right. Which is I think actually where a lot of Macro econ has moved. Right. Which is like, how do we influence like the individual firm's decisions Yeah. To get the overall aggregate change that we want in the economy. Yeah. And I think that's, that's potentially a better way of approaching it. Right. A thing that I desperately [00:46:30] Ben: want now is Uhhuh a. I'm not sure what they're, they're [00:46:35] actually called. Like the, you know, like the metal, like, like, like the [00:46:37] Tim: prep cycle. Yeah, exactly. Like, like, like the giant diagram of, of like metabolism, [00:46:43] Ben: right. I want that for, for research. Yeah, that would be incredible. Yeah. If, if only, I mean, one, I want to have it on [00:46:50] Tim: my wall and to, to just get across the idea that. [00:46:56] Ben: It is like, it's not you know, basic research, applied [00:47:01] Tim: research. Yeah, totally. Right, right, right. When it goes to like, and what I like about kind of metabolism as a way of thinking about it is that we can start thinking about like, okay, what's, what's the uptake for certain types of inputs, right? We're like, okay, you know like one, one example is like, okay, well, we want results in a field to become more searchable. Well what's really, if you want to frame that in metabolism terms, is like, what, you know, what are the carbs that go into the system that, like, the enzymes or the yeast can take up, and it's like, access to the proper results, right, and like, I think that there's, there's a nice way of flipping in it [00:47:35] that, like, starts to think about these things as, like, inputs, versus things that we do, again, because, like, we like the aesthetics of it, like, we like the aesthetics of being able to find research results instantaneously, but, like, the focus should be on, Like, okay, well, because it helps to drive, like, the next big idea that we think will be beneficial to me later on. Or like, even being [00:47:53] Ben: the question, like, is the actual blocker to the thing that you want to see, the thing that you think it is? Right. I've run into far more people than I can count who say, like, you know, we want more awesome technology in the world, therefore we are going to be working on Insert tool here that actually isn't addressing, at least my, [00:48:18] Tim: my view of why those things aren't happening. Yeah, right, right. And I think, I mean, again, like, part of the idea is we think about these as, like, frameworks for thinking about different situations in science. Yeah. Like, I actually do believe that there are certain fields because of, like, ideologically how they're set up, institutionally how [00:48:35] they're set up, funding wise how they're set up. that do resemble the block diagram you were talking about earlier, which is like, yeah, there actually is the, the basic research, like we can put, that's where the basic research happens. You could like point at a building, right? And you're like, that's where the, you know, commercialization happens. We pointed at another building, right? But I just happen to think that most science doesn't look like that. Right. And we might ask the question then, like, do we want it to resemble more of like the metabolism state than the block diagram state? Right. Like both are good. Yeah, I mean, I would [00:49:07] Ben: argue that putting them in different buildings is exactly what's causing [00:49:10] Tim: all the problems. Sure, right, exactly, yeah, yeah. Yeah. But then, again, like, then, then I think, again, this is why I think, like, the, the macro view is so powerful, at least to me, personally, is, like, we can ask the question, for what problems? Yeah. Right? Like, are there, are there situations where, like, that, that, like, very blocky way of doing it serves certain needs and certain demands? Yeah. And it's like, it's possible, like, one more argument I can make for you is, like, Progress might be [00:49:35] slower, but it's a lot more controllable. So if you are in the, you know, if you think national security is one of the most important things, you're willing to make those trade offs. But I think we just should be making those trade offs, like, much more consciously than we do. And [00:49:49] Ben: that's where politics, in the term, in the sense of, A compromise between people who have different priorities on something can actually come in where we can say, okay, like we're going to trade off, we're going to say like, okay, we're going to increase like national security a little bit, like in, in like this area to, in compromise with being able to like unblock this. [00:50:11] Tim: That's right. Yeah. And I think this is the benefit of like, you know, when I say lever, I literally mean lever, right. Which is basically like, we're in a period of time where we need this. Yeah. Right? We're willing to trade progress for security. Yeah. Okay, we're not in a period where we need this. Like, take the, take, ramp it down. Right? Like, we want science to have less of this, this kind of structure. Yeah. That's something we need to, like, have fine tuned controls over. Right? Yeah. And to be thinking about in, like, a, a comparative sense, [00:50:35] so. And, [00:50:36] Ben: to, to go [00:50:36] Tim: back to the metabolism example. Yeah, yeah. I'm really thinking about it. Yeah, yeah. [00:50:39] Ben: Is there an equivalent of macro for metabolism in the sense that like I'm thinking about like, like, is it someone's like blood, like, you know, they're like blood glucose level, [00:50:52] Tim: like obesity, right? Yeah, right. Kind of like our macro indicators for metabolism. Yeah, that's right. Right? Or like how you feel in the morning. That's right. Yeah, exactly. I'm less well versed in kind of like bio and medical, but I'm sure there is, right? Like, I mean, there is the same kind of like. Well, I study the cell. Well, I study, you know, like organisms, right? Like at different scales, which we're studying this stuff. Yeah. What's kind of interesting in the medical cases, like You know, it's like, do we have a Hippocratic, like oath for like our treatment of the science person, right? It's just like, first do no harm to the science person, you know? [00:51:32] Ben: Yeah, I mean, I wonder about that with like, [00:51:35] with research. Mm hmm. Is there, should we have more heuristics about how we're [00:51:42] Tim: Yeah, I mean, especially because I think, like, norms are so strong, right? Like, I do think that, like, one of the interesting things, this is one of the arguments I was making in the long science piece. It's like, well, in addition to funding certain types of experiments, if you proliferate the number of opportunities for these low scale projects to operate over a long period of time, there's actually a bunch of like norms that might be really good that they might foster in the scientific community. Right. Which is like you learn, like scientists learn the art of how to plan a project for 30 years. That's super important. Right. Regardless of the research results. That may be something that we want to put out into the open so there's more like your median scientist has more of those skills Yeah, right, like that's another reason that you might want to kind of like percolate this kind of behavior in the system Yeah, and so there's kind of like these emanating effects from like even one offs that I think are important to keep in mind [00:52:33] Ben: That's actually another [00:52:35] I think used for simulations. Yeah I'm just thinking like, well, it's very hard to get a tight feedback loop, right, about like whether you manage, you planned a project for 30 years [00:52:47] Tim: well, right, [00:52:48] Ben: right. But perhaps there's a better way of sort of simulating [00:52:51] Tim: that planning process. Yeah. Well, and I would love to, I mean, again, to the question that you had earlier about like what are the metrics here, right? Like I think for a lot of science metrics that we may end up on, they may have these interesting and really curious properties like we have for inflation rate. Right. We're like, the strange thing about inflation is that we, we kind of don't like, we have hypotheses for how it happens, but like, part of it is just like the psychology of the market. Yeah. Right. Like you anticipate prices will be higher next quarter. Inflation happens if enough people believe that. And part of what the Fed is doing is like, they're obviously making money harder to get to, but they're also like play acting, right? They're like. You know, trust me guys, we will continue to put pressure on the economy until you feel differently about this. And I think there's going to be some things in science that are worth [00:53:35] measuring that are like that, which is like researcher perceptions of the future state of the science economy are like things that we want to be able to influence in the space. And so one of the things that we do when we try to influence like the long termism or the short termism of science It's like, there's lots of kind of like material things we do, but ultimately the idea is like, what does that researcher in the lab think is going to happen, right? Do they think that, you know, grant funding is going to become a lot less available in the next six months or a lot more available in the next six months? Like influencing those might have huge repercussions on what happens in science. And like, yeah, like that's a tool that policymakers should have access to. Yeah. Yeah. [00:54:11] Ben: And the parallels between the. The how beliefs affect the economy, [00:54:18] Tim: and how beliefs [00:54:19] Ben: affect science, I think may also be a [00:54:21] Tim: little bit underrated. Yeah. In the sense that, [00:54:24] Ben: I, I feel like some people think that It's a fairly deterministic system where it's like, ah, yes, this idea's time has come. And like once, once all the things that are in place, like [00:54:35] once, once all, then, then it will happen. And like, [00:54:38] Tim: that is, that's like how it works. [00:54:40] Ben: Which I, I mean, I have, I wish there was more evidence to my point or to disagree with me. But like, I, I think that's, that's really not how it works. And I'm like very often. a field or, or like an idea will, like a technology will happen because people think that it's time for that technology to happen. Right. Right. Yeah. Obviously, obviously that isn't always the case. Right. Yeah. Yeah. There's, there's, there's hype [00:55:06] Tim: cycles. And I think you want, like, eventually, like. You know, if I have my druthers, right, like macro science should have like it's Chicago school, right? Which is basically like the idea arrives exactly when it should arrive. Scientists will discover it on exactly their time. And like your only role as a regulator is to ensure the stability of scientific institutions. I think actually that that is a, that's not a position I agree with, but you can craft a totally, Reasonable, coherent, coherent governance framework that's based around that concept, right? Yes. Yeah. I think [00:55:35] like [00:55:35] Ben: you'll, yes. I, I, I think like that's actually the criteria for success of meta science as a field uhhuh, because like once there's schools , then, then, then it will have made it, [00:55:46] Tim: because [00:55:47] Ben: there aren't schools right now. Mm-Hmm. , like, I, I feel , I almost feel I, I, I now want there to be schools because. I want a, a better thing to, to craft my disagreements with people on. [00:55:56] Tim: Right. [00:55:56] Ben: Right. And be like, Oh, like, you know, right now it's, it's like individual people. That's right. Yeah. So it's like, I [00:56:02] Tim: want, I want some team. Yeah. I think, I don't know. I think so one of my favorite museums in the world is this museum called the Pitt Rivers Museum, which is in Oxford. It's like, it's preserved like many things at Oxford from like when it was first founded in whatever century it was. And what's great about it is that you walk into it and you're like, what is this? Like it builds itself as a museum, but it's just like a closet of stuff that this guy collected and it's basically like this early I'm like, yeah, this is the early phase of every Science or every field. Yeah, it's like you're we're still in the phase of like that's interesting. I guess I'll put it in the [00:56:35] box That's interesting. I guess I'll put it in my back and we're just collecting at the moment, right? Yeah, but I think like, you know, you can only you can only do that for so long, right? Ultimately, you have to have a point of view because if it's gonna be more than a purely observational field It's gonna be a thing that actually should inform science policymaking Yeah, it has to come with some normative judgments that we're not gonna always have empirical results for And part of it is, like, these really hard to deal with questions epistemologically of, like, does science discover the idea, like, immediately upon all the resources being available? Or are there, like, lots of provisionalities to science that would require intervention? There's no way of proving that's a really hard thing to prove or disprove. It ends up being a matter of, like, what's the philosophy that will dominate how... Like science planners think about the issue. [00:57:35]

Reach and audience

Public platform figures. Ratings count people who left a rating, not total listeners.

Apple Podcasts (US)

4.7 / 5

13 ratings

Spotify

5.0 / 5

1 ratings

Contact Idea Machines

Guest appearances

Does not typically book guests

Based on episode analysis; this does not confirm that the show is currently accepting guests.

Shows with a similar audience

• 

  The Foresight Institute Podcast

• 

  Conversations with Tyler

• 

  Dwarkesh Podcast

• 

  a16z Podcast

• 

  Invest Like the Best with Patrick O'Shaughnessy

• 

  Odd Lots