STFC-RAL-CR03 R61: But my background is in physics. And so actually in, you know, a lot of the. So I hope today to be able to put a lot of work that we do some context that might be useful for you guys. And then
STFC-RAL-CR03 R61: one of the interesting things that I'm gonna talk about today is that computing.
STFC-RAL-CR03 R61: So science has a massive computational problem. It's existed in particle physics for a long, long time. In fact, a lot of the technologies that
STFC-RAL-CR03 R61: use that talk about are either used by cern or come out of cern. So it's a big driver on the kind of data front. But I'm gonna focus on what we've actually done, which is to build this for bioscience. So talk a little bit about the data landscape of bioscience right now, and kind of what the main sort of challenges are facing.
STFC-RAL-CR03 R61: So I've got some computational example challenges in bioscience.
STFC-RAL-CR03 R61: Then I'm going to talk about some specific examples that we have Franklin and then I'm going to talk about how our platform meets our challenges. And lastly, and just a few slides on how we work together. Because when you give a seminar, it's always nice to try to find collaborations and space
STFC-RAL-CR03 R61: so computational challenges and violence.
STFC-RAL-CR03 R61: Bioscience has come a long way last 10 years, especially with the advent of prior electronic and data volumes have just slowed. So one of the grand challenges in in bioscience is to actually
STFC-RAL-CR03 R61: find out what's happening. Natively, biological organisms are incredibly complex. And
STFC-RAL-CR03 R61: as a non-biologist working in biology, I'm kind of amazed. It actually works for, given how complex and how difficult science is to do.
STFC-RAL-CR03 R61: And and one of the desires is, you know, when we.
STFC-RAL-CR03 R61: the last big acceleration of of bioscience and structural biology. Right? You know, diamond across the road does crystallography in a big regularity. And that's allowed us to find the structure of lots of different seasons most recently covid. But actually, one of the problems with crystallography is
STFC-RAL-CR03 R61: you're locking your proteins into a structure such that they can be crystallized and proteins by their very nature, flexible architecture, and they are designed to move around the various jobs within the cell. So actually, what you want to do is you want to image these things natively
STFC-RAL-CR03 R61: in their home? And one of the things we want to do at Franklin is going from himself. So here we can see
STFC-RAL-CR03 R61: an alpha cell. And this is part of the Golgi apparatus which sits in the cell membrane and does something to do the surface. I'll ask my colleagues more on that one to the tissue. So this is actually a bunch of cells. This is not tissue, but you can actually go up the length scales. And then you can also couple this with sort of molecular dynamics.
STFC-RAL-CR03 R61: and looking at how proteins that sit in and-, and what kind of forms they- they take.
STFC-RAL-CR03 R61: Sorry.
STFC-RAL-CR03 R61: What this has resulted in really is a massive explosion of data within, and that is cool with
STFC-RAL-CR03 R61: a whole lot of scientists who are massively skilled in laboratory work which is extremely difficult to do these proem. I'll go through the cryoem process later. But to do these samples is extremely hard to do proteomics, and a lot of the work we do in Franklin is very hard. One thing that you don't know anything about is you don't know about computation. You don't know about Htc. And so when I 1st asked for Franklin.
STFC-RAL-CR03 R61: I remember a Phd. Student coming up to me and going. I've just collected 20 TB of data, and I can't open it. Well, 2020 TB is a lot. And they went. Oh, but that's not really a big scam. So then we had to sit down with them and try to understand. You know.
STFC-RAL-CR03 R61: do you really need 20 TB of data is this, this is what you what you want and and how are we going to to work out?
STFC-RAL-CR03 R61: Process is 20 TB per second. But this is a little bit of a
STFC-RAL-CR03 R61: sort of view of the Franklin. So this is just a final plot over this is in actual months and days. So it's a bit weird. So it's a it's 6 months of last year.
STFC-RAL-CR03 R61: And this is where we hit our 1st petabyte of a Franco. We had a big cake. Actually, we celebrated and had a petabyte party when we collected our 1st petabyte. It took us 5 years to get to a petabyte in Franklin so pretty small fry when we compare it to particle physics. And we're now at 1.6 petabytes within a year. So we almost increased our data rate, 60% within a year over the previous 5.
STFC-RAL-CR03 R61: So this is present. Quite a large computational challenge. Also, one of the reasons we've managed to increase. So much is because of the work that we've done on making these data pipelines easier for people to use. And that's accelerated the amount of data they're able to deposit.
STFC-RAL-CR03 R61: This is an example from the Uk. Biobank. So Uk Biobank was in the news recently about the new. I think it is a 10,000 more patient samples in Uk. Biobank has a huge amount of data. It's really going to be variable. And it's grown over the years. It's currently 30, and it's expected to grow to 40.
STFC-RAL-CR03 R61: And these large scale repositories are not in common with websites. So when one of these island.
STFC-RAL-CR03 R61: you need to make your data public
STFC-RAL-CR03 R61: because you shouldn't be repeating and expounding on an animal that has already been done.
STFC-RAL-CR03 R61: So you need to make sure that that data says there, so that we can reduce our reliance on work.
STFC-RAL-CR03 R61: And so, and repositories, vast data set
STFC-RAL-CR03 R61: people that we use are very, very powerful in in biology and something that we want to continue building. And so how we get all of this data or the relevant data from here into a similar repository to Biobank is also a big question, and that we work on.
STFC-RAL-CR03 R61: So yeah, big challenge, massive, massive challenge.
STFC-RAL-CR03 R61: The second challenge, which I think physicists we're probably
STFC-RAL-CR03 R61: less surprised by is new methods that require advanced.
STFC-RAL-CR03 R61: So you know, you're talking to a community that may have
STFC-RAL-CR03 R61: worked in crystallography, and what you get is your structure. Back you go. Do experiment. Get your structure back. The computations are handled. You might be doing stuff in excel spreadsheets.
STFC-RAL-CR03 R61: You might be doing something. R. Maybe. But you would, never having to run things with Gpu. You were never having to write Gpu accelerated code. A lot of people in the space. Can't write code, don't know what Gpu is. And so you're very dependent on gui applications to allow biologists to run their pipelines.
STFC-RAL-CR03 R61: So you know, I've taken some examples here. So one big one is is out of fault, too. So
STFC-RAL-CR03 R61: Alpha Fold has really disruptive bioscience. You know, it solved the problem of protein fold inputs or protein stretch prediction. We just didn't understand it. So I've mentioned crystallography many times during this talk already. But one of the large problems is some proteins don't crystallize.
STFC-RAL-CR03 R61: and you need to know their structure. So the fact you can predict it of their sequence is saves you a huge amount of experiment time and can inform you on things like drug development, or all of the kind of mechanism is hugely used in the biology community.
STFC-RAL-CR03 R61: However, it needs at least one ability to do.
STFC-RAL-CR03 R61: Of RAM.
STFC-RAL-CR03 R61: It needs at least a hundred 28 GB of of Vram, at least CPU, with at least 36 available cores.
STFC-RAL-CR03 R61: and you know, quite a large SSD drive. Now, if you're buying that computer, that's very hefty bit
STFC-RAL-CR03 R61: and so and you've also got to understand how to deploy such a thing, or no. So I think, Alpha 2 is actually installed by now. But you know you have to have to understand your drivers and things like this. So if you want to run out of fault to someone who doesn't know anything about computers, the 1st thing you have to do is find somebody who does know something about computers to help you get started.
STFC-RAL-CR03 R61: Then we've got generative. AI. So this is a very interesting space. And this is where I think there's quite a lot of collaboration between the software world and the AI world. And the biological world. And that is because
STFC-RAL-CR03 R61: in biology you're often looking at very small data sets. And we'll we'll go into
STFC-RAL-CR03 R61: that area. So biological data is often hard to capture
STFC-RAL-CR03 R61: some data set to massive genomic sequence data, you know, that's massive. You have lots of examples.
STFC-RAL-CR03 R61: But other things on both seasons on, you know, rare, rare, recurring proteins. It's much harder to find that data. And
STFC-RAL-CR03 R61: it's often something you're interested in studying Zoom.
STFC-RAL-CR03 R61: generative AI. And and then also, you have this kind of hospital application. Generative AI, which is.
STFC-RAL-CR03 R61: you don't really want to have to put people through
STFC-RAL-CR03 R61: invasive imaging procedures right? So you don't want to have people do like 5 or 6 x rays. So they just do. One don't have to put people through Ct scans. If you could just give them an X-ray so generative AI is used to try to get
STFC-RAL-CR03 R61: is to try to make more patient candidates, to train AI models to do, to do. Prediction of health outcomes so generative. AI is
STFC-RAL-CR03 R61: getting larger in the in the biological space.
STFC-RAL-CR03 R61: Again, you need a high end, Gpu, to run this fast storage. You need lots of RAM.
STFC-RAL-CR03 R61: And again, this pretty computer that you need to do. And actually.
STFC-RAL-CR03 R61: you know, this, this space needs both the main knowledge and competition, knowledge to do so.
STFC-RAL-CR03 R61: These 2 have
STFC-RAL-CR03 R61: that needs to be done in collaboration. And then the last one is the so I mean, everyone seems to be.
STFC-RAL-CR03 R61: and biological sciences are different, and we have the space of sequence analysis, which is something tokenize, and these provide very interesting methods in looking at Bioinfortics, and this is a very nascent world. Now, I actually haven't booked the computation against Llm. Mainly because of this Obama project, which is allowing you to compress these things to a micro laptop. But
STFC-RAL-CR03 R61: you know, if you want to, this is only inference here.
STFC-RAL-CR03 R61: If you want to train these things, you need vast amounts, vast, vast amounts of data. Yeah.
STFC-RAL-CR03 R61: So challenge number 2
STFC-RAL-CR03 R61: is more and more complex to be used. And usually the same specialists don't know specifically.
STFC-RAL-CR03 R61: One thing I would say, which big elephant in the room is cost.
STFC-RAL-CR03 R61: You know, I've mentioned this pretty heavy desktop desktop. We the fact of use, Cloud, and I'll go into a bit more about Cloud. But I took some estimates from digital ocean just to run out before $100 dead. So you know, getting more pipelines getting your research pipelines optimal is also really important to keep the cost down.
STFC-RAL-CR03 R61: It's also a little bit of a contention, because
STFC-RAL-CR03 R61: we have the big tech companies that obviously this is not a lot of money, and they can train us for. And they can do a lot of
STFC-RAL-CR03 R61: different things. So one of the questions is.
STFC-RAL-CR03 R61: how can we work today in that space? And how can we use what technological advancements that are made open to us to advance science further.
STFC-RAL-CR03 R61: And then
STFC-RAL-CR03 R61: the last challenge is software. And I feel that this also might be quite close to this communication part. So
STFC-RAL-CR03 R61: anybody here written a Grant software to write some software or nobody.
STFC-RAL-CR03 R61: Yeah. Okay, well, this-, this might be a familiar story. If you work with a software that's simple.
STFC-RAL-CR03 R61: So you get a branch of all your software
STFC-RAL-CR03 R61: and work package for is build community. You get to your end of your branch. You finish writing the software you've run out of time and work package for is in with it. You are very proud of the software that you've written, you go to your conferences, you build your community all of your own time, and then
STFC-RAL-CR03 R61: 5 years down the line. You've got a new job. You're working on a different brand. But suddenly, your software is a critical dependency of the community.
STFC-RAL-CR03 R61: And
STFC-RAL-CR03 R61: the operating systems have updated. And your software is not being maintained. There's our Gpus and your software is not being maintained. The version of programming language no longer exists, and your software is not being maintained. So we have this massively in my good software that we just can't run?
STFC-RAL-CR03 R61: Or is it giving you all different from it?
STFC-RAL-CR03 R61: And also you might be familiar with the Rc. Community. It's a sort of movement that started in the Uk around 30 years ago.
STFC-RAL-CR03 R61: but essentially most of us started off as physicists, but mathematicians and other people. And we got into writing code. And then we realized that actually, all we did was write code, and we can't be such.
STFC-RAL-CR03 R61: But you know.
STFC-RAL-CR03 R61: when I was when I started out, I didn't even really understand the need for documentation. I certainly didn't know how to build an installer. So you find all this software, especially on Git, which has no installers, no documentation. And people go. Oh, I read this in in the paper. Can you install it for me. They got really good results. And you
STFC-RAL-CR03 R61: start with the exam like
STFC-RAL-CR03 R61: this. This hasn't run on on systems that sent off spy. So probably not at this point.
STFC-RAL-CR03 R61: and then this is a little bug there as well. So
STFC-RAL-CR03 R61: sometimes your code might be, something might be accessible, doesn't mean it's open source.
STFC-RAL-CR03 R61: So if you take, for example.
STFC-RAL-CR03 R61: people might publish a table of the code
STFC-RAL-CR03 R61: attached to a paper that's accessible. That is an open source. You know. There isn't a way to fill up that code. There isn't a way of understanding
STFC-RAL-CR03 R61: how to to.
STFC-RAL-CR03 R61: So this happens reasonable. These are big challenges, like quite a lot of time my team spend is installing software.
STFC-RAL-CR03 R61: And some communities have active software.
STFC-RAL-CR03 R61: development. But a lot of community don't. And it does really
STFC-RAL-CR03 R61: So
STFC-RAL-CR03 R61: this challenge. So the other challenge that I alluded to is that you now need new skills in a bioscience sector to deal with all of these things right? So I just talked about research software engineering.
STFC-RAL-CR03 R61: So it was basically a bunch of people
STFC-RAL-CR03 R61: who are academics who writing code
STFC-RAL-CR03 R61: and realized, actually, this is a really important job. Now for science or for research in general. With digitization of libraries and things like this.
STFC-RAL-CR03 R61: And as digital presence increases, you actually need to do more.
STFC-RAL-CR03 R61: research. And this is a valid academic. So this, this is, this is the field, so beautiful
STFC-RAL-CR03 R61: a bunch of skills that quite a lot of people have, but a lot of people don't have in-in sector, and certainly bioscience. These are hard to come up. So people have buyer knowledge and and these skills.
STFC-RAL-CR03 R61: Okay, so those are the sort of general challenges. And I'm gonna kind of talk about examples of those challenges in the Franklin and then go on to strike. How we solve.
STFC-RAL-CR03 R61: I thought I'd probably give a quick introduction. I have never been to this building. Oh, that's the fire alarm. But you know
STFC-RAL-CR03 R61: a lot quieter than the Frankies that I love very early. That's what made that.
STFC-RAL-CR03 R61: And so we just opposites you probably walk past us from the the entrance. And and the building is actually these colors because it was the colors of the old directors. Plans tartan. So that's why, it's this yellow and then blue color. And what we do is we
STFC-RAL-CR03 R61: drive, we do technology innovation technology in a national institute for technology innovation to transform life, science to improve human health outcomes. So we aim to kind of bring lots of collaborators together into our space so that we can actually drive forward like high risk projects. And one of our many targets is to
STFC-RAL-CR03 R61: understand how we can
STFC-RAL-CR03 R61: break the drug development cycle, find new ways of exploring drug development because quite a lot of drug development is based around the same chemistries, same biologies. And we have to find more and more new inroads to to drug development
STFC-RAL-CR03 R61: which AI so we have. We are split into 2 parts. We've got life size challenges and technology innovation challenges. I sit in the AI and Informatics fixed biology challenge. And basically the idea is that you know, you've got 2 sides of the coin that work together. So the one side.
STFC-RAL-CR03 R61: We've got this tons of technology. Look at Llm. What can you do with it? And the other side is going. I have this problem that has never been solved. Do you have any technology, you develop any technology to solve it. And so by having these 2 sides kind of iteratively develop?
STFC-RAL-CR03 R61: And so we have
STFC-RAL-CR03 R61: a bunch of technical challenges inside the spectrum. Okay, so we have instrument challenges. So this is something that's quite special about
STFC-RAL-CR03 R61: is that we actually don't build any of our kids ourselves. It always doesn't in conjunction with an industry part.
STFC-RAL-CR03 R61: And so often what that means is
STFC-RAL-CR03 R61: we can't do data acquisition at the point of measurement. Unlike.
STFC-RAL-CR03 R61: Yes, sure, Cool remarked, because we don't own the control system. So that's a very interesting point.
STFC-RAL-CR03 R61: And so we have to get all our data off our instruments before they run out of storage to an off site to a different storage
STFC-RAL-CR03 R61: without having, without being able to access software and basically restrict network access.
STFC-RAL-CR03 R61: We have quite a large data challenge. We have 70 TB a month. Roughly, that's kind of what we produce. It's a lot very talked about proteomics, which is a different world proteomics. It's large database, but millions of Kilobytes files rather than a large
STFC-RAL-CR03 R61: terabyte huge. Let's see.
STFC-RAL-CR03 R61: compute. And I mentioned software challenges. And so this, these challenges require these infrastructure requirements. So we want our data transfers to be fast, secure and robust.
STFC-RAL-CR03 R61: And we want to automate our workflows as much as possible. So efficient.
STFC-RAL-CR03 R61: So I want to talk about this live science challenge. So this is kind of a bit more exciting, a bit more. AI,
STFC-RAL-CR03 R61: that's about bits of infrastructure. So this is about using generative AI to understand me, attract.
STFC-RAL-CR03 R61: So micrornas are type of non coded Rna, based in the influence of genes. So do you know how the central dogma of proteins DNA? So DNA produces Rna produces proteins so
STFC-RAL-CR03 R61: monitoring Rna services, proteins.
STFC-RAL-CR03 R61: So what do they do? Why, the why do they even exist? And then neutrons are a type of these Micro Rna. So people are kind of wondering, like
STFC-RAL-CR03 R61: what's- what's happening with these, with these, with these micro?
STFC-RAL-CR03 R61: Yes, and they're hard to identify, because they're very. They've got very small sequence. But anyway, small data set. So if you want to do any AI with small data sets.
STFC-RAL-CR03 R61: you are prone to overfitting who have not many features. It's difficult
STFC-RAL-CR03 R61: in the world of image data we have a way of augmenting our data so you can take, and that you can do this all the time with images.
STFC-RAL-CR03 R61: If you want to do a machine learning or AI method on A, on images, you could take your original image, and you can do all of this to it.
STFC-RAL-CR03 R61: and that to the machine is a is muted right? So you can generate a bunch of synthetic data to make it a second
STFC-RAL-CR03 R61: also to make it. This biased. So people do that all the time, however, for sequence data which looks like this.
STFC-RAL-CR03 R61: none of these physical augmentations actually exist.
STFC-RAL-CR03 R61: And so what you want to do is actually generate states. So you have enough data to train models. So you use general
STFC-RAL-CR03 R61: networks to do this stuff. So
STFC-RAL-CR03 R61: is a very important word here. So you basically have a generator. And you have discriminator. So people- people often refer to this is the 4 Gen.
STFC-RAL-CR03 R61: And the and the investigator. So what the generator is trying to do is generate something
STFC-RAL-CR03 R61: so that the discriminator
STFC-RAL-CR03 R61: cannot tell the real from the synthetic. So, firstly, what you do is you train the discriminator on whatever it is you want your generator to see.
STFC-RAL-CR03 R61: So it knows what that thing looks like.
STFC-RAL-CR03 R61: Are they an actual generator? To make that thing without training
STFC-RAL-CR03 R61: to generate my 1st attempt, like what I think it is, and after a few more attempts, it might start to get closer.
STFC-RAL-CR03 R61: and eventually you will get something that passes discriminator as an example of that thing.
STFC-RAL-CR03 R61: And so in this work. What by my colleague as well. What they did was they. They took a bunch of micro Rnas, and they wanted to predict whether a Micro Rna was in the.
STFC-RAL-CR03 R61: And so they had to increase the sample size of the micro Rnas.
STFC-RAL-CR03 R61: And what they found was where they included similar data.
STFC-RAL-CR03 R61: That prediction was actually better.
STFC-RAL-CR03 R61: Then, when they did, however, they still needed 70% of real data, the data set to make that work.
STFC-RAL-CR03 R61: So and I, this is a sort of
STFC-RAL-CR03 R61: example of how we are starting to use these new techniques to.
STFC-RAL-CR03 R61: to sort of answer challenges that we can't really do experimentally.
STFC-RAL-CR03 R61: It has these computational requirements huge amount of storage that this thing produce, had a word with the Phd. Student Allison, who, like.
STFC-RAL-CR03 R61: excuse me, can you please think about moving your data at 300 TB, and we all of this work was done on a virtual machine.
STFC-RAL-CR03 R61: So the second example I wanted to talk about is another piece of work that we're doing, which is analyzing placenta to improve birth outcomes. So this is funded by welcome week. And the main thing that we need to do here is
STFC-RAL-CR03 R61: The main thing you do is
STFC-RAL-CR03 R61: is to analyze the presenter in a quantitative manner, so
STFC-RAL-CR03 R61: quantities of information I don't have too many information helps us understand things better.
STFC-RAL-CR03 R61: So one of the things that we do here is we. This is an image taken by Xract on the center. And then we segment various areas for certain. So yellow is the tissue. The pink is the fecal blood, and all the gaps in between where the blood flow is.
STFC-RAL-CR03 R61: So by doing the segmentation we can really understand.
STFC-RAL-CR03 R61: you know, sizes. We can even do fluid, dynamic kind of simulation. We can try and understand
STFC-RAL-CR03 R61: how the simple structure may be linked to.
STFC-RAL-CR03 R61: So that's just some pictures of it in case the video doesn't play. And I always put that in. Now, this requires a huge amount of of compute. So this really requires type. It needs a high level of vram because we're working in voxel space here. So 3 images, it's very hard to do deep networks in 3D.
STFC-RAL-CR03 R61: The the amount of lia is who needs to create the network so optimally with 2D network, and then
STFC-RAL-CR03 R61: and bring it together into these 3.
STFC-RAL-CR03 R61: And then the last one is about priority which I talked about before. So this is one of our microscopes. This is from Dorothy, and it's our project microscope and what you, when you're working on microscope, you take lots of slices.
STFC-RAL-CR03 R61: This is a cell in the right version.
STFC-RAL-CR03 R61: Then you have to pick out all the buttons. You then
STFC-RAL-CR03 R61: average together all the ribosomes in case density.
STFC-RAL-CR03 R61: and then you do some polishing and finally get structure of what your ribosome looks like in the cell.
STFC-RAL-CR03 R61: So this is a real big workhorse. It requires a load of software to do it, half of which is
STFC-RAL-CR03 R61: falls into the category of software I mentioned earlier. It requires a lot of heck you need to compute. And so it's this, this pipeline can take months to get to here. And actually, only once you've got to here. Do you know that you did it wrong.
STFC-RAL-CR03 R61: So so it's a real time set, quite likely. So what we want to do is actually
STFC-RAL-CR03 R61: do more pipeline on this. We want to use Kubernetes to do it. So Kubernetes is type of compute which we call it container orchestration. But basically what that means is, it can choose when a program is run where it's run and can scale it. So for all our steps, we can run them on the correct compute step. We can scale them, depending how much data we have, and we can have hundreds of minutes or something.
STFC-RAL-CR03 R61: So this is what we want to do with that.
STFC-RAL-CR03 R61: So here's a bit of a summary of by size challenges and the different solutions that we have put in place. So
STFC-RAL-CR03 R61: you can see there's a lot here. You've got Kubernetes. We've got Cloud, HD,
STFC-RAL-CR03 R61: so there's quite a lot of different compute requirements we kind of meet in the.
STFC-RAL-CR03 R61: So what do we need to change. So I head up the advanced research computing team which brings together research infrastructure. And these are people in my team. What I think is very special about this is that you need all parts to really understand things well and to optimize things. Well, you need to understand the research, to understand what you can do with this to the research challenges you need to understand
STFC-RAL-CR03 R61: infrastructure to better write your software. You need to understand your software to better write the infrastructure to
STFC-RAL-CR03 R61: to build this. And so this is what I really like about is that it brings all these these things together to meet the challenges.
STFC-RAL-CR03 R61: So this is quite an interesting time to give this presentation, because I don't know if you actually use Scc. Cloud, or you know what's going on at the moment with it.
STFC-RAL-CR03 R61: But I have to say this is the absolute worst outage I've ever seen in my entire 3 and a half weeks. 3 weeks. Yeah, 3 weeks. Yeah.
STFC-RAL-CR03 R61: so this is kind of made this talk slightly different to what I would talk to you about. But
STFC-RAL-CR03 R61: it's given us Gpu and CPU virtual machines. It gives us huge like.
STFC-RAL-CR03 R61: I'm logic, and it has been pretty recently
STFC-RAL-CR03 R61: one thing that we have the other partner.
STFC-RAL-CR03 R61: Again, if anything is starting to lose funding, so is kind of at the end of its funding cycle now.
STFC-RAL-CR03 R61: But we were in partnership with Diamond University phone, and we worked with the system, which was really great, and we did a lot of work with the Admins of Vascular to actually run priority and got a lot of good
STFC-RAL-CR03 R61: good work out of that.
STFC-RAL-CR03 R61: And then Isabella AI. So I was on a journey to Bristol last week, and I was actually at the launch event of Isabella AI. So frankly, we're really lucky that the center project that I showed you just now we got time to do the calculations on Isabel AI. And, as I mentioned in that project, the RAM is really important. And so, having these big growth and more Vram, we really hope that we can actually accelerate this project.
STFC-RAL-CR03 R61: And so this is me. And Dr. So I think it no longer looks like that. There's no other building around it. I did actually get to see the shipping container which was signed by the Minister for Science.
STFC-RAL-CR03 R61: So
STFC-RAL-CR03 R61: this is our platform, and so it's not spend. And I'm running good on time. I do want to spend time talking about this, because there are a few important technological decisions that we made early on that has turned turned out to save our skins a little bit in the current outage of
STFC-RAL-CR03 R61: the platform. So I would love to share these nuggets with you, and you may be already doing these things.
STFC-RAL-CR03 R61: So our platform basically have a big storage layer which is echo. So this is also the tier 2 storage for so I'm sure you know we don't break to it by the rattle gateway. Break to it by S. 3.
STFC-RAL-CR03 R61: So we have
STFC-RAL-CR03 R61: chosen a gateway that can be used with this cross against the connector. And then you can basically
STFC-RAL-CR03 R61: move it to any other cloud that has the S. 3 style.
STFC-RAL-CR03 R61: right? We haven't written it on it yet, but we could
STFC-RAL-CR03 R61: transfer between clouds very easily. This
STFC-RAL-CR03 R61: we have our instruments, they have their hard drives, we have our cloud which is served by.
STFC-RAL-CR03 R61: and then we have
STFC-RAL-CR03 R61: to transfer data between these systems. Now we pay for a license which I know is not not suitable for everybody, but because of the size organization we are
STFC-RAL-CR03 R61: we. It's very economic for us to have license.
STFC-RAL-CR03 R61: What about gives us the S in connection
STFC-RAL-CR03 R61: into global say into into the storage. And it also allows us to use of authentication in a really cool way on
STFC-RAL-CR03 R61: we have contributed back to global over the years. And we have quite good working relationship with them. And so they make a lot of developments from our use cases. Api, if anyone's interested, which is no resource. Api.
STFC-RAL-CR03 R61: we have to have a business
STFC-RAL-CR03 R61: account to actually use the Api. So and no, it's it's accessible.
STFC-RAL-CR03 R61: And so this is actually. And we do everything via ansible and later kubernetes, but actually with infrastructure as code.
STFC-RAL-CR03 R61: So you basically write a bunch of gamma files that configure what you want to spin up in cloud, and then you can deploy systems very friendly. And this is this has given us 3 really important things. One documentation.
STFC-RAL-CR03 R61: Your code is basically documentation. Whenever I get something new during the team, I'm like, right, look at the scripts. You will then understand how these things are set up.
STFC-RAL-CR03 R61: It's also platform agnostic. So you can use. You can spin up many different places. And so during the current outage, we have been able to move some of our services to a different cloud provider
STFC-RAL-CR03 R61: to have those risks to do it, and took us
STFC-RAL-CR03 R61: one week one week to get the cloud device signed off.
STFC-RAL-CR03 R61: And to start putting those things.
STFC-RAL-CR03 R61: 1st thing we needed to make this group because our instruments filling up, we don't want to stop the experiments because we don't have any cloud.
STFC-RAL-CR03 R61: So this is really allowed us to to move. And the second thing really super useful is that we're not really.
STFC-RAL-CR03 R61: they don't have loads of services running. We've been running humans for about 2 years, and the idea is to start to move most of our services towards this kind of architecture.
STFC-RAL-CR03 R61: One of the things we want to build is a human S cluster that spans Sdsc cloud, and in the
STFC-RAL-CR03 R61: the ability that when one plan drops the other one can provide services so we can verify availability.
STFC-RAL-CR03 R61: And so this is a these are, I think.
STFC-RAL-CR03 R61: on lessons learned. I would say, instructors code definitely.
STFC-RAL-CR03 R61: And yeah, working towards sort of the Ops style of manner.
STFC-RAL-CR03 R61: I'm gonna just get through this because I'm running out of time.
STFC-RAL-CR03 R61: And I just want to talk briefly about slycat. So slycat is a an open source.
STFC-RAL-CR03 R61: Metadata feel that this is required. People
STFC-RAL-CR03 R61: so metadata is all the data that adds context to your data. When was the experiment done? Who did it? What was the sample? What was machine learning? Asked, what was the detection. What software did you?
STFC-RAL-CR03 R61: And so you want to store that somewhere to give you a bit
STFC-RAL-CR03 R61: truly opens up that can be integrated with almost any architecture.
STFC-RAL-CR03 R61: It used to the kids. Well, it was. It was
STFC-RAL-CR03 R61: delivered by the panels. But actually, we've been using it in frankly 5 years. So it's very, very flexible doesn't just sit in space. And part of it is because it has this very flexible meta, a scientific metadata part that you can actually add, visit the other thing as is you can have. You can
STFC-RAL-CR03 R61: plug it into your. You can plug it into your architecture. So it gives you roots to like publish data, sets, archive data sets, treat them from storage. So it gives people like Gui interface. It gives you an authentication layer to to do that. So that's a very, very cool project.
STFC-RAL-CR03 R61: So I think the last thing I just want to say is.
STFC-RAL-CR03 R61: did we solve the biological science challenges? Well up until 3 weeks ago. Yes, now not so much but we by by using a
STFC-RAL-CR03 R61: by using a variety and having and again using infrastructure code so that we can actually provide that compute in a in a in a repeatable way that people in the team understand how they develop. This actually helps us expand, develop, and keep on maintenance at a reduced amount.
STFC-RAL-CR03 R61: So yeah, I think. Lastly.
STFC-RAL-CR03 R61: considering this, it can be a problem, it is. It is hard.
STFC-RAL-CR03 R61: Think we all know this is really hard. The question I get asked
STFC-RAL-CR03 R61: pretty much every day is, is what you do. Science? I would argue very much. It is. It's, it's, you know, become
STFC-RAL-CR03 R61: equipment that needs to be optimized. You need to run experiments on how data move around. And ultimately, you don't know if what you're building is the right thing. So I would argue that this is very much base science. So I am out of time.
STFC-RAL-CR03 R61: So I mentioned the cycap. So this is a this is working with us.
STFC-RAL-CR03 R61: So this is the most important 2 slides. So I'm glad I've got 5 min to go through it. As I mentioned before, scicat is an international metadata catalogue. I'm part of a wonderful development community of Scicast. You can find my website here, and one of the things it's not really used very much in the Uk. We have quite a lot of
STFC-RAL-CR03 R61: Institute, Germany. The big so Max Hall, Ess, and the Aos and psi all use sidecat. So they're outside of Germany. But there's a big international community of this and us. And one of these I'm right, quite interested in is seeing what the appetite for this in the Uk.
STFC-RAL-CR03 R61: Right now, talking about a digital research infrastructure. And one piece of puzzle that has never been mentioned is metadata.
STFC-RAL-CR03 R61: And so actually, if you're interested in net data, capture, storage, and how that could be used. Do feel free to contact me about cycamp kind of interested in seeing if there would be a Uk community cycap. Uk, which I really like. So yeah, if you're interested in psychap, please let me know.
STFC-RAL-CR03 R61: And then we also run a organizational. We have a bunch of software that we publish open source. I've got a few projects here that I think that this community might be interested in. There are obviously a few more.
STFC-RAL-CR03 R61: So one is the flow crop so flow planners actually work with flows which you can access without license flows. And what that allows you to do is transfer your data, run a slurge or bring your data back all into one kit. And it's very user friendly, because
STFC-RAL-CR03 R61: you can, just the user can just click the flow, and that will run for them. So it democratizes agency in a good way.
STFC-RAL-CR03 R61: This tracks is a really cool project that never really got. Everyone was super interested in it. We took. We took it to a lot of conferences, but I don't think anyone's ever used it because of one slight change. The Linux kernel we've got to do which knows really likes. But it's an interesting project, because it's about
STFC-RAL-CR03 R61: hopefuls competing. So when you have pipelines, presumably you have lots of notes.
STFC-RAL-CR03 R61: Often you have stare, RAM on those notes.
STFC-RAL-CR03 R61: and often you have a very expensive file system that you would want to write out to
STFC-RAL-CR03 R61: if you want to stop I/O thrashing.
STFC-RAL-CR03 R61: So what you do is you right? You can make a distributed storage system in the RAM that's left in your notes.
STFC-RAL-CR03 R61: and, in fact, to that is then writing back to us first, st
STFC-RAL-CR03 R61: this makes more sense to try and do Htc. In the cloud, because the storage network file systems in cloud vastly expensive. So actually, if you use this fact, you actually don't need files. You just use you take the data in from an object store. You run it in your
STFC-RAL-CR03 R61: your storage space and brand. When you finish it you'll be right back post talking there.
STFC-RAL-CR03 R61: and we've we've found reduce on actual Hpc real physical. You can reduce the the effect of I/O. In some processes.
STFC-RAL-CR03 R61: Parakeet is the Tm, Tdm, digital twin, something that we're doing frankly is getting more into digital twins. So I think potentially the particle physics is something we could collaborate our digital twin or our big digital twin project is the digital twin of cell.
STFC-RAL-CR03 R61: So that's going to take. It's going to be a huge undertaking. And we're going to build it kind of step by step. So if people have got experience with this twins or building digital twins, I'd love to hear more. And then, lastly, the Rfi global Api, plus the site which will come soon, which is all our Apis to interact with different parts.
STFC-RAL-CR03 R61: And so, lastly, I just wanna thank our people. This is the the crew at Franklin, and obviously our competitors.
STFC-RAL-CR03 R61: and now afterwards, who from from months ago
STFC-RAL-CR03 R61: so I hope that was interesting and somewhat relevant if it wasn't relevant. I hope it was entertaining. And yeah, any questions.
STFC-RAL-CR03 R61: So it's interesting to see all this sort of stuff. Some of these things were related to what we're doing. Some of these are not, I think we have a lot of different one thing. I have many questions, but one thing on one of our problems right now is we don't control the hardware. So so we want to have one piece of software. Is there anything you're doing on that
STFC-RAL-CR03 R61: skipped over time. So one of the things that we are using is this guy to start a workflow.
STFC-RAL-CR03 R61: both pins, the ability to
STFC-RAL-CR03 R61: It's very much compute agnostic, so it'll work on any hardware. If you don't have control of your hardware, you can run it as a layer. If your Admins allow you to to run it as a layer. So actually, we don't use St. We've written our own on top of their own stack, right? And one of the things we want to do is actually write it on bare metal. So with No- No.
STFC-RAL-CR03 R61: if I may, my question is slightly different. So example, you've got an algorithm which runs on a Gpu algorithm. Do you have any effort going to that? Or do you just type in? No, this is also my question for everybody at conferences, which is, how do we break the individual on all this stuff?
STFC-RAL-CR03 R61: Is somebody going to be able to write a driver that work with any of these, all of these gpu's when I was at the day I was talking to the head, who obviously I don't know anything about door assist has intel gpu
STFC-RAL-CR03 R61: rather than video? So the only thing I know of is the layer above, which is high torch. But I heard. It's still hard.
STFC-RAL-CR03 R61: Think every thing so I'm very curious to know
STFC-RAL-CR03 R61: love to hear what people are doing, because I think this is something. You know. I don't think the
STFC-RAL-CR03 R61: well, if enterprise has any motivation to do this, and I think that we can't answer your question.
STFC-RAL-CR03 R61: Other questions.
STFC-RAL-CR03 R61: It's
STFC-RAL-CR03 R61: do you have any specific tools for sort of data. Archival. We saw the data going up and up. Is it going to go forever? Or do you have data, deletion policies as well. It's a really good question. So I am a big
STFC-RAL-CR03 R61: But being a supporter of data, it's because.
STFC-RAL-CR03 R61: quite frankly, if you don't have context. If you haven't saved your metadata, what's the point of moving around? You're just costing, you know, 2 client budget storage budget.
STFC-RAL-CR03 R61: So.
STFC-RAL-CR03 R61: But we don't have policy right now. We were. We were following Ukri's called out of open data which said, you had to keep data for the project for 10 years, whichever was longer. And so we were basing that on the dates data policy. But that is now just been updated precisely for the environmental rules that I just mentioned. So
STFC-RAL-CR03 R61: one useful thing inside caps is, you can actually trigger archive kind of ones. So this is what we would plan to do is we will archive to take at the moment we've got 7, but we don't really need to keep that.
STFC-RAL-CR03 R61: All live will archive tape. I think we will. We will allow people to delete data that is no longer relevant.
STFC-RAL-CR03 R61: No use.
STFC-RAL-CR03 R61: And you can mark that inside us. So you can still have the metadata from that data set.
STFC-RAL-CR03 R61: you know, few Kilobytes.
STFC-RAL-CR03 R61: And you can have a reason why that data is limited. And yeah, I'm a big support of that. Because, you know, there's just so many reasons not to keep some of the data.
STFC-RAL-CR03 R61: Certainly the mid processing sets
STFC-RAL-CR03 R61: like, I feel you should take the raw data and maybe finish a project. But your metadata should be enough, that if anyone wants to keep that experiment they could, they could do so.
STFC-RAL-CR03 R61: And so.
STFC-RAL-CR03 R61: in short, it's definitely part of the plan. We certainly built it for more data archival stuff. The other thing I just mentioned is one of the things that we want to work on this open source. Sorry object store, friendly farm formats and compression.
STFC-RAL-CR03 R61: So we, I want to go away from this idea that you have to transfer data down onto a file system and run your compute. Want to do more streaming and work with data in a in a much more
STFC-RAL-CR03 R61: sort of interact in the way which has 3 back end of the lives. So that's kind of where we work a little bit to reduce our.
STFC-RAL-CR03 R61: I mean, it sounds like you're basically doing everything which we do except we're kind of locked into very bespoke tools which got this in 1015 years ago and still doing the job now. But yeah, I just don't see how we
STFC-RAL-CR03 R61: how we escape that environment into sort of more open source and more
STFC-RAL-CR03 R61: collaborative environment. Maybe it's too late for us to to make that jump.
STFC-RAL-CR03 R61: Yeah, it's very interesting, because when I go to
STFC-RAL-CR03 R61: conferences and like, I love. I love attending conference sessions on legacy code, because I used to work in industry. And so yeah, legacy code was something that taught us. And
STFC-RAL-CR03 R61: it was when I came to the Franklin. I was like, Yeah, there's really nothing on the table. And now, and
STFC-RAL-CR03 R61: one of the things I think the hardest thing with moving away from legacy is you have to bring the whole community with you. That takes time.
STFC-RAL-CR03 R61: You know. And one thing that we're doing right now is we're we're moving.
STFC-RAL-CR03 R61: More of our processes are accountable towards capabilities. So we're actually starting to hit our road legs right now.
STFC-RAL-CR03 R61: But getting the space to do that.
STFC-RAL-CR03 R61: you know it. It being seems like to do. That's a really hard sell people if you don't.
STFC-RAL-CR03 R61: the technology remaining skill for these ancient things will far exceed the time we have since.
STFC-RAL-CR03 R61: I think there's a big problem in the space right now, because it can't be
STFC-RAL-CR03 R61: resurrect the language and keep full time.
STFC-RAL-CR03 R61: More questions is, oh, sorry. The TV, please. Actually continuing on this thing of the legacy software in particular.
STFC-RAL-CR03 R61: yeah. When you were saying that I kind of felt your pain is there, do you? Is there a kind of something to be gained by kind of making a
STFC-RAL-CR03 R61: by combining different. So so by by having people whose job is to maintain to to maintain old software,
STFC-RAL-CR03 R61: so to have a community to do that rather than each each software, the maintainer of that is always the only expert that could do that. And then they've got to learn how to, how to update that
STFC-RAL-CR03 R61: think it brings the people on with you, on. Why you want to do that.
STFC-RAL-CR03 R61: The other tool that has now entered into this mix, which I think is really interesting is elements.
STFC-RAL-CR03 R61: because in theory you should be able to give some Llm. Some I mean see code. Ask them to translate that into us.
STFC-RAL-CR03 R61: Now, I'm not saying that a human can't do that, because, of course, you can, but they can do it much quicker, potentially. I've not tried this. An Llm. Is yet to give you a piece of vulnerable code off the bat, but maybe I'll just tell the content in it. But you know, and so I'm kind of I'm you know.
STFC-RAL-CR03 R61: We have really pushed time, and I think we do need to do that human approach bring the humans together. But could we actually use AI in this space? Because, you know, at the end of the day there's a reason why languages don't lie. There's a reason why
STFC-RAL-CR03 R61: why projects become obsolete, and it's so hard to act in to get money for maintenance, and I just don't see that changing
STFC-RAL-CR03 R61: the software, sustainability and Ability Institute released a call at the start of this year called the Research Software Maintenance Fund, and
STFC-RAL-CR03 R61: hopefully do another call again. But what was really interesting is so what was interesting is, they had to change the terms midway through the call with so many people. So I hope maybe there'll be money. So I don't, I think. But yeah, basically, I think
STFC-RAL-CR03 R61: I think it's important to maintain it.
STFC-RAL-CR03 R61: But
STFC-RAL-CR03 R61: I don't see that we'll ever get the timeline to do it. So could we try some innovative approaches like building as community
STFC-RAL-CR03 R61: or using elements?
STFC-RAL-CR03 R61: Oh, are we looking to set the zoom account.
STFC-RAL-CR03 R61: Yeah, I just have a question of cycat. So one thing is, we have something which I believe is similar. Which is hep data. Oh, yes, I'm just wondering how that differs. And then, because obviously, we're tied into that as our thing. I'm just wondering if you, if you encountered that as well, or how it differed. Yeah. So I actually don't know too much about it. I've only sort of come across it when I'm writing grants or talking about cycat. Why you should use cicat.
STFC-RAL-CR03 R61: The
STFC-RAL-CR03 R61: the big thing about scicat is it has a no SQL. Back end, which makes it incredibly flexible. So this is why it's useful for the panelist community, because they have all metadata catalogs like icat icat plus and a few others. And one of the problems with those is, they have SQL backends. And what that meant was as they wanted to add more and more metadata fields. This thing grew smarter and smarter and bigger and harder to handle.
STFC-RAL-CR03 R61: So
STFC-RAL-CR03 R61: that was like the main driver for sidecar. Then, because it's got this really flexible document store, they realized it could be used outside of the of the
STFC-RAL-CR03 R61: house world. The other good thing for us is, it allows us to join dates together from vastly different fields. So there is message catalogs, and there are messages, catalogs from that spec, but there's not one
STFC-RAL-CR03 R61: for both. And so that's the kind of situation we are in.
STFC-RAL-CR03 R61: I don't know if we have data, there is that kind of heterogeneous use case that you would need for
STFC-RAL-CR03 R61: just curious. Yeah. Curious, curious. I mean, yeah. And a related thing on this. And one thing which we're trying to explore now. And I'm wondering if you're doing similar thing is also rather than metadata, also storing the dock container that actually had the software rather than that sort of thing
STFC-RAL-CR03 R61: software. And then building up the configuration things from, I feel like
STFC-RAL-CR03 R61: what you would probably need instead of storing full docker tech containers, I would probably
STFC-RAL-CR03 R61: down. I probably saw the docker file based image and binaries
STFC-RAL-CR03 R61: because of the way we run.
STFC-RAL-CR03 R61: But at the moment most of us in person to automate the message, it's just ingestion into side paths
STFC-RAL-CR03 R61: with these 2 Apis. And one of the problems we have with message generation, we can only get that from file. So one of the routes we're working in is.
STFC-RAL-CR03 R61: can we get users, can- can we automatically get the version of software? They ran space. They run in because a lot of them are running them on vms that we built. So we probably put it like version of the day. And things like that.
STFC-RAL-CR03 R61: It's something we want to do, I think. But what's really interesting is in publication in the Bio space.
STFC-RAL-CR03 R61: you know, I ran quite a smart version. 5. That's all you need to say. You might say I ran on an Rtx. 4,000 Gpu.
STFC-RAL-CR03 R61: But you don't need to put in all of the levels. And what a lot of scientists find is they can't repeat claims in papers, and then they come to us and say, Oh, I've been running this on a 180, and they're getting, but it's not as good as an Rtx. 4,000. Can I have an Rtx. 4,000? I'm like, well.
STFC-RAL-CR03 R61: that's a vastly different call like, what do you mean by better. And then- and then you dig deeper. And you're like, well, actually, there's a lot of stuff about this environment. We don't. So we can't replicate what they what they do so
STFC-RAL-CR03 R61: definitely, we're not-. We're not hold on for you.
STFC-RAL-CR03 R61: I don't see a raised hands. But yeah, if anyone on Zoom would like to ask.
STFC-RAL-CR03 R61: I don't see mostly by email, if you have any other questions.
STFC-RAL-CR03 R61: Yes, thanks to you again.
STFC-RAL-CR03 R61: So Laura has agreed to join us for lunch, so we'll be ascending the corridor.