STFC-RAL-CR03 R61: Good morning. Thank you for being here.
STFC-RAL-CR03 R61: But they have been dominant everywhere. There is no shortage of talk.
STFC-RAL-CR03 R61: Regarding its crucial potential.
STFC-RAL-CR03 R61: Today, I'm excited to move beyond that height.
STFC-RAL-CR03 R61: And do what we have actually done here.
STFC-RAL-CR03 R61: We have created a system That is designed to support our project managers, PIs in real-time decision making.
STFC-RAL-CR03 R61: My goal today is to focus on the theory and practical solution.
STFC-RAL-CR03 R61: So, I want to keep this high level and accessible.
STFC-RAL-CR03 R61: How do these tools actually work?
STFC-RAL-CR03 R61: Rather than going to the code.
STFC-RAL-CR03 R61: To get the heads up, Got it.
STFC-RAL-CR03 R61: This presentation, throughout the presentation, you'll see some lot of sci-fi material to help demonstrate the data journey and the future we have built in.
STFC-RAL-CR03 R61: During the today's talk.
STFC-RAL-CR03 R61: Remaining good life.
STFC-RAL-CR03 R61: We will look at the specific problem, the technical problem.
STFC-RAL-CR03 R61: Dr. Pierre is…
STFC-RAL-CR03 R61: And in the field up for the monument. And the methodology will tell you what tool
STFC-RAL-CR03 R61: On how we actually transform the raw data into actionable insights.
STFC-RAL-CR03 R61: Then we will look at the…
STFC-RAL-CR03 R61: under the hold of the AI engine, the IT infrastructure, and how we optimize the model.
STFC-RAL-CR03 R61: And finally.
STFC-RAL-CR03 R61: We will share some of our performance data metrics of our models and framework we use to ensure our data is fair and unbatched.
STFC-RAL-CR03 R61: Before I start, I want to say that this implementation, is a direct result of the operating system.
STFC-RAL-CR03 R61: It has been a great opportunity.
STFC-RAL-CR03 R61: To apply the training to our department's specific challenges.
STFC-RAL-CR03 R61: So, about the business problems, we are looking at one of the specific project management models.
STFC-RAL-CR03 R61: Known as the East Coast Management.
STFC-RAL-CR03 R61: Since 2020, we have maintained Detailed the truth, broad.
STFC-RAL-CR03 R61: And today, we are tracking over 200 active issues.
STFC-RAL-CR03 R61: That is, over 20,000 words of technical updates.
STFC-RAL-CR03 R61: Currently, our project managers, PIs, work package managers, are maintaining this log, as well as driving the project delivery.
STFC-RAL-CR03 R61: I've just given time.
STFC-RAL-CR03 R61: So, when a leader had to manually Go through all of this.
STFC-RAL-CR03 R61: And decide what needs immediate attention.
STFC-RAL-CR03 R61: It creates what I call as the paragraph gap.
STFC-RAL-CR03 R61: And this gap has got two major risks, too. So, one is the black entry.
STFC-RAL-CR03 R61: This is a difficult delay.
STFC-RAL-CR03 R61: Who… This is typically, like, about,
STFC-RAL-CR03 R61: about the history being updated and declared at the GED.
STFC-RAL-CR03 R61: In a multi-million project, time is a luxury which will always don't have.
STFC-RAL-CR03 R61: And the second minister is paid, subduede.
STFC-RAL-CR03 R61: We are often seeing a class of perspective.
STFC-RAL-CR03 R61: For example, a work package manager may naturally see an issue in their domain a top priority.
STFC-RAL-CR03 R61: Well, the project manager, looking at the project launch day.
STFC-RAL-CR03 R61: I mean, let's say that is true as a medium for locality. So, without the standards.
STFC-RAL-CR03 R61: We do not have the ranking become more inconsistent. Therefore, our PIs and project managers need to see the big picture instantly, without spending the green… I mean, without spending too much of time, and…
STFC-RAL-CR03 R61: update.
STFC-RAL-CR03 R61: So, that challenge, we are solving between that, yeah.
STFC-RAL-CR03 R61: So, transforming all these documented, technical, manual logos into an immediate, actionable, Interesting.
STFC-RAL-CR03 R61: This is the entire map.
STFC-RAL-CR03 R61: I took joked.
STFC-RAL-CR03 R61: The model we use, the AI model, it is 12, 2.5 total. This isn't a simple question and answer setup. It's a multi-stage hybrid pipeline where Python and AI work together.
STFC-RAL-CR03 R61: The computer workflow is split into two stages. One, we use the T1 linear extraction, and the analyzed test.
STFC-RAL-CR03 R61: I'll talk about this unit in a moment.
STFC-RAL-CR03 R61: But what makes this workplace pressure?
STFC-RAL-CR03 R61: It's how different girls work together.
STFC-RAL-CR03 R61: We have a Python project.
STFC-RAL-CR03 R61: Managing the high precision tasks, like pulling the data.
STFC-RAL-CR03 R61: Type question, and got a privacy marketing.
STFC-RAL-CR03 R61: Meanwhile, the AI handles the language, that is, the attraction, the reality, and explanation, and systemarization.
STFC-RAL-CR03 R61: all cornered with each other.
STFC-RAL-CR03 R61: The entire practice is controlled by the strong motivator.
STFC-RAL-CR03 R61: So these are specific problems that define the rules of engagement, telling the AI exactly how to test to ensure it meets our specific challenges.
STFC-RAL-CR03 R61: Our example is data privacy might be filled later on, but in the next few slides.
STFC-RAL-CR03 R61: This is a rock, rooted from.
STFC-RAL-CR03 R61: And the air engine, and the detector.
STFC-RAL-CR03 R61: Now we are looking at a plum.
STFC-RAL-CR03 R61: More good.
STFC-RAL-CR03 R61: So, if you look at the map, what we just saw on last slide, yeah.
STFC-RAL-CR03 R61: So, this is a blueprint of our entire system.
STFC-RAL-CR03 R61: We keep our code clean by putting all our rules or prompts into this area.
STFC-RAL-CR03 R61: Didn't it? I was just turned very modular.
STFC-RAL-CR03 R61: So, if we need to order anything.
STFC-RAL-CR03 R61: See, individual prompt or anything related to, you know, the instruction, and we only update in the prompt area, total rest remaining the same.
STFC-RAL-CR03 R61: But Dr. Bron.
STFC-RAL-CR03 R61: We need the example to teach the AI.
STFC-RAL-CR03 R61: Also known now to fine-tune.
STFC-RAL-CR03 R61: And this… This tells the AI exactly what item to look for.
STFC-RAL-CR03 R61: But it's reality analysis, that is the model.
STFC-RAL-CR03 R61: We provide a clear… so that the AI knows what to do.
STFC-RAL-CR03 R61: And the Trump is… the Trump is split into four paths.
STFC-RAL-CR03 R61: the system prompt, and the user prompt, which I'll show you in the next slide.
STFC-RAL-CR03 R61: The system prompt defines the AI's problem and how it needs to act. And the user prompt is a set of tasks that tells the AI what to do.
STFC-RAL-CR03 R61: So, what makes this area unique is the helper function.
STFC-RAL-CR03 R61: You have built into it.
STFC-RAL-CR03 R61: They work like a kitten.
STFC-RAL-CR03 R61: Before the AI looked into the world, these tools
STFC-RAL-CR03 R61: the destruction, cleaning, and other tasks like privacy matching. And once the AI is done, these students load the process data back into our database.
STFC-RAL-CR03 R61: By doing the heavy work of formatting here, we allow the AI to focus only on the regional plane and dimples.
STFC-RAL-CR03 R61: So, to make model work, We balance the AI with our local hardware and our data sites.
STFC-RAL-CR03 R61: First, we use a method called quantitations, or optimization. This runs the model at a reduced size.
STFC-RAL-CR03 R61: It's helped the system to run faster on our local cluster.
STFC-RAL-CR03 R61: It's the mathematical way of making the model smaller, so it uses the less memory while giving the high quality.
STFC-RAL-CR03 R61: Our raw data comes from the toasters.
STFC-RAL-CR03 R61: And more will be helper function.
STFC-RAL-CR03 R61: And so, if you are on the other side.
STFC-RAL-CR03 R61: We have the prompt, which I didn't talked about earlier, victim handling the prompt.
STFC-RAL-CR03 R61: So, to do that, to get that.
STFC-RAL-CR03 R61: We have the coordination layer that does the heavy lifting.
STFC-RAL-CR03 R61: And this layer, I have to attach it to generate a report.
STFC-RAL-CR03 R61: So, by keeping the rules separate and the coordination, we have created a system that is far, fair, and deeper.
STFC-RAL-CR03 R61: So, we will look at the stage one in your direction.
STFC-RAL-CR03 R61: So… What is the English term for is known entity recognition?
STFC-RAL-CR03 R61: So this is the area the AI will automatically find the label or key information from the raw technical gloves.
STFC-RAL-CR03 R61: into a defined package, such as a non-alitinian entities.
STFC-RAL-CR03 R61: So, under using our tool.
STFC-RAL-CR03 R61: We move to the middle area, where we do not look only at the workers.
STFC-RAL-CR03 R61: But we are finding 11 categories of data.
STFC-RAL-CR03 R61: I agree with them, We turn and line up tech into an order database.
STFC-RAL-CR03 R61: So, why are we doing this?
STFC-RAL-CR03 R61: The reason is Candel Database.
STFC-RAL-CR03 R61: Which we don't really have.
STFC-RAL-CR03 R61: I mean, we didn't… we have a lot of documents, we have reports in the past, but they are not easily searchable.
STFC-RAL-CR03 R61: Therefore, we are building the base for the crucial knowledge graph.
STFC-RAL-CR03 R61: And, AI targeting… drunk.
STFC-RAL-CR03 R61: So, this makes sure that data we find in 2020, or previous data, says this will filter PI and PMs in 2020 tests and beyond.
STFC-RAL-CR03 R61: Moving on to the death tool.
STFC-RAL-CR03 R61: Once we have the structural data, we must solve the project management time.
STFC-RAL-CR03 R61: This issue is a priority.
STFC-RAL-CR03 R61: To confirm the problem, we have a defined setup and structure.
STFC-RAL-CR03 R61: Telling the AI work to trade between 1 to 10, depending on the nature of the issue.
STFC-RAL-CR03 R61: And this is happening fast.
STFC-RAL-CR03 R61: Until we need to harvest, and harvest that manually work.
STFC-RAL-CR03 R61: In the past, ranking them was a matter of opinion.
STFC-RAL-CR03 R61: So, one person is a high risk, while the other says that shop is a medium risk. So, the other AI chopped up by us.
STFC-RAL-CR03 R61: Using those set of rules, you can look at every issue on a clear 1 to 10 scale.
STFC-RAL-CR03 R61: But we do not drop the AI applying it.
STFC-RAL-CR03 R61: We also will be at the guardrails.
STFC-RAL-CR03 R61: Within the… our road, we have mandatory minimum.
STFC-RAL-CR03 R61: For example, if the AI finds a risk to a milestone or a lockup, Lots of guitar.
STFC-RAL-CR03 R61: It cannot drop below that deadline.
STFC-RAL-CR03 R61: This is how we put our expertise directly around AI and logic.
STFC-RAL-CR03 R61: So, one thing really clear, is the AI will look
STFC-RAL-CR03 R61: Only, about the grading and originally.
STFC-RAL-CR03 R61: And the Python would do the math.
STFC-RAL-CR03 R61: So, thanks, making the entire project faster with minimum memory, and of any component requirements.
STFC-RAL-CR03 R61: So, from this, we have 3 distinct regions.
STFC-RAL-CR03 R61: One is the boat.
STFC-RAL-CR03 R61: of the story. The other one with the project manager will look at the limited summary.
STFC-RAL-CR03 R61: of YAI.co.
STFC-RAL-CR03 R61: And for the reporting purpose, We will be, doing the summarization.
STFC-RAL-CR03 R61: Now we are on to the next pillar, the IT infrastructure and optimization.
STFC-RAL-CR03 R61: So, our current infrastructure failures.
STFC-RAL-CR03 R61: It runs on our local cluster. It uses the powerful signals, like MPGRA100, or Tesla 300.
STFC-RAL-CR03 R61: So we are still using condo system.
STFC-RAL-CR03 R61: Which manages the work across and, network in a common book.
STFC-RAL-CR03 R61: The feet popped up this… Workflow with data leakage.
STFC-RAL-CR03 R61: Unlike the consumer AI tools, like ChatGPT or Google Gemini, that send the data to the cloud, our system is entirely local, because our workers are sensitive
STFC-RAL-CR03 R61: We make sure no data brew.
STFC-RAL-CR03 R61: Internet network.
STFC-RAL-CR03 R61: to help
STFC-RAL-CR03 R61: Models run fast, we have reduced ties over the methods, which we talked earlier, and we also project the data in batches.
STFC-RAL-CR03 R61: So, instead of making the model being too heavy, you can't clean and part intake.
STFC-RAL-CR03 R61: To defend the booster.
STFC-RAL-CR03 R61: We aim to build an AI application that runs simultaneously through the cloud for tech model.
STFC-RAL-CR03 R61: by integrating Python here to manage the logic framework, to coordinate complex workflows.
STFC-RAL-CR03 R61: We can leverage the material known as vis-a-prem service.
STFC-RAL-CR03 R61: So, we have the hosted 11 active, you know, that is managed by the group of AI for 50 days. That is run by Jen, and then on the team.
STFC-RAL-CR03 R61: So, we have to… the advantage of this is the rapid development. So, we can scale without managing the physical hardware and reduce complexity.
STFC-RAL-CR03 R61: And allowing the developers to focus only on application logic.
STFC-RAL-CR03 R61: Rather than people.
STFC-RAL-CR03 R61: So… Do we have the Kooling aids yet?
STFC-RAL-CR03 R61: alpaca, and the tuna.
STFC-RAL-CR03 R61: The first one is… works like a strategy during the night. In conclusion, so you can pick whichever models for…
STFC-RAL-CR03 R61: Well, thanks for you.
STFC-RAL-CR03 R61: So, the only thing is, it is private of free, and it needs to be top of your login, and you can switch it.
STFC-RAL-CR03 R61: Oh.
STFC-RAL-CR03 R61: For the second leg, If you want to run your own AI application on the fly.
STFC-RAL-CR03 R61: You can use the check-in link that is equal now.
STFC-RAL-CR03 R61: So, you can join our group.
STFC-RAL-CR03 R61: Just, yeah, to find out more.
STFC-RAL-CR03 R61: Throughout… Just want to highlight that please do… please do link first on the CPD network.
STFC-RAL-CR03 R61: Now we are at the final layer of the top, model validator.
STFC-RAL-CR03 R61: During the initial stages, we have to decide which AI model to use for NER extraction.
STFC-RAL-CR03 R61: So, our first key indicator was the token rulebook.
STFC-RAL-CR03 R61: So, you will soon report the token or the basic unit of test, or think about the delivery, or fragment of words that AI used to process.
STFC-RAL-CR03 R61: and generate the language, well, mathematically. So, we are actually in your custom structure on the tree model.
STFC-RAL-CR03 R61: So, what is the Gemma to…
STFC-RAL-CR03 R61: That is the 9 Boolean parameters. The other one is, mutual parameters, and the last thing is 22.5, that is about 14 Boolean.
STFC-RAL-CR03 R61: primary deaths.
STFC-RAL-CR03 R61: So, activity here at the total output.
STFC-RAL-CR03 R61: Goodbye.
STFC-RAL-CR03 R61: Alright. So just total output of that.
STFC-RAL-CR03 R61: It's much higher than the rate.
STFC-RAL-CR03 R61: And it's important to clarify that.
STFC-RAL-CR03 R61: This higher token output forms, when comparing it with other models, doesn't mean the model is hallucinating for rumbling. We do the heavy lifting, which is actually taking time to attract all relevant categories and provide the detailed reasoning we'll project in our trunk.
STFC-RAL-CR03 R61: And this is what we are looking for, the AI to understand both and how we work.
STFC-RAL-CR03 R61: And this is where I was expecting Commissioner Nemo.
STFC-RAL-CR03 R61: Despite being a 12 millionth parameter, is the 10…
STFC-RAL-CR03 R61: 10, 11 hours, you get October, which is 9, 3 and 4 weeks.
STFC-RAL-CR03 R61: So, this suggests that we find the model is maybe taking the total.
STFC-RAL-CR03 R61: Or likely taking complex entities, or maybe failing to provide some anonymity.
STFC-RAL-CR03 R61: And, however, the smaller models often fail if there are a lot of complexity.
STFC-RAL-CR03 R61: And we will look at the model performed on different categories in the next slide.
STFC-RAL-CR03 R61: So, when we look at the total number of categories, that is the 11 categories, so the attractions seem to be very high. So, whether it's the organization or financial, etc.
STFC-RAL-CR03 R61: And it is only about… 14 other economies come.
STFC-RAL-CR03 R61: And this is really detailed.
STFC-RAL-CR03 R61: So, what makes this special, if you look at the second factor.
STFC-RAL-CR03 R61: So, when I talk about the technical gap between
STFC-RAL-CR03 R61: like FPCA, or PCB, the specific particle physics done.
STFC-RAL-CR03 R61: So, whereas the other model could not identify. So, whereas the plan is able to check all of the… should be able to follow up what we want.
STFC-RAL-CR03 R61: We couldn't get them twice, even for the rent.
STFC-RAL-CR03 R61: 2.5, 13 billion bandwidth model.
STFC-RAL-CR03 R61: This is the general report, etc. How much time it has taken for, Go ahead, pardon.
STFC-RAL-CR03 R61: And the total strong is about $500,000.
STFC-RAL-CR03 R61: So without talking, that would be something to touch on. And do they tend to talk about fund management, and the… and we don't have any problem
STFC-RAL-CR03 R61: Okay, depart.
STFC-RAL-CR03 R61: The GPU infrastructure, we had a maximum output utilization, and the average About, 200 or 400.
STFC-RAL-CR03 R61: Shit.
STFC-RAL-CR03 R61: So, here, we are talking about this test tool, the GUT story.
STFC-RAL-CR03 R61: The literal actual output of our analyzer.
STFC-RAL-CR03 R61: So, out of 211 individual issues, the AI has applied this logic and given its objective distribution. So, 37% is about the hiring
STFC-RAL-CR03 R61: priority, so that means the project managers and the guys actually look at this, and then they can get to decide whether to look about this now or later, and these are the very low issues.
STFC-RAL-CR03 R61: I'm…
STFC-RAL-CR03 R61: But there's something interesting happening here. So, we have something called unknown. So, in the prompt, we have mentioned that if you don't find anything, so please let us know. So, this is what has happened here.
STFC-RAL-CR03 R61: So this is where do I put the woman in the blue? It's, like, for the woman over time.
STFC-RAL-CR03 R61: And this is how we manage from the flat test to the high-grade workflow.
STFC-RAL-CR03 R61: So, talking about these methods.
STFC-RAL-CR03 R61: And it's the past array of, to confirm. That is because
STFC-RAL-CR03 R61: There is a problem with the document from, we didn't understand. So, you need to go back to the prompt, and check it again, and then come back to it. And I understand this, we do have some stickers to it, I'm very high, but it is captured, so we need to check what is actually happening.
STFC-RAL-CR03 R61: Strategic, we have about 650 tokens, and these are the, already dropping 5 seconds, so the whole thing will run about 8 minutes.
STFC-RAL-CR03 R61: So, it's all the… It's about being projected.
STFC-RAL-CR03 R61: So, in the… Second stage, we have the AI running the two CRT expression, and the customization.
STFC-RAL-CR03 R61: So, this is your total amount of what has happened at the… at every level.
STFC-RAL-CR03 R61: So, for the explanatory step, we have also on the different
STFC-RAL-CR03 R61: And the 34, and it's just summarized it, and for the reporting purpose, it,
STFC-RAL-CR03 R61: And the other, tick me.
STFC-RAL-CR03 R61: And it has taken very detailed explanation.
STFC-RAL-CR03 R61: Now, let's talk about the AI edition.
STFC-RAL-CR03 R61: We do not want the fast AI. We also want the AI to be fair and unfair.
STFC-RAL-CR03 R61: So, the air, the whole system is stressed.
STFC-RAL-CR03 R61: It's… the entire system is paragraph.
STFC-RAL-CR03 R61: So, how do we know that AI is fair in one project or another during the analysis?
STFC-RAL-CR03 R61: How do you know if the AI itself is domestic?
STFC-RAL-CR03 R61: So, whether the AI didn't come out.
STFC-RAL-CR03 R61: We meet all these models that are fair, transparent, empower, and accountable as other privacy or other training partner strategy.
STFC-RAL-CR03 R61: So, due to the area which we are currently infected.
STFC-RAL-CR03 R61: The first thing, there is someone in the room. Like, we have seen that. We have detected some issues, so we need to find out. So this is the first step.
STFC-RAL-CR03 R61: One of the things that's to make sure, the repair business, and then giving security, making the proper results.
STFC-RAL-CR03 R61: We do not let the AI go our company, and we have to make sure
STFC-RAL-CR03 R61: There are some tech involved. As you can still need to become a post before.
STFC-RAL-CR03 R61: And to make AI safe and fair, you have not,
STFC-RAL-CR03 R61: tool for the first one. We did the open source Python library, and adjusting the internet of AI and models.
STFC-RAL-CR03 R61: It is very powerful. It allows us to see the balance between the print and fairness. We can also generate reports and pick the best ways to stop any bias we find. This ensures our system is staying smooth and twists every project with the same level of scale.
STFC-RAL-CR03 R61: And then we have a trailback by the government.
STFC-RAL-CR03 R61: And we have got a set guidelines, and what, what, and whatnot.
STFC-RAL-CR03 R61: You have to go to the animal.
STFC-RAL-CR03 R61: Like, and to understand how
STFC-RAL-CR03 R61: we follow the rules and the regulations to meet all the free complement. And you have to download a new
STFC-RAL-CR03 R61: We have the, the questionnaire, where do we have the… about 100 to 200 questions?
STFC-RAL-CR03 R61: We'll need to go through them with our stakeholders of the project.
STFC-RAL-CR03 R61: And we get to decide what are the breakups to be considered in making sure all expenditures are met.
STFC-RAL-CR03 R61: And this is one of the reports that we generated from the survey lab.
STFC-RAL-CR03 R61: So the first one we call it, the fast success rate. So, one of the projects
STFC-RAL-CR03 R61: tour. I thought, It's only about 93% that we are changing about 2%.
STFC-RAL-CR03 R61: And the… the other one is we wanted to check whether
STFC-RAL-CR03 R61: that are being computed or not, and whether there are many issues or not. So, addition to that idea of what's happening using Excel program module, Python modules.
STFC-RAL-CR03 R61: And this is about the hallucination risk.
STFC-RAL-CR03 R61: So, if you want to check all the things, like the fast success rate, and the error invalidation.
STFC-RAL-CR03 R61: And then we do the total what is happening, and then finally we get some sort of, running, which is running. So that means we have… there is some problem with the…
STFC-RAL-CR03 R61: structure of the prompt. So we need to check what's happening, and maybe there's a problem with detection. So that is what we're checking on.
STFC-RAL-CR03 R61: And you do have the data government report, so it's due to, update of what's happening up in that project.
STFC-RAL-CR03 R61: So finally, the conclusion.
STFC-RAL-CR03 R61: This is the roadmap.
STFC-RAL-CR03 R61: What AI applications are in the pipeline in the project of this?
STFC-RAL-CR03 R61: In the next human, you have something called a task categorization.
STFC-RAL-CR03 R61: So, where we categorize the financial data, Provided by the universities.
STFC-RAL-CR03 R61: Is universities have dot their own format or template?
STFC-RAL-CR03 R61: So we need to tackle items like money, or travel, etc. So, this will help to tell project managers, project managers have a sharp manual work.
STFC-RAL-CR03 R61: In the near future, we plan to be able to listen to learn.
STFC-RAL-CR03 R61: From the architect we are building up.
STFC-RAL-CR03 R61: So, this will help us
STFC-RAL-CR03 R61: to… not only for the two-step project, but also to understand what, if anything, had been missed, so we can use it for the
STFC-RAL-CR03 R61: But even if the headquarting development, that's what… Lastly, in the long term.
STFC-RAL-CR03 R61: So we have the predictive forecasting, and the opportunities are in life, like, the generation, and this is what we're trying to build up.
STFC-RAL-CR03 R61: So, this ends my selected dog. I'm happy to say any proximity.
STFC-RAL-CR03 R61: Yes, please.
STFC-RAL-CR03 R61: It's… Have some questions.
STFC-RAL-CR03 R61: Hi. So, any questions about,
STFC-RAL-CR03 R61: should be token outputs for the different models, and with more tokens, and made a comment about, making a decision on the quality of that. How do you determine that more is better, as far as the tokens are concerned?
STFC-RAL-CR03 R61: like you mentioned rambling, how were you able to check that it wasn't just… Excuse me.
STFC-RAL-CR03 R61: So, open my mind, real fast.
STFC-RAL-CR03 R61: Run the model.
STFC-RAL-CR03 R61: So we see the harmony,
STFC-RAL-CR03 R61: The output that is given up.
STFC-RAL-CR03 R61: So, then we will look at the output of every… what we want to the region.
STFC-RAL-CR03 R61: So, based on that, we'll check with the other model.
STFC-RAL-CR03 R61: And then, finally, This is great that maybe we can have the conversation later on, like, about the tokenization.
STFC-RAL-CR03 R61: To think about How we can use it in the future.
STFC-RAL-CR03 R61: That's it.
STFC-RAL-CR03 R61: Okay, because I was interested, the same reason why it's got these different models, and…
STFC-RAL-CR03 R61: You see, in some cases, they behave in a biased way, at least check.
STFC-RAL-CR03 R61: Yeah, to be testing these models for that. So, I see you have this furnace check, software. Did you use that only just with point 3, or were you able to look at the others? Because I think you'd be interesting. Right, again, you didn't see any…
STFC-RAL-CR03 R61: And did they all perform similarly, or it was really out loud? Again, it depends. If you look only from the…
STFC-RAL-CR03 R61: Let's say, mistrial, and it gives the new permanent check.
STFC-RAL-CR03 R61: But in the firm run, you have different terminology type of functions, where you can check, like, halogen within the model, only the 10 model.
STFC-RAL-CR03 R61: So, there are various types that we are looking at. So, we can see, like, the type of token that is being used, or if there is any hallucination, or if there's any, other things, like, you know, the…
STFC-RAL-CR03 R61: Let's jump in.
STFC-RAL-CR03 R61: We will collect the name, and we have the measuring the how the AI is thinking about it. So that, then we can do all that kind of thing.
STFC-RAL-CR03 R61: And you didn't see… do you see any significant differences between the models, or…
STFC-RAL-CR03 R61: But, it's unknown.
STFC-RAL-CR03 R61: Joe, when we talk about the… it's not much different, unless…
STFC-RAL-CR03 R61: In terms of the NBA attraction, Indeed!
STFC-RAL-CR03 R61: That, the second chase, and we should… we started a little bit of,
STFC-RAL-CR03 R61: different. Like, actually, gamma model is having a good quality, because it would be the type of data set that is being trained on. But when you compare with the 12 mod,
STFC-RAL-CR03 R61: It will cover, like, slightly better.
STFC-RAL-CR03 R61: Yes, yes. I mean, we have different type of,
STFC-RAL-CR03 R61: one of the pair, but we have another thing, like, AI4, so we can filter some texting on stage as well.
STFC-RAL-CR03 R61: Oops.
STFC-RAL-CR03 R61: I didn't quite understand how the validation works, where you're…
STFC-RAL-CR03 R61: From the point of view of the…
STFC-RAL-CR03 R61: Human in the loop, in that, is the, other… do you have, then, people
STFC-RAL-CR03 R61: Checking, or experts checking, that the output of the model
STFC-RAL-CR03 R61: for a subset, or, I mean, how do they… because I think if the people can do everything, then you don't need the AI, so how do you…
STFC-RAL-CR03 R61: Good luck.
STFC-RAL-CR03 R61: This is a great documentation.
STFC-RAL-CR03 R61: So, we have been…
STFC-RAL-CR03 R61: explanation summary. So, for every score that the AI will test for, the PM or the BIs will read that explanation summary. So, that is a woman in the loop.
STFC-RAL-CR03 R61: So, for example, I mean, they don't go through all of the issues, so our idea, we want to look at the top 10 issues, or top 20 issues. So, and that, we filter them out, and then we will look at the expense internally.
STFC-RAL-CR03 R61: So this is where the woman in the loop's happening, so the PI will check that summary. Yeah, okay. If it's marked something as not critical as a phone.
STFC-RAL-CR03 R61: Yes. How do you check? Do you have some fraction of those also checked that they are, that they really are low priority? Yeah. I mean, we had that incident happen, because one of the actual issues is actually important, but it has gone to a very low.
STFC-RAL-CR03 R61: So, that is where the Fairland helps us to find out. I mean, there's some… there's some issues with the structure.
STFC-RAL-CR03 R61: The… there are two ways happening. One is the problem with the JSL format of the front.
STFC-RAL-CR03 R61: And the second one, if you remember the product module, we need to define the
STFC-RAL-CR03 R61: We need to… to highlight what are the problems that we are facing, and we need to define clear. So, then, for example, we have a setup example to… to highlight, okay, for this issue, it needs to be, the high risk, or higher price, or medium. So.
STFC-RAL-CR03 R61: It's a PI or a PM.
STFC-RAL-CR03 R61: say that, okay, this is… the calculation is wrong. Then we go back to the wrong, and we add an example back to the
STFC-RAL-CR03 R61: Back to the wrong system. So then, again, we run a retexture model again.
STFC-RAL-CR03 R61: Is that how it works?
STFC-RAL-CR03 R61: Yeah, what fraction… Of your… of all the things are checked by human.
STFC-RAL-CR03 R61: At the moment,
STFC-RAL-CR03 R61: And so far, the previous project managers are doing one of them, because, like… So they have their own checks.
STFC-RAL-CR03 R61: It's, yeah, I mean… Because since we are implementing this for the first time.
STFC-RAL-CR03 R61: So, they go through all of them.
STFC-RAL-CR03 R61: Actually. But when the next time, when the new issues get added up.
STFC-RAL-CR03 R61: It doesn't run the whole thing once again, but it takes a new issue, and it used to report on that particular new issue.
STFC-RAL-CR03 R61: So, for example, the students have added up, like, maybe 2 or 3 times in a week.
STFC-RAL-CR03 R61: So…
STFC-RAL-CR03 R61: Sometimes it can be a lower issue or a higher issue, so it keeps running whenever the issues are up, and you choose the prompt in between.
STFC-RAL-CR03 R61: To, at least We set up a tiny type of…
STFC-RAL-CR03 R61: every Friday to run the issues, and give us very short report, saying that, okay, for this issue, what has happened this week, these are the problems, these are the scores, and then we compare with the entire data chain.
STFC-RAL-CR03 R61: Thank you.
STFC-RAL-CR03 R61: Yes, sure.
STFC-RAL-CR03 R61: Oh yeah, sorry,
STFC-RAL-CR03 R61: Do you get… do you get re… okay, I don't get so too popular. Do you get reproducible?
STFC-RAL-CR03 R61: Results, if you run this.
STFC-RAL-CR03 R61: Twice.
STFC-RAL-CR03 R61: on.
STFC-RAL-CR03 R61: refer to the… I mean, do you mean that, without being… So you run this loop, and then you get your text summarization and your explanation summary. You do that the same twice on…
STFC-RAL-CR03 R61: the same input? Do you get the same…
STFC-RAL-CR03 R61: I mean, planting air planted. I mean, it runs, like,
STFC-RAL-CR03 R61: First, it's run on the parental terminal.
STFC-RAL-CR03 R61: After you need mark calculation with quantity explanation summary.
STFC-RAL-CR03 R61: After finishing this, we come back to the test summarization.
STFC-RAL-CR03 R61: You have some set of things you're doing. Do you get the same result if you do it twice?
STFC-RAL-CR03 R61: Oh, okay, got it. I mean, when you run it, yes, yeah. And because that is where we define the prompt. We made sure that we
STFC-RAL-CR03 R61: That we highlighted that mixture doesn't change every time.
STFC-RAL-CR03 R61: So, we did that test before. When we gave very generalized prompts.
STFC-RAL-CR03 R61: And, we tend to exchange it maybe two or three times.
STFC-RAL-CR03 R61: But then we defined… we made a series of check-up prompts, and then when we ran for the second or third time, it remained continuous. Unless…
STFC-RAL-CR03 R61: when the project managers, because the issue doesn't remain changed for the entire project, so it keeps changing. So, maybe after 2 or 3 weeks, they update the issue. So, again, you want it.
STFC-RAL-CR03 R61: And then it also highlights, I mean, we wrote the prompt in such a way that it also highlights what is the previous score, and this is the current score.
STFC-RAL-CR03 R61: So, that is one of the tech…
STFC-RAL-CR03 R61: Thank you for that idea. That is one of the steps that we are going to put. What was the previous, and if there is any changes that happened. Okay. And then a kind of follow-up question, so…
STFC-RAL-CR03 R61: If you run this on an experiment like G, So June is a publicly…
STFC-RAL-CR03 R61: There is public information out there, right?
STFC-RAL-CR03 R61: So there are articles in New Scientists saying June is a deeply troubled project from, you know, 3 or 4 years ago. So then that sentiment analysis goes into the LLM, which then thinks.
STFC-RAL-CR03 R61: things differently about that compared to, say, HyperK or, you know, Atlas or CMS, depending on the publicly available information that it has. So if you try changing the experiment name, so you could change it to a generic one that doesn't exist, you could start calling it good experiment or bad experiment, and see if it actually makes a difference to your output.
STFC-RAL-CR03 R61: Don't.
STFC-RAL-CR03 R61: to try to avoid that, this is one of the fair Python modules, It has one feature that…
STFC-RAL-CR03 R61: We can find out whether the AI is by us or not.
STFC-RAL-CR03 R61: That's one… one of the things that I'm touching on. But the second one, again, back to the prompt, that we need to make sure, again, we define the
STFC-RAL-CR03 R61: Particular, prompting that, okay, we need to follow this.
STFC-RAL-CR03 R61: And do not, you know, they do not provide any other
STFC-RAL-CR03 R61: idea, apart from this on Lake.
STFC-RAL-CR03 R61: There's a difference between the prompt and knowing that the AI is, is actually doing it.
STFC-RAL-CR03 R61: I mean… Aren't they always…
STFC-RAL-CR03 R61: You can say in the prompts, be fair, but I don't know if you can verify that it's being fair without trying to trick it somewhere.
STFC-RAL-CR03 R61: So, in that case, we… when we look at the results from the, you know, the PI or the project manager.
STFC-RAL-CR03 R61: That's where we look at the text summarization.
STFC-RAL-CR03 R61: So we try and see if there's any change in the behavior, or something like that.
STFC-RAL-CR03 R61: So that is one of the checks that we're doing, look.
STFC-RAL-CR03 R61: Alright. That's good.
STFC-RAL-CR03 R61: Yes. That was sort of overlap.
STFC-RAL-CR03 R61: Okay.
STFC-RAL-CR03 R61: Can I… sorry, this is a very specific question.
STFC-RAL-CR03 R61: if you're… you're running on the PPD, AI machine that Jens has set up, I understand,
STFC-RAL-CR03 R61: what sort of times do you run this? Because I'm also using this one.
STFC-RAL-CR03 R61: Stop.
STFC-RAL-CR03 R61: It's a different segment.
STFC-RAL-CR03 R61: Sandeep's running these as condo jobs, right? Yeah. So they get submitted to the same type of machines, but because this machine is taken out of the queue, they're not competing. They're competing with other people submitting GPU jobs.
STFC-RAL-CR03 R61: Okay, maybe I should do this, I think this is…
STFC-RAL-CR03 R61: I was just wondering whether we'd be competing.
STFC-RAL-CR03 R61: Sounds good.
STFC-RAL-CR03 R61: So, that's quite interesting that these are run as… They can run on many machines, can they? Yes. I mean, we have only set up for 5 GPUs. So, what I have done is, in the condo, and with LipJo.
STFC-RAL-CR03 R61: I have checked the time, like, every Friday.
STFC-RAL-CR03 R61: 6 o'clock in the evening, after the end of the day.
STFC-RAL-CR03 R61: So… And I'll try to see if there is any other…
STFC-RAL-CR03 R61: whether the… whether the GPUs on both time.
STFC-RAL-CR03 R61: I try not to interrupt or dictate related to
STFC-RAL-CR03 R61: lab, if I remember correctly. It's a 12-minute job, I don't think it matters. Yeah, if it's a 12-minute job once a week, I think.
STFC-RAL-CR03 R61: But I think, because of the condo job, it won't, on… based on their priority.
STFC-RAL-CR03 R61: Maybe.
STFC-RAL-CR03 R61: Sorry, they were just wondering whether, that seems to be quite…
STFC-RAL-CR03 R61: If you're always running it on a Friday evening, is that the best time to get
STFC-RAL-CR03 R61: results, because people then don't have time to look at them until Monday. Normally, the…
STFC-RAL-CR03 R61: But the last week of August, we will look on Monday.
STFC-RAL-CR03 R61: So we have the system set up.
STFC-RAL-CR03 R61: So, for the previous week, last week of the update, like, in the chat will be updated on Monday. So, we collect all the data.
STFC-RAL-CR03 R61: that this happened until due Friday.
STFC-RAL-CR03 R61: So, at the moment, nobody's looking on Fridays, but the report that is generated, it goes to the email, and they're looking at whenever
STFC-RAL-CR03 R61: pandemic.
STFC-RAL-CR03 R61: Yeah, so coming back to the, the issue severity analysis stuff…
STFC-RAL-CR03 R61: So, the main purpose of that is to generate this summary document, is it, that would then be read by PIs and PMs, or is that for…
STFC-RAL-CR03 R61: what do you call it, Rose, that thing? Task managers or workstream managers as well.
STFC-RAL-CR03 R61: It's different. I mean, normally it's just we call… we call each other project stakeholders, so who are involved in that.
STFC-RAL-CR03 R61: particular work package, or something like that. So, normally, the project manager will work along with the PIs, or work package managers, and they coordinate with each other. So, whatever the project manager said, that information will be shared
STFC-RAL-CR03 R61: Yeah, so, because in that case, yeah, so I work on, one of the, you know, one of the workstream managers for the CMS upgrade side, for the hardware work package, and,
STFC-RAL-CR03 R61: So there, we often… we have chats with Lei every… every month or two to, update on the issues.
STFC-RAL-CR03 R61: And then we also update the risk register as a result of that, but then…
STFC-RAL-CR03 R61: what I was thinking was that at that stage, then, we're effectively ranking the severity of each of the issues in the risk register itself, right?
STFC-RAL-CR03 R61: So then, yeah, we've kind of fed in all of the information, we know all of the information for this, so the main purpose is then for the DI to be able to read this as a summary, or…
STFC-RAL-CR03 R61: Yeah, I'm trying to think out of this to the… into the workflow.
STFC-RAL-CR03 R61: Yeah, I mean, that is what we planned. At the moment.
STFC-RAL-CR03 R61: This is particularly trademarked me, because
STFC-RAL-CR03 R61: the project managers are focused more on the project… on the risk register, so they are handling it by themselves. At some point, we will work… we will do the AI on the risk register as well. But we have, in our primary software, we have the… our own Monto Pablo analogy.
STFC-RAL-CR03 R61: And where they feed into the information, and they calculate, and they come back to it.
STFC-RAL-CR03 R61: So, because… Sometimes the issues are too long, or so…
STFC-RAL-CR03 R61: It changes all the time, very frequently, so there is some sort of
STFC-RAL-CR03 R61: summary of what's happening, and then they decide what happens, with their risk register or something. So normally, I think some days, too, we update the risk register, and, password. So, it's like…
STFC-RAL-CR03 R61: Okay, but then the future change that you mentioned around the AI, the risk register, would that be the… one of these, AI setups taking the risk register, the project.
STFC-RAL-CR03 R61: Workstream… the workstream managers have…
STFC-RAL-CR03 R61: Updated with the… with the project manager, and using that as input, and then, using that to do some kind of analysis of likely future costs.
STFC-RAL-CR03 R61: It is, I'm… When we talk about the future, I mean, I would expect not.
STFC-RAL-CR03 R61: And near future, because…
STFC-RAL-CR03 R61: which is very sensitive management, so we need to understand a bit more. Okay. So once our… once we get comfortable about it, and then we'll start working on the concept of the AI model.
STFC-RAL-CR03 R61: Okay, okay.
STFC-RAL-CR03 R61: I'm good, but…
STFC-RAL-CR03 R61: as soon as we start, we will come to know about it, and this is one of the pipelines that is happening in the future of now. So, at the moment, we are looking at the gap.
STFC-RAL-CR03 R61: That's a good idea. It takes a bit of time, because you need to analyze the data, the financial data. Right.
STFC-RAL-CR03 R61: So hopefully, maybe after this, maybe we can update this job, hopefully.
STFC-RAL-CR03 R61: Okay, and that would be to… to…
STFC-RAL-CR03 R61: rank items based on the risk register? Oh, is that? We do have the system, I mean, already we have this top term of a software that is automatically pipeline.
STFC-RAL-CR03 R61: But at the moment, the project managers, they're feeding the numbers.
STFC-RAL-CR03 R61: Yeah, so… Again, when we collect that information.
STFC-RAL-CR03 R61: at some point, we will be going to find a prompt and then make it work another way. Okay. But you need to be a little bit careful.
STFC-RAL-CR03 R61: That is about the risk, and because there's other factors, you know, like the cost.
STFC-RAL-CR03 R61: And the tile, and the chip wall. Yeah. So…
STFC-RAL-CR03 R61: You need to find out, like, how they work together. I mean, to say that that becomes a big pipeline.
STFC-RAL-CR03 R61: Right. So, we need to… we need a bit of time to look on that. Okay. Well, yeah, I mean, in a sense, I would have thought the risk register is easier to analyze because it's effectively just a set… actually, what I remember is that it's a set of numbers of the probabilities and the cost ranges, associated with risks.
STFC-RAL-CR03 R61: And so that's already quantitative, whereas…
STFC-RAL-CR03 R61: So it could just be ranked based on that without AI, whereas the issue is the text, so…
STFC-RAL-CR03 R61: maybe lend themselves more to a tech sub. Yes, and one of the factors is, the Monte Carlo.
STFC-RAL-CR03 R61: Which we haven't started working on that yet.
STFC-RAL-CR03 R61: So, we need to understand a bit about the Monte Carlo, and then we'll start looking at the AIS.
STFC-RAL-CR03 R61: Because…
STFC-RAL-CR03 R61: we need to run the model, for the Monte Carlo iteration, like, thousand to a thousand times and 3, and so this is where you don't really understand that Monte Carlo analogy from the AI point of view.
STFC-RAL-CR03 R61: till I'm doing… but… But would… would you consider replacing that Montecolor analysis with an AI one instead?
STFC-RAL-CR03 R61: No, I mean… If you are talking about the normal population, that can be done.
STFC-RAL-CR03 R61: But, we want the Monte Cablo as well, because… Okay.
STFC-RAL-CR03 R61: So, because we want to…
STFC-RAL-CR03 R61: let's say, unlike the delay in path, unlicatur, and what permit, takes probably the, 50% or 90%. So.
STFC-RAL-CR03 R61: the running beta calculation is what the one you're
STFC-RAL-CR03 R61: to application of Quanto Carlo, is where…
STFC-RAL-CR03 R61: It requires a little bit of detailed understanding. So, which is why we are relying on our primary software to do that sort of analysis for us at the moment. So, at the moment, that is why we are manually creating
STFC-RAL-CR03 R61: We are trying not.
STFC-RAL-CR03 R61: Okay.
STFC-RAL-CR03 R61: We have a raised hand on Zoom. Chris.
Christopher Townsley - STFC UKRI: Yes, I think…
STFC-RAL-CR03 R61: So, would you like to speak?
Christopher Townsley - STFC UKRI: Yeah, it was just about the question before about potential bias on the sentiment analysis, say if a project has got into the news. I think my understanding is that this is a completely closed system, so wouldn't actually have access to the news, so this wouldn't creep in. Is that correct, Sandeep?
STFC-RAL-CR03 R61: So…
STFC-RAL-CR03 R61: So, the question was, depending on when the model was trained, that knowledge is part of the model, because it might have been based on text that is out in the open.
STFC-RAL-CR03 R61: So once it has no access to external systems and can't query the web.
STFC-RAL-CR03 R61: It might have been trained in that way beforehand, and it's a bias that is inherent to the model that then needs to be… you need to figure out whether it's in there or not.
STFC-RAL-CR03 R61: Sorry, paraphrasing, Stuart.
STFC-RAL-CR03 R61: Yes, Chris, if I sat me.
STFC-RAL-CR03 R61: Okay.
STFC-RAL-CR03 R61: Oh, I appreciate you.
STFC-RAL-CR03 R61: Okay, so, are there other questions?
STFC-RAL-CR03 R61: I have one last question, maybe. I was curious how to measure the
STFC-RAL-CR03 R61: how you quantify the input data, and I… yeah, that's… we lost this line, but…
STFC-RAL-CR03 R61: I think on the first slide, or second one, you've had,
STFC-RAL-CR03 R61: The number of words. So, could you explain a bit what the meaning of that… that is?
STFC-RAL-CR03 R61: It's looked like a little number of words, but maybe it has another meaning, which… Yes.
STFC-RAL-CR03 R61: So what is this amount of data that you, your users?
STFC-RAL-CR03 R61: What that's available at the moment. At the moment, we have the exclusive log.
STFC-RAL-CR03 R61: Yes, the… when we worked on the project, we entered all that information.
STFC-RAL-CR03 R61: So, like, for example, there is a crystal with an history, or a CPA, or any sort of car.
STFC-RAL-CR03 R61: So, it extracts all of that information, and it counts all of it, and we trust it for the 1700.
STFC-RAL-CR03 R61: So that is the input data from the juice log.
STFC-RAL-CR03 R61: So you only use the issues log? At the moment, because we are focusing on the issues management, so this is one of the specific model of the project management, so we are focusing on the issues we are. So this is the data that we've got, it's out there.
STFC-RAL-CR03 R61: And this is only for the open districts family, so we have closed issues, it's more than…
STFC-RAL-CR03 R61: But this is, already open to about, 200 pictures for the entire project regular manager.
STFC-RAL-CR03 R61: Oh, my seat.
STFC-RAL-CR03 R61: Okay, thank you very much. So, let's thank Sandeep again, and we close it.
STFC-RAL-CR03 R61: Yeah, please, like, some cookies, and, lunch. And lunch.
STFC-RAL-CR03 R61: Lots of applause from Zoom, so it was very nice.