WEBVTT 00:00:02.000 --> 00:00:07.000 Good morning. Thank you for being here. 00:00:07.000 --> 00:00:14.000 There is no shortage of talk. regarding its crucial potential. 00:00:14.000 --> 00:00:20.000 I'm excited to move beyond that hype. And during what we have actually done here. 00:00:20.000 --> 00:00:29.000 We have created a system. That is designed to support our project managers, PIs, and dual-time statistical methods. 00:00:29.000 --> 00:00:34.000 My goal today is to focus on the theory and practical solution. 00:00:34.000 --> 00:00:40.000 So, I want to keep this high level analysis. 00:00:40.000 --> 00:00:44.000 how this goes after it works rather than going to the code. 00:00:44.000 --> 00:01:00.000 So, just a heads up. This presentation is a global presentation which is a lot of sci-fi visuals to help demonstrate the data journey and the future we are building. 00:01:00.000 --> 00:01:09.000 So in today's talk. With the AI come later. 00:01:09.000 --> 00:01:14.000 many good life. We will look at the specific problem. 00:01:14.000 --> 00:01:26.000 that the BIA can be financially. and independent and the methodology which I will walk through. 00:01:26.000 --> 00:01:34.000 And I'll be actually done from the raw data into actionable insights? 00:01:34.000 --> 00:01:42.000 Then we will look at the… And then the code of the AI engine, the IT infrastructure, and how we optimize the model. 00:01:42.000 --> 00:01:56.000 And finally. framework we use to ensure our AI is fair and unbiased. 00:01:56.000 --> 00:02:08.000 Before I start, I want to say that this implementation is a direct result of the operating system, and it has been a great opportunity. 00:02:08.000 --> 00:02:19.000 To apply the training to our department specific challenges. 00:02:19.000 --> 00:02:25.000 So about the business problem. We are looking at one of the specific project management model. 00:02:25.000 --> 00:02:33.000 No, not the resource management. Since 2020, we have maintained. 00:02:33.000 --> 00:02:42.000 detailed history below. And today, we are tracking over 200 active issues. 00:02:42.000 --> 00:02:47.000 That is almost 17,000 words I've written down in. 00:02:47.000 --> 00:02:58.000 Currently, our project managers, PI, growth package managers are maintaining this log as well as driving the project delivery. 00:02:58.000 --> 00:03:03.000 at the same time. So when I then had to manually. 00:03:03.000 --> 00:03:13.000 go through all of this. And decide what Mr. Immediate attention. It creates what I call as the parietal gap. 00:03:13.000 --> 00:03:19.000 And this gap has got two major risks. So one is the latency. 00:03:19.000 --> 00:03:32.000 Um, this is a typical delay. This is difficulty about, um… about the issue being updated and stakeholders actually see it. 00:03:32.000 --> 00:03:38.000 In the multi-million project, time is a luxury which will always go high. 00:03:38.000 --> 00:03:53.000 And the second most risk is subjectivity. We are often seen a class of perspectives. For example, a growth party manager may naturally see an issue in their domain a top priority. 00:03:53.000 --> 00:04:05.000 Right? The project manager, looking at the project launched, may clarify that issue as a medium or locality. So without the standards. 00:04:05.000 --> 00:04:20.000 We do not have the ranking become more inconsistent. Therefore, our PIs and project managers need to see the big picture instantly without spending… I mean, without spending too much of. 00:04:20.000 --> 00:04:26.000 No, um… Okay. 00:04:26.000 --> 00:04:31.000 So, that challenge we are struggling between that gap. 00:04:31.000 --> 00:04:40.000 transferring all the technical manual logs into an immediate actionable. 00:04:40.000 --> 00:04:48.000 Okay. 00:04:48.000 --> 00:04:51.000 This is the entire map. AI TikTok. 00:04:51.000 --> 00:05:09.000 The model we use the AI model is 12.5 billion. This isn't a simple question and answer data, which are multi-stage hybrid pipeline where Python and AI work together. 00:05:09.000 --> 00:05:19.000 We can do the workflow displayed in 15 stages. One with the data from any extraction, and it used to be analyzed test. 00:05:19.000 --> 00:05:30.000 I'll talk about this too in a moment. What made this workplace pressure is how different groups work together. 00:05:30.000 --> 00:05:38.000 We have a Python J. Uh, managing the high precision tasks, like pulling the data. 00:05:38.000 --> 00:05:50.000 I do external and data privacy marketing. Meanwhile, the AI handles the language that is the attraction, the utility and the creation and sectorization. 00:05:50.000 --> 00:06:01.000 They're all familiar with each other. The entire process is controlled by the front model. 00:06:01.000 --> 00:06:13.000 that defined the rules of engagement, telling the AI exactly how to think to ensure it means our specific challenges. 00:06:13.000 --> 00:06:19.000 data privacy market later on, but in the next few slides. 00:06:19.000 --> 00:06:32.000 We will drop through the drum and the real engine and the disket. 00:06:32.000 --> 00:06:41.000 Now we are looking at the plumb. So, if you look at the map, what we just saw on last slide here. 00:06:41.000 --> 00:06:45.000 So this is a blueprint of an entire system. 00:06:45.000 --> 00:06:51.000 These people are caught playing by putting all of the rules or prompt into this very young. 00:06:51.000 --> 00:07:10.000 This meant our system very modular. So, if we need to all get anything in the background or anything related to the, you know, the instruction, and we only update in the prompt area to the left remaining the same. 00:07:10.000 --> 00:07:15.000 But yeah. attracted the wrong. 00:07:15.000 --> 00:07:21.000 We used example to teach the AI. also known as the fine-tuning. 00:07:21.000 --> 00:07:27.000 And this gave the AI exactly what item to look for. 00:07:27.000 --> 00:07:37.000 But if you have the analysis, that is the gut model, we provide a clear instructions so that the AI knows what to do. 00:07:37.000 --> 00:07:46.000 And Trump is… the prompt is split into four parts. The system prompt and the user prompt, which I'll show you in the next slide. 00:07:46.000 --> 00:07:52.000 the system prompt defines the AI program and how it needs to act. 00:07:52.000 --> 00:07:58.000 And the user program is a set of tasks that tells API is what to do. 00:07:58.000 --> 00:08:04.000 So what made this area unique is the helper function. 00:08:04.000 --> 00:08:12.000 We have built into it. They work like a system. Before the AI looked into the world, these tools. 00:08:12.000 --> 00:08:24.000 extraction, cleaning, and other tasks like privacy marketing. And once the AI is done, these students wrote the project data back into our database. 00:08:24.000 --> 00:08:39.000 By doing the heavy work of formatting here, we allow the AI to process only on the region and maintaining laying on campus. 00:08:39.000 --> 00:08:54.000 So, to make model work, we balance the AI with our local hardware and our data types. First, we used a method called quantitative or optimization, which runs the model at a reduced size. 00:08:54.000 --> 00:08:59.000 We've helped the system to run faster on our local cluster. 00:08:59.000 --> 00:09:10.000 If the mathematical way of making the model smaller. So it uses the less memory while giving the high qualities. 00:09:10.000 --> 00:09:16.000 The raw data comes from the toaster. Um, and what will be the function? 00:09:16.000 --> 00:09:28.000 And so we are on the other side. We have the prompt that I didn't talk about earlier, victim under the problem. 00:09:28.000 --> 00:09:34.000 So to get that. We are… we have the coordination layer that we look at. 00:09:34.000 --> 00:09:40.000 And this player, I thought that it will generate a report. 00:09:40.000 --> 00:09:53.000 So, by keeping the rule top right under, we have created a system that is far, fair, and ethical. 00:09:53.000 --> 00:09:58.000 So we will look at the third one, and we are equator. 00:09:58.000 --> 00:10:06.000 So what will be the end lifetime for these main entity recognition? 00:10:06.000 --> 00:10:16.000 So this is the area, the AI will that automatically find the label for key information from the raw technical laws. 00:10:16.000 --> 00:10:22.000 into a defined type of things that you like to normalize in entities. 00:10:22.000 --> 00:10:31.000 So, under using our tool. We move to the middle area, where we do not look only the worst. 00:10:31.000 --> 00:10:36.000 We are finding relevant data. By the good mean then. 00:10:36.000 --> 00:10:41.000 really done a lineup L into an ordered database. 00:10:41.000 --> 00:10:46.000 So, why are we doing this? The reason is contact database. 00:10:46.000 --> 00:10:56.000 Which we don't really have. I mean, we have a lot of documents, we have a report in the past, but they are not easily searchable. 00:10:56.000 --> 00:11:01.000 This book, we are building a base for the social knowledge graph. 00:11:01.000 --> 00:11:26.000 Yeah. So this mixture of the data we find in 2020 or previous data says this filter PI and PM in 2026 and beyond. 00:11:26.000 --> 00:11:37.000 We went on to return to… Once we have the tactile data, we must call the project management time. 00:11:37.000 --> 00:11:48.000 from the prong, we have a defined setup instruction telling the AI what to predict between 1 to 10, depending on the nature of the issue. 00:11:48.000 --> 00:11:53.000 And this is happening fast during the harvest and harvest up manually work. 00:11:53.000 --> 00:11:57.000 In the past, ranking them was a matter of opinion. 00:11:57.000 --> 00:12:06.000 So one percentage tells you is a high risk, while the other says. So our AI chopped up by us. 00:12:06.000 --> 00:12:15.000 If you look at every issue on a clear 1 to 10K. 00:12:15.000 --> 00:12:21.000 But we do not drop the AI blindly. We also will be a guardrails. 00:12:21.000 --> 00:12:31.000 within the… our road, we have mandatory minimum. For example, if the AI finds a risk to a milestone or a lockup. 00:12:31.000 --> 00:12:36.000 Lots of each job. It cannot just blur that debt level. 00:12:36.000 --> 00:12:43.000 This is how we put our expertise around AI and logic. 00:12:43.000 --> 00:12:46.000 The one thing you need here is the AI will look. 00:12:46.000 --> 00:12:53.000 Only, uh, about degrading and the reasoning. And the Python would do the math. 00:12:53.000 --> 00:13:03.000 So I think making the entire project faster with minimum memory and of any component requirement. 00:13:03.000 --> 00:13:10.000 So, from this, we have 3 distinct regions. One is the boat. 00:13:10.000 --> 00:13:17.000 The other one with the project manager will look at the implementation summary. 00:13:17.000 --> 00:13:30.000 Oh, right? And for the important focus. We need, uh, the check summarization. 00:13:30.000 --> 00:13:36.000 Now you are also the next pillar, the IT infrastructure and optimization. 00:13:36.000 --> 00:13:49.000 So, our current infrastructure business. It runs on our local cluster, it uses the power to execute like NVIDIA 100 or Tesla 300. 00:13:49.000 --> 00:13:58.000 So you have this tool using condo system. Which manages to work across the world, neglecting the common voice. 00:13:58.000 --> 00:14:06.000 If he popped up this… The data leakage. 00:14:06.000 --> 00:14:18.000 Unlike the consumer AI tools like ChatGPT or Google Gemini that generic data to the cloud, our system is entirely local because our logs are sensitive. 00:14:18.000 --> 00:14:23.000 We went to know where they can't believe. Uh, internal network. 00:14:23.000 --> 00:14:34.000 To help models run fast, we have reduced the time earlier, and we have to project the data in batches. 00:14:34.000 --> 00:14:44.000 So, instead of making the model too heavy, you can clean and pack each. 00:14:44.000 --> 00:14:53.000 We aim to build an AI application that runs completely through the cloud architect model. 00:14:53.000 --> 00:15:00.000 By integrating Python kit to manage the logic firmware and to coordinate complex workflows. 00:15:00.000 --> 00:15:17.000 We can leverage the technology. So we have the question a lot of active that is managed by the group called AI for quickly does it run by Jen and paying. 00:15:17.000 --> 00:15:28.000 So we have the advantage of this is the rapid development. So you can scale without managing the physical hardware and reduce competency. 00:15:28.000 --> 00:15:34.000 and allowing the developers to focus only on application logic. 00:15:34.000 --> 00:15:39.000 Rather than being positive. So we have accumulates here? 00:15:39.000 --> 00:15:54.000 One is the alpaca and the Rituna. The first one is just like a ChatGPT during Gemini and concluded the general model for. 00:15:54.000 --> 00:16:04.000 What? Like for you. Yeah, the only thing is private and free, and it needs to pick up your login, and you can wait. 00:16:04.000 --> 00:16:14.000 Oh, for the second link, if you want to run your own AI application on the fly, you can use the check-in link that is written now. 00:16:14.000 --> 00:16:21.000 So we can join our group. Um, just, yeah, to find out more. 00:16:21.000 --> 00:16:34.000 So just want to highlight that this tooling was qualified network. 00:16:34.000 --> 00:16:39.000 Now we are at the final pair of the talk, Model Validation. 00:16:39.000 --> 00:16:46.000 During the administrative stages, we have to decide which AI model to use for any adoption. 00:16:46.000 --> 00:16:53.000 So, our first key indicator was the token group. 00:16:53.000 --> 00:17:03.000 So, you will quickly put the token or the basic unit of test, or finger match or still available or fragment of words that AI used to process. 00:17:03.000 --> 00:17:11.000 and generate the language, mathematically. So you are actually in your question on the three models. 00:17:11.000 --> 00:17:27.000 So, when they take them out to… And the last is 22.5 that is about 14 billion parameters. 00:17:27.000 --> 00:17:33.000 So, I couldn't give a total output. 00:17:33.000 --> 00:17:45.000 Goodbye. Okay. So tutorial output of them… It must have done correctly. 00:17:45.000 --> 00:18:03.000 And if it's important to clarify that. This higher broken output points when comparing with other models such as meaning the model is highly proof of models we do the heavy lifting. This is actually taking time to extract all the learned categories. 00:18:03.000 --> 00:18:08.000 And provide a detailed reasoning will protect in our prompt. 00:18:08.000 --> 00:18:16.000 And this is what we are looking for, the AI to understand both and how we work. 00:18:16.000 --> 00:18:21.000 And this is where it was definitely chemical memo. 00:18:21.000 --> 00:18:32.000 Despite being a 12 million parameter is the 10… 00:18:32.000 --> 00:18:44.000 It may be taking the short term. or likely typing complex entity or maybe failing to provide an analytic. 00:18:44.000 --> 00:18:52.000 often fails at this level of complexity. 00:18:52.000 --> 00:19:02.000 And we will look at the model performed on different categories in the next slide. 00:19:02.000 --> 00:19:18.000 So, it looks like a total number of categories, that is 11 categories, so to attract them seem to be very high. So, for those organizations or financial, etc. 00:19:18.000 --> 00:19:24.000 And it is only about… for the opportunity to come. 00:19:24.000 --> 00:19:32.000 And this is pretty detailed. If you look at the settings that go. 00:19:32.000 --> 00:19:43.000 So, when I talk about the technical term between… Like, if we appreciate it or PCP, the specific particle is done. 00:19:43.000 --> 00:19:55.000 So whereas the plan is able to tap all of the, to be able to follow up what we want. 00:19:55.000 --> 00:20:03.000 price even for a brand. 00:20:03.000 --> 00:20:16.000 And due to the gender. The total crop is about 500,000. 00:20:16.000 --> 00:20:38.000 And the and we don't have any problem. The GPU infrastructure. We had a maximum output utilization and the average. 00:20:38.000 --> 00:20:46.000 uploaded on 200. 00:20:46.000 --> 00:20:52.000 Yeah. So here we are talking about the test tool, the GMUT story. 00:20:52.000 --> 00:21:09.000 So, output of our analyzer. So, out of 211 individual issues, the AI has applied this logic and given its objective distribution. So if the 37% thinks about the high. 00:21:09.000 --> 00:21:23.000 Priority product manager, and then they will get to decide whether to look about this now or later. These are the very low issues. 00:21:23.000 --> 00:21:38.000 And… But it is something interesting happening here. We have something called unknown. So in the front, we have mentioned that if you don't find anything, so please let us know. So this is what has happened here. 00:21:38.000 --> 00:21:56.000 It's like part of human power type. workflow. 00:21:56.000 --> 00:22:02.000 So, talking about these methods. 00:22:02.000 --> 00:22:17.000 That is because there is a problem with the AI and the second term we can understand. So you need to go back to the prompt and check it again, and then come back to it. 00:22:17.000 --> 00:22:26.000 I understand this. This will have some tickets you can verify, but it is faster, so we need to check what is actually happening. 00:22:26.000 --> 00:22:46.000 about eight minutes. So all the checkout being projected. 00:22:46.000 --> 00:22:56.000 So in the… Second thing is, we have the AI running the two realities, extraction and the customerization. 00:22:56.000 --> 00:23:01.000 So, this is your total amount, but what has happened at every level? 00:23:01.000 --> 00:23:20.000 And the customization for the reporting purpose is, uh… 00:23:20.000 --> 00:23:28.000 And very interactive and very detailed explanation. 00:23:28.000 --> 00:23:38.000 Now, let's talk about the AI edition. We do not want to pass AI. We also want the AI to be fair and unfair. 00:23:38.000 --> 00:23:45.000 So, the AI, the whole system used this with the entire system is back. 00:23:45.000 --> 00:23:52.000 So how do we know that AI failed in one project or the another during the analysis? 00:23:52.000 --> 00:23:59.000 How do you know if AI is domestic? So, without the AI decision to come out. 00:23:59.000 --> 00:24:13.000 We need all these modules. Other privacy policy. 00:24:13.000 --> 00:24:25.000 So, given the area which we are currently infested, the first thing is women in the room, like we have seen that it has detected some issues, so we need to find out. So this is the first step. 00:24:25.000 --> 00:24:37.000 One of the things that to make sure the business and then you need to take a look at making the proper exam. 00:24:37.000 --> 00:24:43.000 We do not let the AI go wrong, and we have to make sure. 00:24:43.000 --> 00:24:49.000 There are some tech involved. As you can still need to become a person before. 00:24:49.000 --> 00:25:02.000 And to make AI cheap and fair, we are using a… Python library and testing the environment of AI and models. 00:25:02.000 --> 00:25:14.000 It is very powerful, we tell your child to see the balance between the unfairness. We can also generate reports and pick the best ways to stop any bias we find. 00:25:14.000 --> 00:25:22.000 This ensures our system is stay neutral and treat every project with the same level of skills. 00:25:22.000 --> 00:25:25.000 And then you have the threadback by the government. 00:25:25.000 --> 00:25:30.000 And if I got the guidelines and what about what and what not. 00:25:30.000 --> 00:25:36.000 You have to go through the lane and to understand how. 00:25:36.000 --> 00:25:43.000 We found that everyone will understand compromise. And we have to download and use DLV. 00:25:43.000 --> 00:25:55.000 We have the questionnaire. Where do we have the about 100 to 200 questions. We need to go through them with all the decoding of the project. 00:25:55.000 --> 00:26:19.000 making sure all activities are met. And this, I mean, one of the reports that we generated from different lab. So the first one is called the past success rate. So one of the projects for the. 00:26:19.000 --> 00:26:23.000 Because we have about 93% that we have team, or 2%. 00:26:23.000 --> 00:26:43.000 The other one is we wanted to step with a… With that being completed or not, and whether there have been issues or not. So, this interactivity idea of what's happening, which is like a module. 00:26:43.000 --> 00:26:53.000 And this is about the hallucination risk. the disruption check order things like the power supply and the error in validation. 00:26:53.000 --> 00:27:04.000 And then we did the total, what is happening and then finally, he gets some sort of running the dictated one. So that means you have, there is some problem. 00:27:04.000 --> 00:27:16.000 With the structure of the drug. So we need to check what happens and emerging from the detection. So that is what we are taking on. 00:27:16.000 --> 00:27:30.000 And you do have the data governance report due to update project. 00:27:30.000 --> 00:27:34.000 So finally, the conclusion. This is the roadmap. 00:27:34.000 --> 00:27:39.000 What AI application, or in the pipeline, in the project office. 00:27:39.000 --> 00:27:44.000 When they left German, you have something called a categorization. 00:27:44.000 --> 00:27:55.000 where we start with the financial data. provided by universities. These universities have Dr. Old Forma or template. 00:27:55.000 --> 00:28:09.000 So we need to tackle items like the effect of money or travel. So this will help detail project management, project managers, how to start manual work. 00:28:09.000 --> 00:28:16.000 In the near future, we plan to… we aim to build the lessons learned. 00:28:16.000 --> 00:28:37.000 building up. Um, so this will help us… Not only what a two-step project, but also to understand what, if anything had been missed. So, we can use it to study. 00:28:37.000 --> 00:28:50.000 Lastly, in the long term. We have the potential for question and the opportunities are the generation, and this is what we are trying to build up. 00:28:50.000 --> 00:29:00.000 So, this ends my today. Yeah. 00:29:00.000 --> 00:29:08.000 Yes, please. It's. I have some questions. 00:29:08.000 --> 00:29:24.000 So your question about should be talking about different models and with the altitude more tokens and made comments about making a decision on the quality of that. 00:29:24.000 --> 00:29:39.000 How do you determine that more is better as far as the topics are concerned? Like you mentioned rattling, how did… how are you able to check that it wasn't just… Excuse me. 00:29:39.000 --> 00:29:48.000 Andy Morden. So we see the harmony, um… The output that is driven out. 00:29:48.000 --> 00:29:56.000 So then we look at the output of every what we want to to the region. 00:29:56.000 --> 00:30:00.000 So based on that, we checked with the other model. 00:30:00.000 --> 00:30:10.000 And then finally. later on, like, about the tokenization. 00:30:10.000 --> 00:30:17.000 to think about how we can use it in the future. 00:30:17.000 --> 00:30:30.000 They've got these different models, and… See in some cases, they behave in a biased way at least to check. 00:30:30.000 --> 00:30:42.000 Yeah, to be testing these models further. So I see you have this fairness check software. Did you use that only to be quite free, or were you able to look at the others? Because I think it'd be interesting. 00:30:42.000 --> 00:31:08.000 But again, you didn't see any… Did they all perform similarly, or was ready out loud? I think it depends. If you look only from the… But in the Finland, you have different family type of function where you can check like hallucinations within the model, only the same model. 00:31:08.000 --> 00:31:23.000 So, there are various types that we are looking at this, so we can see, like, um, the type of token that is being used, or if there is any hallucination, or if there's any other things, like not the. 00:31:23.000 --> 00:31:36.000 For example… We will collect a name and we have the urge to have the AI is thinking about it. So that then we can do all that kind of stuff. 00:31:36.000 --> 00:31:52.000 and you didn't see DC insignificant differences between the models, or… But I'm not similar. So when you talk about the, it's not much difference unless… 00:31:52.000 --> 00:31:58.000 In the second chest, we start a little bit of, uh. 00:31:58.000 --> 00:32:12.000 different. Like, actually, gamma model is having a good quality compared to being the type of data set that is being trained on. But when you compare with the 12 mod, um. 00:32:12.000 --> 00:32:22.000 If they doubled, like, slightly better. But we actually talk to you more about it later. I mean, we have different type of, uh. 00:32:22.000 --> 00:32:37.000 We're gonna have to make a pair that you have another 10, like, uh, AI4. So we can do some testing on stage as well. 00:32:37.000 --> 00:32:54.000 I… I didn't quite understand how the validation works, where you're… from the point of view of the… um, human, in the loop, in that, um, is the, um, are the… do you have then people? 00:32:54.000 --> 00:33:01.000 checking, or experts checking, um, that the output of the model. 00:33:01.000 --> 00:33:17.000 For a subset, or, I mean, how do they… because they… if the people can do everything, then you don't need the AI. So how do you… 00:33:17.000 --> 00:33:32.000 So, we have the… So for every call that the AIM or the BI will read that explanation journey. So that is a woman in the law. 00:33:32.000 --> 00:33:42.000 So, for example, I mean, they don't go through all of the issues. So our ITI, we want to look at the top 10 issues or top three issues. 00:33:42.000 --> 00:33:47.000 So, and that we filter them out, and then we will look at the explanation generally. 00:33:47.000 --> 00:34:01.000 So, this is where the movement will look happening. So the PI will check that summary. Yeah. If it's marked something as not critical. 00:34:01.000 --> 00:34:18.000 Yes. How do you check? Do you have some fraction of those also checks that they are, um, that they really are low priority? I mean, we had that incident happen because one of the actual issues is actually important, but. 00:34:18.000 --> 00:34:35.000 If I go into very low. So that is very confident and help us to find out. I mean, there's some… there's some issue with the structure. So there are things that's happening. One is the problem with the changing format of the front. 00:34:35.000 --> 00:34:39.000 And the second one, if you remember the product module, we need to define them. 00:34:39.000 --> 00:34:56.000 You need to highlight what are the problems that we are facing, and we need to define clear. So then, for example, we have a setup example to highlight the bucket. For this issue, it needs to be the highest score. 00:34:56.000 --> 00:35:16.000 So it's a PI or a PM. Okay, this is… the calculation is wrong. Then we go back to the prompt, and we add a jamble back to the… Um, but the problem. So then again we run a retexture model again. 00:35:16.000 --> 00:35:28.000 Is that how it works. Yeah, what fraction… of your of all the things are checked by human. 00:35:28.000 --> 00:35:40.000 At the moment, uh… And so far, the field project managers are doing is one of them because of their own checks. 00:35:40.000 --> 00:35:47.000 Yeah, I mean… Because since we are implementing this for the first time. 00:35:47.000 --> 00:35:56.000 So, they go through all of them. But when the next time, when the move is to get our data. 00:35:56.000 --> 00:36:03.000 It doesn't run the whole thing once again, but it took a mean issue, and it's due to reports on that particular new issue. 00:36:03.000 --> 00:36:10.000 So, for example, the students have added up, like, maybe two or three times in a week. 00:36:10.000 --> 00:36:20.000 So sometimes it can be a lower issue at the highest kick running whenever the utilities are up and reduce the prompt in between. 00:36:20.000 --> 00:36:25.000 to, um, at least we set up the same type of. 00:36:25.000 --> 00:36:41.000 Every Friday to run the issues, and give a very short report saying that, okay, for this issue, what has happened this way, these are the problem, uh, these are discovered. And then we compare it with the entire data chain. 00:36:41.000 --> 00:36:47.000 Thank you. Yes, sure. Oh, yeah. Sorry. 00:36:47.000 --> 00:36:51.000 Do you get… do you get re… okay, I guess it's too popular. Do you get reproducible? 00:36:51.000 --> 00:36:58.000 Results if you run this… Twice. 00:36:58.000 --> 00:37:12.000 I mean, do you mean that, uh, whether being… So you run this loop, and then you get your text summarization, and your explanation summary. You do that the same twice on… the same input, do you get the same… 00:37:12.000 --> 00:37:21.000 I mean, it runs like, uh… first, it's run on the experiential thermal. 00:37:21.000 --> 00:37:26.000 I thought you knew my isolation with one deer explanation summary. 00:37:26.000 --> 00:37:36.000 We'll come back to the test summarization. You have some set of things you take, do you get the same result if you do it twice? 00:37:36.000 --> 00:37:52.000 Oh, okay, got it. I mean, when you run it, yes, yeah. And because that is where we define the program. We make sure that we… that we highlighted that mixture that doesn't change every time. 00:37:52.000 --> 00:37:56.000 So you did that test before. When we get a generalized prompts. 00:37:56.000 --> 00:38:12.000 And the general change in maybe 2 or 3 times. But then we defined, we made a series of checked up prompts. And then when we ran for the second or third time, it remained consistent. Unless when the project managers. 00:38:12.000 --> 00:38:30.000 Because the issue doesn't maintain for the entire project. So it keeps changing. But maybe after two or three weeks, they update the issue. So again, it launches, and then it also highlight, I mean, we wrote the prompt in such a way that it also highlights what is the previous code. 00:38:30.000 --> 00:38:35.000 And this is the current door. So that is one of the steps. 00:38:35.000 --> 00:38:50.000 Thank you for that idea. One of the checks that we are going to put. What was the previous, and if there is any changes that have happened. The kind of follow-up question, so… If you run this on an experiment like G. So gene is a publicly. 00:38:50.000 --> 00:39:05.000 There is public information out there right? So there are articles of new scientists saying June is a deeply troubled project for, you know, 3 or 4 years ago. So in that sentiment analysis goes into the Llan, which then thinks. 00:39:05.000 --> 00:39:20.000 things differently about that compared to, say, Hyper-K or, you know, Atlas or CMS, depending on publicly available information that it has. So if you try changing the experiment name? So you could change it to a generic one that doesn't exist. You could start calling it good experiment or bad experiment. 00:39:20.000 --> 00:39:34.000 It actually makes a difference to your output. Good. Um… To try to avoid that. This is one of the fair Python model. 00:39:34.000 --> 00:39:43.000 It has one feature that… We can find out whether the AI is biased or not. 00:39:43.000 --> 00:39:53.000 One of the things that I want to touch in on. But the second one, again, back to the problem that we need to make sure, again, we define that. 00:39:53.000 --> 00:39:58.000 particular, uh, prompting that, okay, we need to follow this. 00:39:58.000 --> 00:40:06.000 Do not, you know, do not provide any other… idea, apart from this only. 00:40:06.000 --> 00:40:12.000 There's a difference between purchasing the property knowing that the AI is, uh. 00:40:12.000 --> 00:40:26.000 is actually doing. I mean, I always… You can send the prompts, be fair, but I don't know if you can verify that it's being fair without trying to trick it somewhere. 00:40:26.000 --> 00:40:39.000 So, in that case, um, we… when we look at the result from the, you know, the PI or the project manager. That's where we look at the customerization. 00:40:39.000 --> 00:40:44.000 Um, so we try and to retain any change in the behavior or something like that. 00:40:44.000 --> 00:40:48.000 So that is one of the text that we look. 00:40:48.000 --> 00:40:56.000 Yeah, alright. Yes. That was sort of overlap. 00:40:56.000 --> 00:41:14.000 Sorry, this is a very specific question. If you're running only PPD AI machine that T has set up, I understand. 00:41:14.000 --> 00:41:19.000 What sort of times do you run this? Because I'm also using this one. 00:41:19.000 --> 00:41:39.000 So it's different. Sandeep's running these as condo jobs, right? Yeah. So they get submitted to the same type of machines. But because this machine is taken out of the queue, they're not competing. They're competing with other people submitting GPU jobs. 00:41:39.000 --> 00:41:51.000 Okay, maybe I should do this. I'm just wondering whether we'd be competing. 00:41:51.000 --> 00:42:07.000 So, uh, that's quite interesting that these are run as they can run on many machines, can they? I mean, we haven't only set up for 5 GPUs, so what I have done is, uh, in the bundle and did job. 00:42:07.000 --> 00:42:15.000 Like every Friday about six o'clock in the evening at the end of the day. 00:42:15.000 --> 00:42:23.000 So and I've tried to see if there is any other time. 00:42:23.000 --> 00:42:39.000 I try not to interrupt or distract related to lab experiment or anything. It's a 12-minute job. I don't think it matters. Yeah, it's a 12-minute job once a week. Not a week. 00:42:39.000 --> 00:42:48.000 Okay. But I think because of the condo job, it runs based on the priority. 00:42:48.000 --> 00:42:58.000 Maybe. Sorry, they're just wondering whether that seems to be quite. 00:42:58.000 --> 00:43:18.000 If you're always running it on a Friday evening, is that the best time to get… um, results, which people then don't have time to look at them until Monday. Normally, then… But the last week of Monday. So we have the system set up. 00:43:18.000 --> 00:43:27.000 Like in this chat, you will be updated on Mondays. So, we collect all the data. 00:43:27.000 --> 00:43:45.000 But it's happened until Friday. So, at the moment, nobody's looking on Friday, but the reports that we generate, it goes to the email, and they're looking at whenever… Um, and they have gone. 00:43:45.000 --> 00:43:50.000 Um, yeah, so going back to the issues severity analysis stuff. 00:43:50.000 --> 00:44:09.000 So the main purpose of that is to generate this summary document, is it, that would then be read by PIs and PMs, or is that the… What do you call it? Where is that for the task managers or work stream managers as well? 00:44:09.000 --> 00:44:18.000 It's different. I mean, normally just… we call each other project stakeholders, so if you are involved in that. 00:44:18.000 --> 00:44:45.000 are particular about package or something like that. So normally, this project managers will work along with the PIs or practice managers, and they coordinate with each other. So… What about the project manager? Is that information will be shared, uh, corresponding? Yeah, so, because in that case, yeah, so I work on one of the workstream managers for the CMS upgrade side for the hardware. 00:44:45.000 --> 00:44:57.000 And so there we often we have chats with Lay every every month or two to update on the issues. 00:44:57.000 --> 00:45:11.000 Um, and then we also update the risk register as a result of that, but then… What I was thinking was that at that stage then, we're effectively ranking the severity of each of the issues in the risk register itself, right? 00:45:11.000 --> 00:45:26.000 Um… So then, yeah, we've kind of fed in all of the information. We know all of the information. So the main purpose is then for the DI to be able to read this as a summary, or. 00:45:26.000 --> 00:45:30.000 Yeah, I'm even trying to think how it fits into the… into the workflow. 00:45:30.000 --> 00:45:39.000 Yeah, I mean, that is what we plan at the moment, particularly the treatment because. 00:45:39.000 --> 00:45:54.000 the project managers are focused more on the project and the strategy. So they are handling it by the time. At some point, we will work using AI on the risk register as well. But we have it in our primary software. 00:45:54.000 --> 00:46:04.000 We have our own Monte Carlo analyze it and where did they fit into the information, and they calculate, and they come back to it. 00:46:04.000 --> 00:46:15.000 So because sometimes the issues are long or it changes all the time, very frequently. So there needs some shut up. 00:46:15.000 --> 00:46:45.000 happening and then they decide what happened with something. So normally, I think some of these tools, we update the risk register or accurate so it's like… Okay, but then the future change that you mentioned around the AI and the risk register, would that be the one of these AI setups taking the risk register, the project. 00:46:45.000 --> 00:47:02.000 We're extremely… the workstream managers have… Uh, updated with the project manager, and using that as input, and then using that to do some kind of analysis of likely future costs. 00:47:02.000 --> 00:47:26.000 Yes, um, when we talk about the future, I mean… I was expecting my… Yeah, because… We take very tentative management. So we need to understand a bit more. Okay. Um, so once… once we get comfortable properly, and then we start working on the concept of AI model. 00:47:26.000 --> 00:47:39.000 Okay, okay. I can agree to start to come to know about it. This is one of the pipelines that is happening in the future or not. So at the moment, we are looking at the gap. 00:47:39.000 --> 00:47:48.000 That's a good idea. You need to analyze the data, the financial data. 00:47:48.000 --> 00:47:55.000 So hopefully, maybe after this, we'll be looking at the risk register. 00:47:55.000 --> 00:48:11.000 Okay, and that would be to to rank items based on the risk register. We do have to already we have to talk about software that will automatically calculate. But. 00:48:11.000 --> 00:48:19.000 At the moment, uh, the project managers, they're feeding the numbers. Yeah. So… Okay, when when we collect that information. 00:48:19.000 --> 00:48:34.000 At some point, we will identify the prompt and then make it work in other way. Okay. But you need to be a little bit careful telling that is about the risk. And because. 00:48:34.000 --> 00:48:50.000 Yeah. So we need to find out like how they work together. I mean, to China, but it starts becoming the big pipeline. So we need to. 00:48:50.000 --> 00:49:05.000 We need to look on that. Okay. Well, yeah, I mean, in a sense, I would thought the… the risk register is easier to analyze because it's effectively just a few why I remember is that it's a set of numbers of the probabilities in the cost ranges. 00:49:05.000 --> 00:49:12.000 Uh, associated with risks. And so that's already quantitative, whereas. 00:49:12.000 --> 00:49:26.000 So it could just be ranked based on that without an AI. Whereas the issue is the text, so… Maybe lend themselves more to a textile. Yeah, and one of the factors is the Monte Carlo. 00:49:26.000 --> 00:49:29.000 Uh, which we haven't started working on that yet. 00:49:29.000 --> 00:49:37.000 So we need to understand a bit about the Monte Carlo, and then the AI path. 00:49:37.000 --> 00:49:54.000 Because… You need to run the model, uh, for the Monte Carlo iteration, like, and so this is where you don't really understand that Monte Carlo analysis from the AI point of view. 00:49:54.000 --> 00:50:02.000 but would would you consider replacing that Monte Carlo analysis with an AI one instead? 00:50:02.000 --> 00:50:09.000 No, I mean, if you are talking about the normal isolation, that can be done. 00:50:09.000 --> 00:50:28.000 But we've gone to Montecoblo as well because. Because we want to, uh, let's say, unlike a delayed path, and detection, or the source, and what permit probably they are 50% or 90. 00:50:28.000 --> 00:50:47.000 The running embedded pipe legend is working HGA, but when you're… to application of Monte Carlo is where it requires a little bit of detailed understanding. So which is why we are relying on our primary software to do that sort of analysis for us at the moment. 00:50:47.000 --> 00:50:53.000 So, at the moment, that is why we are manually fitting into a training now. 00:50:53.000 --> 00:50:59.000 Okay. We have a raised hand on Zoom. Chris or Christopher would like to speak. 00:50:59.000 --> 00:51:16.000 Yeah, so I think… Yeah, it was just about the question before about potential bias on the sentiment analysis. Say, say if a project has got into the news. I think my understanding is that this is a completely closed system, so wouldn't actually have access to the new. So this wouldn't creep in. 00:51:16.000 --> 00:51:21.000 Is that correct, Sandeep? 00:51:21.000 --> 00:51:39.000 Um, so… So the the question was, depending on when the model was trained, that knowledge is part of the model, because it might have been based on text that is out in the open. 00:51:39.000 --> 00:51:57.000 So, once it has no access to external systems and can't query the web, it might have been trained in that way beforehand, and it's a bias that is inherent to the model, but then needs to be the you need to figure out whether it's in there or not. 00:51:57.000 --> 00:52:07.000 Sorry, paraphrasing. Yes, Chris is very happy. 00:52:07.000 --> 00:52:22.000 Okay. Oh, we see here. Okay, so. Are there other questions? 00:52:22.000 --> 00:52:38.000 I have one last question, maybe. I was curious how you measure the… how you quantify the input data. And I yeah, that's we lost this line, but… I think on the 1st slide or second one, you had the. 00:52:38.000 --> 00:52:46.000 the number of words, so could you explain a bit what the meaning of that that is? 00:52:46.000 --> 00:52:56.000 It's looked like a little number of words, but maybe it has another meaning, which… Yeah. 00:52:56.000 --> 00:53:04.000 So what is this amount of data that you're using? 00:53:04.000 --> 00:53:11.000 Well, that's available at the moment. Yeah, at the moment, we have. 00:53:11.000 --> 00:53:16.000 When we work on the project, we entered all that information. 00:53:16.000 --> 00:53:33.000 For example, there is true with an history or anything. So it attracts all of that information and it counts all of it. And we trust it for the. 00:53:33.000 --> 00:53:41.000 I'm doing something to… So that is the input data from the resource log. 00:53:41.000 --> 00:53:56.000 So you only use the issues now? At the moment, because we are focusing on the issue of management. So this is going to a specific model of the project management. So this is the data that we've got. It's on there. 00:53:56.000 --> 00:54:09.000 This is only for the open distribution, so we have no issues. It is more than 1,000 or something like this is already open to about 200 pictures. I've got the entire project we are managing. 00:54:09.000 --> 00:54:16.000 Oh, I see. Okay. 00:54:16.000 --> 00:54:26.000 Okay. Thank you very much. So let's thank Sandeep again and we close. 00:54:26.000 --> 00:54:32.000 Yeah, we explain. 00:54:32.000 --> 00:54:44.000 Lots