Understanding AI Agents for Research Administration

Webinar start and attendance (0:06)

Robert Morris: Right. Hello, everyone. We’re just going to wait another minute here as we wait for a few more attendees to join and then we’ll get started real soon. All right, well, hello, everyone, and thank you for being here. I’m Robert Morris, and I’m part of the team at Streamlyne, and our webinar topic for today is understanding AI agents for research administration. It looks like we’ve got a massive turnout, so we’ve got nearly 500 registrants and 200 people already in here, probably a little bit more trickling in as we get started. But what this tells me is that this topic is clearly something that is important and a lot of you are actively thinking about. By now it’s obvious that AI is starting to have a huge impact on research administration. So with that in mind, our goal today is simple and practical. We’re going to take a look at the top AI tools to see how they can help you out as a research administrator, discover what sets them apart, and dive into how human review is still essential. By the end of this, you’ll have a clear picture of how research administrators are putting AI to work and which tools are right for the job. A quick note, today’s session is being recorded and will be shared with you afterwards. Leading us through everything today are two members of the Streamlyne team. We got Irma Roman, who is our Implementation and Support Manager, as well as Janior Valle, our Senior VP of Tech and Product. So let me hand it over to Janior to get us started.

Janior Valle: Thank you, Robert. All right. So just to kick us off and get going here. Streamlyne, if you haven’t heard of us, we’re an electronic research administration provider. We’ve been building these different software workflows and institutional systems for 20 plus years. I myself am the Senior VP and Chief Technology Officer. My background is as a developer, so I have spent a lot of time building these products, the architecture, the technology behind them, software and integrations, and now a lot of the AI-enabled workflows around them. Irma, you want to go ahead and just briefly introduce yourself?

Irma Roman: Yes. Hello, everyone. Thank you for joining. My name is Irma Roman. I’m the Implementations and Business Analyst Manager at Streamlyne Research. Prior to joining Streamlyne, I was a research administrator in various roles for 15 years, from sponsor project specialist to AOR for the University of Puerto Rico Medical Sciences Campus. Back to you, Jay.

Session background and goals (4:17)

Janior Valle: Thank you very much. Before we get going, I just wanted to give you a little bit of context as to where this session is coming from and what we’re going to be doing today. This is not the first session that we’ve hosted. We’ve actually been doing this for a little bit over a year, year and a half now. We’ve been doing a lot of AI-focused workshops where we’re talking to the research community, trying to help provide a benefit in terms of how this technology works. It’s constantly shifting in terms of new models and new agents that are coming out all the time. We really just wanted to provide a benefit to the community to give them something practical that they can take with them. Back at the end of the session, no matter what tool you’re using, whether it’s ChatGPT, Copilot, or any other AI model, what you’re going to learn today is usable for any tool, so it’s a practical session to help in your day-to-day. A quick note: our Q&A may not be working. This is a new platform that we’re using, so I apologize for that. If there are any questions and we do get that working here during the session, it is a rather large group. We’ve got an overwhelming amount of registrants for this session. If the Q&A does happen to come on later, we’ll try our best to answer any questions that come up. If anything, we will disseminate the responses to any additional questions we were not able to answer after the fact. At the end of the session the webinar platform will automatically send you a recording of the session today so you’ll have that available to refer back to as well. So I guess let’s start off with why AI? Why now? Why are we covering this? How does it help? What’s the point of AI and how can it help me in my day-to-day? We’re going to be looking at a lot of different models: ChatGPT, Copilot, Gemini, Claude. We’re not necessarily talking about how it can replace you, but how it can augment your work. A lot of work that happens in research administration can be repetitive, and that’s where AI tends to shine — taking that load off of you so you can focus on being the expert. Normally, I would ask for a quick temperature check on what tools the ecosystem or the industry is using. Some of the things I’ve heard before in the past are ChatGPT, of course, Copilot, Claude, Gemini — a lot of tools that we’re going to cover today. They all work well; they have pros and cons. There are app-specific agents available. For example, Copilot does have the ability to integrate with the different apps within Microsoft. Other platforms like ChatGPT, Claude, even Gemini, offer connectors that allow you to connect your Outlook emails or Gmail or anything like that to those agents. There are different agents that operate with different applications. A few things we’re going to cover today: some basics in terms of what context is, what are good prompting patterns, what does this tool landscape look like, and then once we cover some of the basics so we’re all aligned on terminology and agents in general, I’m going to turn that over to Irma who’s going to cover four live use cases using different models so you can see how they work and how to prompt for those use cases. If there’s time, we’ll cover Lyn Pro. It’s a model we’ve worked on internally. It’s not the point of the session, but if there’s time, because we have a lot to cover, we’ll spend a few minutes on that. So let’s start off with some basic knowledge transfer. A common question we get very often is: what is an AI agent and how does that differ from an assistant or a chatbot? Let’s start there.

Agents versus chatbots (9:22)

Janior Valle: To keep this plain, your assistant is typically something that is more of a chatbot. Think of it like I make a statement, a comment, or ask a question, and that bot comes back and responds. It’s very one-to-one. An agent differs because it typically has a lot of tools under the hood that allow it to complete multiple steps in one turn. Think of it like if I ask for an FOA to be summarized. That agent is going to, number one, consider the instructions it’s been given by its creator — OpenAI, Anthropic, Google, depending on which model you’re using — and it has its context.

Janior Valle: So any documentation that you provide, so that FOA that you’re providing, that’s the context. It has tools. For example, it may have a web search tool so that it can find additional information. It may have a tool to read the file or create files. A tool is just some sort of action or thing that the agent can use to enhance the context that it’s already been given. And then finally, a goal. The goal in this case is summarizing an FOA. An agent can perform multiple steps to complete some sort of action or some sort of goal; it’s moving the work forward. That’s really what an agent is. A common question that we also get is context.

Janior Valle: So what is the context? What are tokens, and what is a context window? To kind of without diving too technical into the subject, the context window is typically the total number of tokens that an agent is able to ultimately remain in memory. When you’re working with a lot of these agents there’s a window in terms of how much memory it can retain before it has to summarize everything internally and start over or start fresh. Different models have different windows. If I have to equate it to something, you can think of it like your hard drive. Every hard drive has a certain amount of space before you’ve exceeded the space and you need another hard drive. A context window is based on tokens. And it’s not the ones that you put in the arcade. Tokens are just a measurement for words or characters. A good rule of thumb is one token is about three to four characters. It’s not precise, but that’s a good rule of thumb. In terms of the context windows, this is becoming less and less important over time. It used to be that these agents would only have like 200,000 tokens as a context window, meaning that if you give it two or three PDFs, if they’re really large PDFs, for example, the agent would not be able to process those PDFs. It would kind of stall within the chat. You may see some of that today when we go over the live use cases. Over time, this is becoming less and less important because the window is increasing in size. It used to be about 200,000 tokens. Now many models are moving towards 1 million tokens. It’s not that the window is infinite, but it’s growing, and it allows you to read more PDFs, process more information. Maybe now you can also process an Excel file. It opens up more possibilities in terms of what you’re able to do with these agents. In general, it’s just something to know that these agents do have a limit in terms of the amount of information that they can store before they essentially reset. What happens if you hit that window? The agents cannot retain all of the information within that context window. It’s almost like if you hit your limit on your hard drive you can’t save anything else. What does the agent do in those cases? The agent will summarize what it’s learned during your session to keep the chat moving. That’s good so that you don’t completely stall out in terms of being able to talk more with the agent. But it’s also something to be cognizant of because if you talk to the agent over time and you hit that limit, that one million limit, and it summarizes, the information that’s summarized is not going to necessarily reflect everything that you discussed with it during that session. Maybe in that FOA that you summarized with the agent you gave it some feedback on a certain verbiage that you want output or some other requirement. Maybe the agent drops that requirement during the summarization. So it’s something to be aware of. You’ll kind of see it in the chats when it happens; it’ll tell you “compaction.” That’s another word that they use for that summarization.

Hallucinations and causes (15:34)

Janior Valle: Hallucinations — you may have heard this term. Really what this means is the AI is making up information. This can happen for a lot of different reasons. This happens a lot less nowadays. If you were to ask me maybe a year ago how often this hallucination happened, I would say it happened very frequently, and you have to be very careful in terms of what the AI outputs and making sure that you review that output because it could be falsifying or providing incorrect information. Nowadays, it is a lot better. It still happens. As with anything AI generated, you definitely want to review the output that is provided. Usually, when it comes to hallucinations, what it usually means nowadays is that you’re not giving the agent enough context. We talked about the context window; let’s talk about context. What is context? Context is relevant source information. If I’m asking for an FOA to be summarized, I need to provide that FOA. That is a piece of the context. If I want that summary to be done in a certain way — maybe I want bullets instead of paragraphs or precise and succinct bullets versus more verbose bullets — whatever those requirements are, all of those requirements are pieces of context. If you provide weak context, meaning that maybe you’re not providing what you expect back from the model, maybe you don’t give it that FOA but you’re asking for the FOA to be summarized, you’re not providing enough context for the model and that will lead to more hallucinations. So that’s something to be aware of. That was just some of the basics in terms of the AI terminology that we’re going to be using today and all the different things we’re going to be covering.

Janior Valle: Let’s talk about the landscape and then after this we’ll go into prompting and turn it over to Irma. There are different types of AI. Most of us at this point are used to using the chat-based applications, the text-based models — your ChatGPT, Copilot, those models. They’re not all equal. You have text, you have image and vision, and you have multimodal. Text is the one everybody’s familiar with: that’s your ChatGPT, your email autocompletes, your customer service chatbots. Image and vision are models that can create images. For example, ChatGPT very recently created an image generator. Now you can do image generations with AI and edit images. Vision is where the model can “see” that image. Multimodal is a model that works across different modalities: it can produce text, read text, produce images, see images, and in some cases even sound. There are models now used for customer service that can produce voice, so audio. A lot of different types of AI there. I’ve already mentioned some of the models we’ll cover today. If there’s time, we’ll cover Lyn Pro, but really we’re going to be covering these four models that are heavily used today.

Image generation example (19:36)

Janior Valle: Here’s an image just as a little bit of context. This is an original photo. This is my pup, Abby. She’s a rescue. She’s going to be 10 years old this year in a few months. Last year, ChatGPT came out with this image generator for studios. I’m sure you all remember this timing last year; this was mid 2025 or so, and then late 2025 they increased the capabilities of that model and it can now produce sharper images in terms of painterly style or different styles. Now you can use ChatGPT or even other models to edit the original image, which wasn’t possible before. And so now we have Abby with a pretty hat and Abby as a queen.

Janior Valle: My kids call her Queen Abby. The reason I’m showing you this is that this is one of the reasons we’re doing this webinar: the technology moves very quickly. Even this year or last year around this time, this is what ChatGPT was able to produce in the middle. Now it’s far superior to anything reproducible last year. So it’s very important to keep up on the technology and make sure that you’re understanding the new features, the new tools that are coming out and just being aware of what’s out in the ecosystem. We’ll cover budget justification here in a little bit, that 50 page FOA summary, some sponsor inquiry responses, and a subaward red flag for review. Before I turn it over to Irma, let’s go ahead and cover some prompt engineering 101 real quick. You really need to think of prompting as a delegation: treat the agent as a junior employee, first day on the job. They have no context into your policies or your procedures.

Janior Valle: They’re starting from a blank slate. One good takeaway in terms of prompting is to try to give your prompt or your agent a role. This helps provide the model context around what the expectation is. If you’re a pre-award reviewer or a contracts analyst, that sets up the stage for the model to understand what it is you’re going to be asking it, what task it’s going to be doing, and what the expectation is. A contracts analyst is going to be focused on contracts, obviously, but what if you need a budget reviewed or a budget template created? That’s not necessarily going to be tied to a contracts analyst. You very much want to set up the role for what it is you’re going to be asking the agent to do. Then you want to provide a context.

Janior Valle: Give it as much information as possible. I know there’s a big concern with data privacy in the ecosystem when it comes to AI. Several solutions out there, for example Copilot, ChatGPT, Claude, Gemini, have the ability to turn on data privacy, meaning any information you provide to that model is not retained for training purposes. That does require an enterprise contract. If you’re using the free plan on ChatGPT or the free plan on Gemini, they will train on anything you put in that prompt. So that specifically has to be a negotiated enterprise agreement. If you’re ever questioning what you can put in the prompt versus not, double check with your legal and your IT resources.

Janior Valle: Keeping that in mind, you may be a little limited if you’re using something that’s on a free plan. You definitely don’t want to give it any private information; stick with publicly available information. If you do have data privacy enabled and it’s cleared by your IT and legal, then you’re good to go. Provide as much context as possible. In that FOA example, provide the document for the FOA or at least a link to where that FOA is. Provide the role that agent should take.

Janior Valle: Provide requirements on what you expect. I talked about bullets versus paragraphs, not being verbose, and just giving the facts. All of those requirements that you have of the agent, provide them upfront. We actually have a template that we’re going to share today so you can leave with that template to help formulate prompts, and you’ll see Irma use those in her use cases shortly. As with anything AI generated, double check its output.

Janior Valle: I refer to it as a trust but verify system. Trust the AI that it can do the work given the context you’ve set up, but verify its work because it could be hallucinating or lacking enough information. This is what I call the four C’s of effective prompting: clear, contextual, complete, and conversational. You’ll see this in the prompt template that Irma is going to cover shortly. Treat the agent as a junior employee, first day on the job; they’re starting from a clean slate and don’t know your policies and procedures.

Janior Valle: If you go into that conversation with that in mind, your outputs will drastically improve in quality. AI is not perfect. I use the technology every day during my work. I can’t recall a single time where an agent output something I accepted 100%. The best way to work with AI is to be conversational. You can’t expect a perfect prompt that gives you what you need on the first try. Like a junior employee, you’ll have to review the work and give feedback. Over time you’ll refine the prompt so that the iterative back and forth becomes less necessary.

Janior Valle: Here’s a sneak peek at that prompt template. Set up the role: tell it to act as an experienced research administrator reviewing a sponsor requirement or whatever the role is for your task. Then set up any context. For FOA summarization, provide the documentation and who the audience is.

Janior Valle: If you’re summarizing, clarify why and who will be reading that output. Think about institutional constraints. If you have policies and procedures you can feed the model as part of the context, that will help produce the output. Be clear about the task: if you’re asking it to summarize the FOA, tell it to summarize the FOA document and specify what you want summarized. Are you looking for a general summary or something specific in that FOA? Make sure you’re telling it exactly what the requirements are. Tell it whether the output should be a Word doc, a PDF, or another format.

Janior Valle: Provide the model a way to validate its work. Give it a check: when you summarize this FOA, do it in bullets for my research administration team. Tell the agent how to validate the work: read the output, check that it has bullets and not paragraphs, and that the audience is correct. That validation piece becomes critical because it’s like us as humans: we double check our work. I’ll turn it over to Irma after this slide. We already discussed this, but double check with your legal and your IT on what’s safe to put in prompts and what’s not. That will be tied to whether you have the data privacy piece enabled within your institutional ChatGPT or Copilot license or whatever platform you’re using.

Janior Valle: All right. I’m going to go ahead and turn it over to Irma.

Overview and test cases (30:41)

Irma Roman: Thank you, Jay. All right. I will be sharing my screen with you and showing some live cases, some we prepared prior to this webinar and, in the interest of time, others we will see what happens as we go. I think everybody can see my screen. I can see it on the webinar page. Let’s get started. Jay talked to you a little bit about the guidelines to prompt the AI agent, so I’m just going to dive right in and do the test cases that we have for you today. As you can see on my screen, we have Copilot. Copilot we will be using in the auto thinking mode.

Irma Roman: I’m going to ask Copilot to draft a budget justification narrative from an R&R budget form. Using the guidelines that Jay talked about earlier, I’m going to write my prompt, Copilot, and then I’m going to upload that R&R budget file and have it go and see what it comes up with. That was pretty quick; here’s the summary that it produced from the R&R budget per year.

Irma Roman: I want a Word document that I can download.

Irma Roman: It’s thinking. Let’s see what it comes up with. And here’s the justification document.

Irma Roman: The next case that I am going to use, I will use the same prompt, the same R&R budget PDF, but for ChatGPT. I asked ChatGPT to, as a research administrator at a university, write the budget justification to be included in the proposal to NIH using the R&R budget. It thought for 20 seconds, and this is the output for the budget justification—much more detail than the Copilot output. I can just download that response; it’s much more complete. For ChatGPT we are using the high thinking model.

Claude and Gemini summaries (34:44)

Irma Roman: Our next case, we will take a look at Gemini and Claude. I will start with Claude. In the interest of time, I uploaded a close to 50-page PDF for a Department of War funding opportunity. This is the prompt for Claude: I asked it to act as a pre-award research administrator at a university, summarize this funding opportunity, identify key requirements, and give me a compliance checklist document to share with a team of investigators and other research administrators. Claude had the funding opportunity announcement, then it started checking all the skills that it needed, validated the information, and then in the output I have the document.

Irma Roman: And there’s my checklist: any additional forms necessary, formatting and content rules, etc. This is a pretty comprehensive summary and checklist.

Irma Roman: Then we want to do the same summary in Gemini, and this one we are going to do live. We are going to use the same exact prompt and the funding opportunity.

Irma Roman: So let’s see.

Janior Valle: I was just going to mention a few things. A common question I get is what thinking mode should you use? A lot of these models have different thinking modes. You’ll notice Irma mentioned earlier she’s using the high thinking mode. It was on high for ChatGPT and it was on high for Claude. For Gemini we have it on standard. One thing to know about these thinking modes is it’s very much trial and error; there isn’t one fast rule for what thinking mode to use. If the task is relatively simple, I would caution against using a high thinking mode. If you use a high thinking mode for an easy task, like summarization, sometimes the model or the agent can overthink or overprocess the information and it actually degrades the response that you get back. Try playing around with your standard thinking modes and your high thinking modes. In this case, Gemini has a flash model versus a heavier model, like the pro model. Flash is lighter; the pro model is a heavier thinking model. It’s very much just trial and error and figuring out what works best for your workflows. Once you realize that a model produces a good response, you can align that workflow to that model. The things Gemini does best are not necessarily the things ChatGPT does well.

Janior Valle: I just wanted to mention that while we’re waiting. Sorry, go ahead.

Irma Roman: Thank you, Jay. I think I’ve been showing while you were talking the output from Gemini. Obviously, all these outputs require human intervention as far as checking: is this what we want? Can we rely on the AI agent to give us accurate information? That comes with experience or just by reviewing your work the first couple of times. As the agent learns your preferences and the type of work that you do, it becomes an easier task. I think it’s a little unfair for me since I tried different models for research administration–related tasks. With that, I will go into the next test case.

Sponsor inquiry draft (40:28)

Irma Roman: We want to use ChatGPT and Copilot to create a first draft response to a sponsor inquiry. I’m going to create a new chat on ChatGPT. I will copy my prompt under the guidelines we previously discussed and then I will show you. This is the sponsor inquiry. The sponsor inquiry is related to a grant application and to people that might appear on the SAM.gov list of individuals that are excluded from participation in federal awards. I put that into a Word document and…

Irma Roman: Let’s see what ChatGPT comes up with. As we see the model and the agent thinking, we can see that it’s analyzing correctly. It wants to craft a polished AOR response, and then it gives me a partial result about what the inquiry is about. The agent discusses formatting items and how the document should be addressed. The inquiry contains a PNG image of what appeared in SAM.gov regarding the two individuals in question. It’s taking a little bit to open the document, and there’s a document with the date of the response, who it is addressed to, who it is from, the subject, and what the official response to be reviewed is. Now we will ask Copilot to do the same exercise with the very same prompt. As we see output by the different agents, even though we try to equalize the thinking model that we utilize to do the analysis, we can clearly see differences between the two cases that we’re comparing. Now for our third use case. We will go ahead and use Gemini and ChatGPT to analyze a subaward or subcontract terms for red flags before negotiation.

Subaward risk analysis with AI (46:48)

Irma Roman: I will go to ChatGPT and input our prompt. We’re going to be asking ChatGPT to act as a contract analyst in a university and to prepare—I’m sorry, I apologize, I copied the incorrect prompt. Here we go. We’re going to ask ChatGPT to act as our grants and contract panelist for a research institution and analyze an attached subcontract for any risky terms and conditions. Let’s see if it provides us a redline of the document as well. This subcontract is related to a research project, a past research project between a university and a research institute. The document contains 36 pages with all the terms that I will share so you can take a look at the document, with all the terms and conditions on reporting, information, invoices, taxes, etc. As it continues, we can see the thinking and the analysis that is taking place while ChatGPT is thinking. In the interest of time, while ChatGPT thinks, I will open the new chat in Gemini and ask Gemini to do the same thing. So let’s go back to ChatGPT and see how it’s doing with our tasks. Let’s see how Gemini is doing.

AI review of contract risks (52:06)

Irma Roman: Still thinking. Same for ChatGPT. However, I’ve been seeing that ChatGPT has made some good points regarding risky terms and conditions. This is taking a long time, but it’s probably faster than getting our institution’s legal division to review it, and Gemini is still stuck on the task.

Janior Valle: So one of the things that we were talking about earlier is a lot of these models are agents, right? It’s not that they’re just a chat-based application, but you’re not necessarily going to get immediate responses because they’re doing multiple steps underneath the hood. In the ChatGPT case here you can see the comments are going back and forth; it’s thinking about everything, processing all the information, going through how it’s going to generate that output, and it’s doing all the processing. So some of these workflows can take several minutes. For example, as I mentioned, I use this technology every single day.

Janior Valle: It’s not unheard of for some of the workflows that I do to run for 15-20 minutes sometimes if it’s something very lengthy. Obviously your general summarization tasks and emails aren’t going to take 20 minutes, but things like this where you’re giving it a role and asking for something more complex, like here where we’re asking for a red line for RIC terms, require a bit more processing. In general, when it comes to typos in the prompt, I wouldn’t read too much into them. The way these systems work, they’re a very powerful autocomplete. Even though Irma wrote ‘risk terms,’ she meant ‘risk.’

Janior Valle: The model does pick that up. Go ahead, Irma.

Irma Roman: Thank you, Jay. Let’s wait to see where our output is from ChatGPT in the meantime. We’ll check on Gemini. Gemini is still stuck thinking even with the typo in the prompt.

Irma Roman: And here we have the output from ChatGPT to grants and contract risk review and red line. There’s an executive summary and recommended negotiation posture.

Janior Valle: Irma, I think you’re sharing only the web UI, so we don’t see the Word doc.

Irma Roman: Let me switch my screen.

Irma Roman: And I think now you’re seeing the ChatGPT output.

Janior Valle: Yeah.

Irma Roman: It included the original document as well, and then we have the red lines on the terms and the risk, even though I had the typo on the prompt.

Irma Roman: I will stop sharing and pass it over to Jay.

Janior Valle: Thank you, Irma. I’m going to go ahead and share my screen again.

Lyn Pro and next steps (58:50)

Janior Valle: So that concludes our session here. We unfortunately did not have much time to cover Lyn Pro. Lyn Pro is one of the models we’ve worked on internally that is more familiar with many of these research administration workflows. Think of it like ChatGPT, Gemini, Claude, Copilot, but in tune with more research administration knowledge. Instead of having to explain what an FOA is or how you want certain things redlined, or if you have institutional policies you want to feed into a model without dealing with data privacy issues some market models present, Lyn Pro is already familiar with many of those workflows.

Janior Valle: I want to thank everybody for the time today. I put a scannable QR code at the top right. You can scan this; it’ll be part of the recording as well. It gives a link to the prompt template I put together; it has a proper outline for you to take and tweak. It can be as small or as large as you need. It has different sections to walk through and help formulate your thoughts. I apologize the chat and the Q&A did not work out today.

Janior Valle: This is a new platform for us, so on the next webinar we’ll have that working. Not sure what happened today. If you have outstanding questions, email us at hello@streamlyne.com. We’ll triage any questions that come through. The recording will be distributed right after I end the call. In general we will do another webinar that will be a bit more advanced than today’s. We wanted to cover foundational topics and concepts to set the stage for future webinars. We’ll post the next webinar on the listserve when it’s ready; it’ll be more about creating your own reproducible workflows alongside agents. Stay tuned and we’ll share those in the research admin communities. All right, thank you so much for your time, everybody. Hope you have a great day and take care.

Irma Roman: Thank you.