AI Application Analysis

This video walks administrators through how to configure the AI Application Analysis in GLM or SLM, including setting up criteria, screening flags, and supporting files, and how to view and interpret the analysis results on submitted applications.

• Feature Overview (:43) - Explains that AI Application Analysis allows administrators to define criteria that automatically generate summaries, scores, and flags when an application is submitted.
• Enabling AI Analysis on Questions (1:01) - Covers how questions added to the application form are automatically included in the AI analysis, and how to deselect the Include in AI Analysis checkbox to exclude a question.
• Adding Analysis Criteria (1:20) - Walks through creating a new analysis list by clicking Choose List.
• Configuring the Analysis List (1:47) - Covers the configuration screen, including AI Summarization, Analysis Criteria, type and weight multipliers, editing and deleting criteria, and uploading supporting files to individual criteria.
• Screening Flags (3:41) - Explains how screening flags automatically flag applications meeting specific criteria for closer review, and walks through adding a flag with a name, description, and category.
• Supporting Files (4:11) - Covers attaching supporting files to the analysis list to provide AI with additional context such as grant guidelines or funder priorities.
• Important Note on List Editing (4:36) - Highlights that once an application has been submitted for the process, the analysis list can no longer be edited or changed.
• Viewing Analysis Results (4:53) - Walks through navigating to the Application Submitted workload page and clicking the View Submission icon to access the analysis results.
• Overall Score and Timestamp (5:14) - Explains that AI calculates an overall score out of 10 and displays a timestamp showing when the analysis was run.
• Application Summary and Screening Flags (5:29) - Covers the generated application summary and how screening flags display with reasoning and the area of the application where the flag condition was met.
• Per-Criterion Scoring (5:57) - Explains how each criterion displays the applicant's score alongside the type, weight, and explanation of the score, and a citation of where the information was found in the application.
• Application Actions (6:19) - Notes that the application can be denied, abandoned, or marked incomplete or complete directly from the analysis results.
• Adding Custom Columns (6:29) - Walks through adding Analysis Score and Analysis Flag Count as custom columns to the Application Submitted and Application Complete workload pages for quick review.

By default, questions added to forms after the Application Analysis setting has been configured are automatically set to be included in the AI Application Analysis.

To enable the AI Application Analysis feature on a question:

1. Click Standard to add a new question.

    

2. Click the Question Type.

    

3. Configure the question, then click Save Question.

    

On forms created before the Application Analysis setting was set to True, administrators can edit the question and check the Include in AI Analysis checkbox.

To add analysis criteria to an AI Application Analysis list, click Choose List then create or copy a list.

1. Click Choose List.

    

2. Click Create New List.

    

3. Enter a name, then click Create Configuration.

    

4. Click the pencil icon.

    

5. Configure the AI analysis criteria, then click Back to Update Process.

    

AI Summarization

AI Summarization generates a high-level summary of the application, emphasizing key strengths and considerations based on the applicant's responses. The toggle is on by default and can be turned off if a summary is not needed for the analysis.

Analysis Criteria

Analysis Criteria are the scoreable rules used by AI to evaluate each application. Each criterion can be configured with a type and weight multiplier, and displays a score and explanation in the application analysis results.

To add a new criterion to the list:

1. Click Add.

    

2. Configure the criteria, then click Save.

    

   • Type - The type of criteria can be set to either a scoring option (0-10) or yes/no.
   • Weight Multiplier - Each piece of criteria can be given its own weight. The weighting options range from x1-x5. The overall analysis score remains out of 10.

    

To edit analysis criteria, click the pencil icon.

To delete analysis criteria, click the trash can icon.

Administrators can add supporting files to individual analysis criteria to provide AI with additional context, such as grant guidelines specific to that criterion.

To add a supporting file to a criterion:

1. Click the pencil icon.

    

2. Click Add to upload documents, then click Save.

    

To view questions on the form included in the AI Application Analysis, click Selected.

Screening Flags

Screening Flags automatically surface applicants that meet specific criteria for closer review. Flags can be categorized as Important or Regular and display alongside the application analysis results. A list can include up to 25 flags.

To add a screening flag:

1. Click Add.

    

2. Configure the flag, then click Save.

    

Supporting Files

Administrators can attach Supporting Files to the analysis list to provide AI with additional context when evaluating applications, such as grant guidelines or funder priorities.

To attach a supporting file to the analysis list:

1. Click Add.

    

2. Add the document, then click Upload.

    

Before running AI Application Analysis at scale, administrators can test an individual criterion, an individual screening flag, or the full criteria list against a real, submitted application. Testing returns the same score, flag, and explanation format as live analysis, so administrators can confirm criteria are producing the expected results before applying them across a process.

Unlike live analysis, testing does not lock the criteria list. Administrators can continue editing criteria and re-test as many times as needed to fine-tune results. Criteria lists that have already been used cannot be tested because using the criteria list locks all of the criteria from edits.

Test an Individual Criterion or Flag

To test an individual criterion or screening flag:

1. Click Tools in the upper navigation bar, and then click Process Manager. 

2. Click the Criteria Lists tab, and then click the criteria list name you want to test.

    

3. Click the beaker icon next to the analysis criteria or screening flag to test.

    

4. Select the Process and the Application within that process from the drop-down lists.

    

5. Click Test.

The result for that criterion or flag displays inline, showing the score, screening flags, and the scoring criteria explanation.

Test the Full Criteria List

To test the full criteria list:

1. Navigate to the Criteria List you want to test.

2. Click Test.

    

3. Select the Process and the Application within that process from the drop-down lists.

    

4. Click Test. 

    

Results display for every screening flag and scoring criterion on the list. The AI Application Summary is not generated during a test, since the summary is not a configurable criterion.

To view the AI Application Analysis results for a submitted application:

1. Click the View Submission icon.

    

2. Click Get Score.

    

Overall Score

An overall score out of 10 is calculated based on the criteria added to the list.

Application Summary

A summary of the application is generated, emphasizing strengths and considerations based on information provided by the applicant as well as criteria added to the list.

Screening Flags

Screening Flags display in the analysis results, identifying which flags were triggered, the reasoning behind each flag, and the area of the application where the flag condition was met. Each flag is labeled as Important or Regular based on the type configured in the criterion.

Scoring Criteria

Each criterion entered in the list displays the applicant's score alongside the type and weight. The AI analysis generates an explanation of the score and cites the area of the application where the information was found.

Administrators can deny, abandon, or mark the application incomplete or complete from the AI Analysis.

Download Analysis Results

To download the Application Analysis results as a .pdf:

1. Click AI Analysis.

    

Administrators can add custom columns to the Application Submitted and Application Complete workload pages to display information about the AI Analysis.

To add custom columns:

1. Click the Edit Columns icon.

    

2. On the Submission tab, click the checkboxes of the columns to add, then click Save Columns.

    

To re-analyze applicants, copy the current analysis list, then select the request statuses to include in the re-analysis.

1. Click the pencil icon.

    

2. Click Copy.

    

3. Configure the analysis list, then click Re-analyze Requests.

    

4. Check the Request Status checkbox, then click Re-analyze Requests.

    

5. Click OK.
   

    

6. Return to the dashboard, then click the workload page of the re-analyzed request.

    

7. Click the View Submission icon.

    

The re-analyzed application displays with the analyzed timestamp updated to the date and time administrators re-analyzed it.

The following examples demonstrate how analysis criteria can be configured in GLM/SLM to score, weight, flag, and evaluate applications based on specific questions and funder priorities.

Weighting Options

Not all criteria carry equal importance. Weighted scoring allows administrators to reflect those priorities by assigning greater influence to the criteria that matter most. The following example shows how a location-based question can be configured with weighted scoring to reflect the organization's geographic priorities for the grant cycle.

Review the "Geographic Area Served" field. If the applicant selected Midwest or West, this is complete alignment with the organization's focus for this grant cycle and should be scored 10/10. If the applicant selected South or Northeast, this is partial alignment and should be scored 5/10. Weight geographic alignment more heavily than other criteria when calculating the overall score.

Focus Area Flag

Not every criterion requires a score. Flagging allows administrators to automatically surface applications that need closer attention based on specific responses, without assigning a numeric value. The following example shows how a focus area question can be configured to automatically flag applications for manual review based on a specific response.

Review the "Focus Area" field. If the applicant selected Social Services, flag this application for manual review and note: "Focus Area: Social Services, requires reviewer attention." No score is needed for this criterion; it is a flag only.

Mission Alignment

AI can evaluate how closely an applicant's mission aligns with the funder's own mission statement, removing the need for reviewers to make that assessment manually for every application. Administrators define what strong alignment looks like and what weak alignment looks like, and the AI scores each application accordingly, freeing reviewers to focus their attention where it matters most. Below are two examples of prompting AI to evaluate an applicant's answers based on mission alignment.

Example 1

Review the "Organization Mission" field. Score alignment with the following funder mission on a scale of 1-10: "Rooted in the wide-open landscapes of the rural West and Northern Plains, we bring together donors, nonprofits, and communities across Montana, Idaho, the Dakotas, and Wyoming to build a stronger, more connected region, one where every small town has the resources to thrive and every resident feels they belong." A score of 10 indicates the applicant's mission directly reflects rural community focus, regional connectivity, or place-based work in MT, ID, ND, SD, or WY. A score of 1 indicates no meaningful alignment.

Example 2

Review the "Organization Mission" field. Score alignment with the following funder mission on a scale of 1-10: "Rooted in the wide-open landscapes of the rural West and Northern Plains, we bring together donors, nonprofits, and communities across Montana, Idaho, the Dakotas, and Wyoming to build a stronger, more connected region, one where every small town has the resources to thrive and every resident feels they belong."

• 9-10: Mission directly reflects rural community focus, regional connectivity, or place-based work in MT, ID, ND, SD, or WY. Language explicitly references small towns, rural residents, or the Northern Plains/rural West region.
• 7-8: Mission is strongly aligned with rural or regional community-building but does not explicitly name the target geography. Themes of belonging, access to resources, or strengthening underserved communities are present.
• 4-6: Mission addresses community wellbeing or nonprofit capacity broadly but lacks a rural or regional lens. Work could apply to urban or national contexts equally.
• 2-3: Mission is tangentially related and may reference community or nonprofit work but with no meaningful rural, regional, or place-based focus.
• 1: Mission shows no meaningful alignment with rural community development, regional connectivity, or the Northern Plains/rural West.

Amount Requested

Requested amount can be an early indicator of potential fit. AI can automatically flag or score applications based on whether the amount requested falls within the organization's established funding range, helping administrators quickly prioritize applications that are realistically aligned with available funding. The following example shows how a requested amount field can be configured to score applications based on alignment with the organization's typical funding range.

Review the "Amount Requested" field and apply the following score: $0.00-$10,000.00 = 10, $10,001.00-$50,000.00 = 5, $50,001.00 or above = 1.