Short answer
A community bank or credit union should run this evaluation in four steps: measure where the hours actually go on a real file, decide whether you are buying analysis or a platform, test every vendor on the same hard borrower rather than on a clean one, and settle the staffing and pricing basis questions before the price. No vendor in this category publishes a price, so the leverage is in scope, references and a written not-to-exceed figure rather than in comparing quotes.
A credit department with three analysts evaluates software under conditions no vendor's process is designed for: nobody is dedicated to the project, the demos happen between committee deadlines, and the person who will administer the product afterwards is also the person underwriting the largest deal in the pipeline. That constraint should shape the evaluation rather than be apologized for. What follows is a sequence built for it, and it deliberately puts the software questions last.
Measure the problem before you look at products
Take three commercial files that took too long and account for the elapsed time honestly. How many days were spent waiting for documents, how many hours identifying and sorting them, how many keying figures into a template, how many on the coverage analysis, how many drafting the memo, and how many waiting for a review or a signature. The distribution is nearly always surprising, and it decides which category of product is even relevant.
This step matters most for a small department because it is the only defence against buying a platform to fix a document problem. It also produces the business case: hours removed from a named workflow, at a known salary cost, is an argument a board can approve. A capability list is not.
- Elapsed days waiting on borrower documents, separated from hours worked
- Hours spent identifying, sorting and matching documents to entities and periods
- Hours keying figures and reconciling between the spread and the memo
- Hours assembling covenant, annual review and examination evidence after the fact
- Hours of genuine credit judgment, which is the part you are protecting
Decide whether this is a platform decision
The two purchases available in this category differ by an order of magnitude in project size. A credit analysis capability can be bought by a credit department and integrated with what exists. A commercial origination platform replaces the workflow, the credit record and often the closing documents, and it is an institutional programme with a project plan and a named owner.
Both are legitimate and the wrong one is expensive. If the origination workflow is broadly tolerable and the calendar is lost to documents, spreads and memos, buy the layer. If the department is running four systems and a shared drive and the file itself keeps getting lost, the platform case is real and should be resourced honestly rather than approved as a software purchase. Ask every vendor to quote the narrow scope and treat the answer as information about the relationship.
The demo file
Use one file for every vendor and make it the file that ruins your week. An operating company on an 1120S, a property entity on a 1065 with four K-1s, two guarantors with 1040s and Schedule E rentals, a nine-month interim statement, and one year that arrived as a photograph of a printout. Send it to each vendor in the same state your borrowers actually send documents, which is to say unsorted and partly illegible.
Then watch four specific things: whether the product identifies each document and assigns it to the right entity and period, whether the combined debt service coverage figure appears without a side spreadsheet, whether policy exceptions are surfaced against your own written policy, and whether you can click a figure in the memo and land on the page it came from. Every vendor looks equivalent on a clean single-entity borrower. This file is where a year of frustration is either avoided or bought.
- Same file, same condition, every vendor, no pre-sorting
- Watch document identification and entity matching, not just extraction accuracy
- Require the combined coverage figure inside the product, not in a spreadsheet afterwards
- Click three figures in the memo and see whether each lands on its source page
- Ask for your own memo template, not the vendor's sample
The staffing question nobody asks
The most common failure at community scale is not capability. It is that the product assumed an internal administrator, a data function or a project manager, and the institution has none of those. That gap does not show up in a demo and it does not show up in a proposal. It shows up eight months later when a policy change needs a rule updated and nobody owns the rule.
Ask every vendor, in named roles rather than hours: who maintains the policy rules when credit policy changes, who owns the integration when the core provider changes a field, who writes the model documentation for anything that extracts or calculates, who administers user access and templates, and what the institution has to staff after go-live. Then ask a named reference institution of your size the same questions separately, because the two answers frequently differ and the difference is the real implementation cost.
Negotiating when nobody publishes a price
No vendor in this category publishes a usable figure, and at least one widely quoted price in circulation has no vendor source behind it. That means comparing quotes is not the exercise. Controlling scope and basis is.
Establish whether the price scales on total assets, commercial balances, users or files, because those four move very differently at a community institution and the choice can double or halve the number. Get a written not-to-exceed figure before committing staff time to a pilot. Ask what happens at renewal, and ask which capabilities in the proposal are generally available today versus announced, since a business case built on undated features is a business case built on nothing enforceable. Where the vendor cannot produce a peer reference at your size, convert that into terms: a paid pilot on your own files, written acceptance criteria, and payment tied to something you can verify yourself.
- Pricing basis agreed before the price: assets, balances, users or files
- A written not-to-exceed figure before a pilot consumes staff time
- Renewal and year-two pricing stated, not left to goodwill
- Availability status per named feature, in writing
- Acceptance criteria you can test on your own files, where references are unavailable
Frequently asked questions
How long should this evaluation take?
Long enough to run the same hard file through every serious candidate and to call at least one peer reference, which in practice means eight to twelve weeks around committee deadlines. The step that gets compressed and should not is the demo file, because that is the only part of the process that tests the vendors rather than their materials.
Is it worth running a paid pilot?
Where a vendor cannot produce a peer reference at your size, a paid pilot is the substitute for that reference and it is worth the money. Write the acceptance criteria yourself, use real files rather than samples, and define what a pass looks like before it starts. A vendor unwilling to be measured on your criteria has answered a different question.
Should we involve the core provider?
For integration facts, yes, and get them from the core rather than from the software vendor. Cores appear on partner pages more readily than connections ship. Ask which named institution is running the integration in production and who owns it when a field changes. Commercial credit analysis is buyable without changing cores, and a core conversation does not belong inside this project.
What if the board wants a name we recognise?
That is a legitimate risk preference rather than an obstacle, and it should be priced. The larger names in this category carry checkable durability and, in two cases, public filings. What they often lack is a documented commercial capability at your size or an availability date on the AI features being sold. Present both columns to the board and let them weigh recognition against evidence explicitly.