This may seem a bit counterintuitive, but this statement could apply to your third-party credit risk review. I once took part as a small crew of three in a portfolio credit loan review for a lender (note the overall items and dollars have been changed in this blog). The institution had deficiencies in their commercial department as outlined by examiners. Our contract included reviews of policy, procedures, people, technology, and loan files. We were to draw up conclusions based upon what we saw, as to the risk identification and management ability of the group. Much of our work was driven by requirements addressed by the officials.
Of all the individual items that needed to be addressed, the biggest one the bank focused on was to have a third-party validation of the credit rating of every relationship over $50,000. The bank was located in a rural community that served small businesses and farmers. It was common for the institution to have quite a few relationships between the $50 - 250K range and the largest relationship was slightly over $3MM. Our contract was to complete the on-site work in one week to limit the cost. This also would limit travel expenses charged to the bank for our little crew. As you may know, the travel expenses for the on-site credit review can be substantial once the bank pays for airline, gas, rental cars, hotel, and meals.
Some third-party firms will transcribe data like financial statements from the files onto their software to complete the analysis. This process is very time consuming and provides little value to the institution. To save time, we skipped this in order to just start writing about our findings. This process allowed us to focus on the credits more and draft conclusions of our findings. The next agreement between the bank and us was to eliminate looking at all relationships between $50K - $150K. It was possible to do a sampling here instead of looking at all of them, but the lender was concerned with a review of as many files as possible. If the bank took a complete loss on all of the relationships below $150K, the aggregate total would be only slightly above the size of their largest relationship.
Even with limiting the number of relationships to review, we were tasked with reviewing around 300 files for this week. The first day of any on-site review is a waste of a detailed look at the files as you deal with logistics, staff interviews, and high-level discussion of the portfolio, policies, and procedures. Day two involved a deep dive of each of us into files, but by the time we had finished the day, we had finished only 9 relationships. This did not give us much progress on the number of files to cover, but it did give us a detailed understanding of how the bank worked.
We were left with 291 relationships divided between three days and three people per day. Doing the math, shows you this divides down into 32 1/3 relationships to review per day per reviewer. Considering we put in a good ten hours of work per day, this averaged 3.23 files per hour per person.
Now if the files were completely in order and all info was easy to find, and if, we had very easy files with only one entity to review and simple amortized payments, perhaps this would be adequate time to render an informed judgement. But if you add in disorganized files, borrowing bases, multiple borrowers/guarantors, missing site inspections, financial information, and other documentation, this greatly complicates the goal of completing a review of all the files.
Note that I said a review of all the files, not “a thoughtful, thorough analysis of the credit risk of each relationship”. This would be the overall goal of a third-party credit review. The purpose is to provide an accurate analysis of the strengths and weaknesses in the bank and not just provide another opinion of the risk in the credit based upon a minimal amount of information reviewed.
This leaves the reviewer and lender with three choices. First, make sure every file is touched, but sacrifice the quality of the review within the week time-frame. Second would be to spend the required time on each loan to do an adequate review. This would take several more weeks of time and expense to the institution. A third choice would be to complete a detailed review of a sample of files that represent the complexity of the various relationships and their management and spend the remainder of the time quickly reviewing the remainder of the files in a summary format on or off site. A final, and the best option, would be to thoroughly review a risk focused sample in collaboration with management.
We settled for option three. Our overall goal was to provide feedback to the institution and where they needed to improve instead of the individual files. The individual files provided mere symptoms that pointed to the overall problem. We ended up with a pretty deep dive into around sixty of the files while doing a cursory review of the remainder. This gave us a pretty good idea of many of the systemic weaknesses that needed to be addressed.
Overall, our small team of three could have either easily spent another week there and completed a pretty deep dive into another 60 or so files. In the end, a detailed look at all 300 relationships would reveal a redundancy of findings which were revealed in the smaller sample. This would have come at much greater cost to the bank while not showing anything new.
But what, you ask, would come if there was a serious risk in a credit which was not part of the deep dive? Of course, that is a risk possible in any review. No matter how good the reviewer is, there is always the possibility that something would be missed. After all, the reviewer is human. A good review provides coachable material that can be used to make you better, not just a file to be put inside a drawer to satisfy the examiners. The review is not to provide you with a 100% guarantee that all deficiencies are adequately identified and provided to the institution.
The problem here is if the primary focus is on looking at every file relationship, and if there are many of them to inspect in a short time, the quality of the work tends to not reveal deep issues with the credit union or bank. The best practice is to not focus on 100%, or even 50%, coverage but to seek a thorough understanding of the depth of the practices, policies, and procedures of credit management. Otherwise, more coverage may leave you exposed.