Domain expertise, combined with data analytics capabilities and packed into user-friendly interfaces: that’s CountryRisk.io. As economists and country risk analysts ourselves, we have come across many different models and tools. And when we founded CountryRisk.io, we developed and maintained further the ones that we find most useful.  In this blog post, we’re sharing some insights around these models and the overall approach we’re implementing regarding data and analytics.

Basic infrastructure first

Before we dive into the various tools and models within our CountryRisk.io universe, we want to stress that our approach benefits crucially from our in-house data platform, CountryData.io. This web-based data platform pools together information like numeric, categorical and text data from various sources, and harmonizes them in a way that we can efficiently work with and have confidence in quality. It is our basic infrastructure and plumbing that we can build upon. You can read more about its development and capabilities here (Link).

Guiding principles

When we develop a new model, we are guided by some basic goal posts:

• Understanding & efficiency: Acquire an understanding of how the world works and the drivers of sovereign and country risk. Increase efficiency through automated monitoring and aggregation of indicators and country narratives (e.g., what are the key strengths and weaknesses based on quantitative indicators or extracting a story from country risk reports).
• It is a tool not a crystal ball. Any model is a support tool with strengths and weaknesses. Never claim that a model can predict the future. The nature of crises is changing over time. Accept that a model is often one step behind but try not to repeat past mistakes.
• Apply the right tool to the right challenge. There is no point of applying very fancy and data-intense modelling approaches to sovereign defaults given the very limited number of observations and explanatory indicator reporting delays can be significant.
• Know the data: An advantage of working with country level data is that they are relatively easier to navigate, explore and monitor compared to corporates. There are only around 200 countries compared to millions of companies – small, medium and large depending on how you want to categorize them – and belonging to a vaster array of industries and sectors. Country-level data are more tractable as a result: their sources; how data are generated; basic data quality aspects like coverage, history, revisions, jumps, etc.; And with these we can visually inspect and get a “feel” for the data.

CountryRisk.io toolset

So, what is in our toolset now? Below we will describe the models that we are currently using and/or are integrated in the CountryRisk.io Insights platform. There are countless more models built for ad-hoc research questions or blog posts; most of those are out of scope for this post.

Sovereign rating model was the first tool that we released in 2016 to enable the interested user to conduct a comprehensive sovereign rating analysis that can be compared in terms of approach to the methodologies applied by S&P and Moody’s, and to a slightly lesser extent, to Fitch’s. CountryRisk.io’s sovereign rating model follows a scoring-based approach where various quantitative and qualitative indicators are assessed following a fixed framework. You can find all the details on our publicly available methodology page (Link).

Risk scores for sovereign credit risk, ESG sovereign risk, AML country risk and supply chain country risk, which are built on the concept of the sovereign rating model. They also apply a scoring-based approach but do not include any qualitative indicators, which need to be assessed or judged by the user. Everyone who has built a scoring-based approach knows that the big question is how to calibrate the model to the model objective (e.g., sovereign default risk or AML country risk). We are the first ones to openly state that the calibration involves a lot of subjective expert judgement. We wrote about the challenges in a recent post (Link). In a nutshell, history is thankfully not rich in sovereign defaults and selecting them also involves some subjective decision-making. Secondly, although calibrating a model towards past sovereign agency ratings may have the benefit of having 5000+ rating observations as base for a quant model, we don’t think that agency ratings are the ultimate benchmark, standard or norm of sovereign risk. And when crossing over to other country risk aspects such as AML, there is simply no objective measure of risk to be used for a statistical model calibration. Details on all the risk score model can be found here (Link).

Early Warning System (EWS) for currency crises tries to estimate the one month ahead probability of a currency crisis (defined either as a 10%, 15% or 20% nominal depreciation against the USD over one month). As we are using monthly FX data, we have much more inputs than for the longer-term risk scores with annual data. We also have a very objective dependent variable, which is fx depreciation. That’s why we can use various statistical models. We apply a Voting Classifier algorithm to get the average prediction of three different models (i.e., a logit model, XGBoost and LSTM neural network). Anyone who has built an EWS knows the challenges: How do you define the pre-crisis periods? How do you deal with repeating crises within a short period of time? How do you report lag of explanatory indicators for a clean performance evaluation, and many more. We value the model output, but we’d be hard-pressed to make decisions solely on the EWS. The model is explained in detail here (Link).

Sentiment model is applied to all text data to see if the sentiment towards a certain topic (e.g., growth prospects) is positive, neutral, or negative. Given the various open-source tools for such tasks, one can implement a sentiment analysis quite quickly. However, one would also realize instantly that the out-of-the-box sentiment text classifier are not very good with domain specific statements of country risk. For example, if a sentence contained the word debt, a standard classifier would very likely assign a negative sentiment to it. In the context of sovereign risk, the sentiment of a sentence containing “debt” comes abundantly from a qualifier like: “unsustainable debt”, “low but gradually rising debt”, “high but rapidly falling debt” and the like. The same goes for many other country risk keywords. As a result, we’ve trained our own classifier based on an in-house labeled dataset of around 4000 sentences. This gives us a much better assessment than the out-of-the-box classifier. And from time to time, we add to our training dataset and update the classifier. The nitty gritty details are described here (Link).   

Large Language Models (LLM) have been a major focus of our R&D efforts over the past months, and we are still in awe by the new possibilities. We currently apply LLMs for two use cases within the CountryRisk.io Insights platform. The first application is in the way to interact with our risks scores. You can post questions to the AI CountryRisk.io Assistant and it will use an augmented version of the risk score data as context (Link). The second is how the AI Assistant can retrieve and summarize information from the various reports available on the platform (Link).

This is clearly just the beginning of LLM within our platform and we have shared some sneak previews in a recent blog post (Link) that explains the upcoming agent based LLM approach that we are currently developing.

Data pre-processing: As so often, a significant part of the effort goes into data pre-processing. This is the case for basically any model, but we would say even more so for LLMs in a retrieval augmented generation (RAG) setup. Here the text that forms the knowledge base often first needs to be made available from various source formats (e.g., PDFs), cleaned and put into smaller chunks to create embeddings. Each of this step would warrant a separate blog post and we are excited about the advancements we see in all those areas (e.g., moving from a token-based splitting to a semantic chunking approach).

While not directly related to LLMs, working with large amounts of text also brings performance challenges for “simple” tasks like keyword tagging. We’ve developed our in-house taxonomy that consists of around 2000 country risk keywords with synonyms and a data pipeline that scales based on the amount of text data that needs to be analyzed daily.  

We hope that this short blog post gave you some insights on how we go about building models and which tools are available to make your life easier that you can focus on making better country risk decisions. Reach out to us if you want to learn more!